1
|
Mendoza J, Hampton E, Singleton L. A theoretical and practical approach to quality improvement education. Curr Probl Pediatr Adolesc Health Care 2023; 53:101459. [PMID: 37980237 DOI: 10.1016/j.cppeds.2023.101459] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 11/20/2023]
Abstract
Quality Improvement (QI) knowledge and skills are required at all levels of physician training. System improvement efforts need to include understanding of health disparities and design of interventions to reduce those disparities, thus health equity needs to be integrated into QI education. Payors, accreditation bodies and health systems' emphases on QI result in the need for QI curricula that meet the needs of diverse learners. This article presents a theoretical background and practical tools for designing, implementing, and evaluating a QI educational program across the spectrum of physician training with an emphasis on competency-based education and a goal of continuous practice improvement. Practice-based learning and improvement and systems-based practice are two core domains of competencies for readiness to practice. These competencies can be met through the health systems science framework for studying improvement in patient care and health care delivery coupled with QI science. Curricula should incorporate interactive learning of theory and principles of QI as well as mentored, experiential QI project work with multidisciplinary teams. QI projects often develop ideas and implement changes but are often inconsistent in studying intervention impacts or reaching the level of patient outcomes. Curriculum design should incorporate adult learning principles, competency based medical education, environmental and audience factors, and formats for content delivery. Key QI topics and how they fit into the clinical environment and teaching resources are provided, as well as options for faculty development. Approaches to evaluation are presented, along with tools for assessing learner's beliefs and attitudes, knowledge and application of QI principles, project evaluation, competency and curriculum evaluation. If the goal is to empower the next generation of change agents, there remains a need for development of scientific methodology and scholarly work, as well as faculty development and support by institutions.
Collapse
Affiliation(s)
- Joanne Mendoza
- Department of Pediatrics, Eastern Virginia Medical School, Children's Hospital of The King's Daughters, Virginia, USA.
| | - Elisa Hampton
- Department of Pediatrics, University of Virginia School of Medicine, University of Virginia Children's, Virginia, USA
| | - Lori Singleton
- Department of Pediatrics, Morehouse School of Medicine, Children's Healthcare of Atlanta, Georgia, USA
| |
Collapse
|
2
|
MacEachern L, Ginsburg LR, Hoben M, Doupe M, Wagg A, Knopp-Sihota JA, Cranley L, Song Y, Estabrooks CA, Berta W. Developing a tool to measure enactment of complex quality improvement interventions in healthcare. BMJ Open Qual 2023; 12:bmjoq-2022-002027. [PMID: 36754540 PMCID: PMC9923287 DOI: 10.1136/bmjoq-2022-002027] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/27/2022] [Accepted: 01/24/2023] [Indexed: 02/10/2023] Open
Abstract
Quality improvement (QI) projects are common in healthcare settings and often involve interdisciplinary teams working together towards a common goal. Many interventions and programmes have been introduced through research to convey QI skills and knowledge to healthcare workers, however, a few studies have attempted to differentiate between what individuals 'learn' or 'know' versus their capacity to apply their learnings in complex healthcare settings. Understanding and differentiating between delivery, receipt, and enactment of QI skills and knowledge is important because while enactment alone does not guarantee desired QI outcomes, it might be reasonably assumed that 'better enactment' is likely to lead to better outcomes. This paper describes the development, application and validation of a tool to measure enactment of core QI skills and knowledge of a complex QI intervention in a healthcare setting. Based on the Institute for Healthcare Improvement's Model for Improvement, existing QI assessment tools, literature on enactment fidelity and our research protocols, 10 indicators related to core QI skills and knowledge were determined. Definitions and assessment criteria were tested and refined in five iterative cycles. Qualitative data from four QI teams in long-term care homes were used to test and validate the tool. The final measurement tool contains 10 QI indicators and a five-point scale. Inter-rater reliability ranged from good to excellent. Usability and acceptability among raters were considered high. This measurement tool assists in identifying strengths and weaknesses of a QI team and allows for targeted feedback on core QI components. The indicators developed in our tool and the approach to tool development may be useful in other health related contexts where similar data are collected.
Collapse
Affiliation(s)
- Lauren MacEachern
- Institute for Health Policy, Management and Evaluation, University of Toronto, Toronto, Ontario, Canada
| | - Liane R Ginsburg
- Health Policy & Management, York University, Toronto, Ontario, Canada
| | - Matthias Hoben
- School of Health Policy and Management, York University, Toronto, Ontario, Canada,Faculty of Nursing, University of Alberta, Edmonton, Alberta, Canada
| | - Malcolm Doupe
- Rady Faculty of Health Sciences, University of Manitoba, Winnipeg, Manitoba, Canada,Centre for Care Research, Western Norway University of Applied Sciences, Bergen, Norway
| | - Adrian Wagg
- Department of Medicine, University of Alberta, Edmonton, Alberta, Canada
| | | | - Lisa Cranley
- Lawrence S Bloomberg Faculty of Nursing, University of Toronto, Toronto, Ontario, Canada
| | - Yuting Song
- Faculty of Nursing, University of Alberta, Edmonton, Alberta, Canada,School of Nursing, Qingdao University, Edmonton, Shandong, China
| | | | - Whitney Berta
- Institute for Health Policy, Management and Evaluation, University of Toronto, Toronto, Ontario, Canada
| |
Collapse
|
3
|
Abraham C, Johnson-Martinez K, Tomolo A. A Scoring Rubric for the Knowledge Section of the Systems Quality Improvement Training and Assessment Tool. MEDEDPORTAL : THE JOURNAL OF TEACHING AND LEARNING RESOURCES 2022; 18:11290. [PMID: 36605542 PMCID: PMC9744987 DOI: 10.15766/mep_2374-8265.11290] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 03/21/2022] [Accepted: 09/27/2022] [Indexed: 06/17/2023]
Abstract
INTRODUCTION Quality improvement (QI) competencies for health professions trainees were developed to address health care quality. Strategies to integrate QI into curricula exist, but methods for assessing interdisciplinary learners' competency are less developed. We refined the Knowledge section scoring rubric of the Systems Quality Improvement Training and Assessment Tool (SQI TAT) and examined its validity evidence. METHODS In 2017, the SQI TAT Knowledge section was expanded to cover seven core QI concepts, and the scoring rubric was refined. Three coders independently scored 35 SQI TAT Knowledge sections (18 pretests, 17 posttests). Interrater reliability was assessed by percent agreement and Cohen's kappa for individual variables and by Lin's concordance correlation for total scores for knowledge and application. Concurrent validity was assessed by comparing responses from two groups with different QI exposure and evaluating whether differences in exposure were measured. RESULTS Total-score interrater reliability average measures of concordance were .89 for all coders and >.70 for six of seven concept scores. The total score discriminated the two groups (p <. 05), and five of seven concept scores were higher for the group with more QI experience. Total scores were significantly higher posttest than pretest (p < .001), with improvement in posttest knowledge scores. DISCUSSION The SQI TAT Knowledge section provides a comprehensive assessment of QI knowledge. The scoring rubric was able to discriminate QI knowledge along a continuum. The SQI TAT Knowledge section is not linked to a clinical context, making it useful for assessing interprofessional learners and varying education levels.
Collapse
Affiliation(s)
- Corrine Abraham
- Associate Professor, Nell Hodgson Woodruff School of Nursing at Emory University; Coordinator, Evidence-Based Practice and Innovation, and Co-Director, VA Quality Scholars Fellowship Program, Atlanta VA Health Care System
| | - Krysta Johnson-Martinez
- Specialty Care Lead and Chief Medical Informatics Officer, VISN 8 VA Sunshine Healthcare Network
| | - Anne Tomolo
- Physician, National Center for Patient Safety; Associate Professor, Emory University School of Medicine
| |
Collapse
|
4
|
Marks BE, Mungmode A, Neyman A, Levin L, Rioles N, Eng D, Lee JM, Basina M, Hawah-Jones N, Mann E, O’Malley G, Wilkes M, Steenkamp D, Aleppo G, Accacha S, Ebekozien O. Baseline Quality Improvement Capacity of 33 Endocrinology Centers Participating in the T1D Exchange Quality Improvement Collaborative. Clin Diabetes 2022; 41:35-44. [PMID: 36714248 PMCID: PMC9845085 DOI: 10.2337/cd22-0071] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 02/01/2023]
Abstract
This article describes the evolution of the Type 1 Diabetes Exchange Quality Improvement Collaborative (T1DX-QI) and provides insight into the development and growth of a successful type 1 diabetes quality improvement (QI) program. Since its inception 8 years ago, the collaborative has expanded to include centers across the United States with varying levels of QI experience, while simultaneously achieving many tangible improvements in type 1 diabetes care. These successes underscore the importance of learning health systems, data-sharing, benchmarking, and peer collaboration as drivers for continuous QI. Future efforts will include recruiting additional small- to medium-sized centers focused on adult care and underserved communities to further the goal of improving care and outcomes for all people living with type 1 diabetes.
Collapse
Affiliation(s)
- Brynn E. Marks
- Children’s National Hospital, Washington, DC
- Children’s Hospital of Philadelphia, Philadelphia, PA
- Corresponding author: Brynn E. Marks,
| | | | - Anna Neyman
- Riley Children’s Hospital, Indiana University School of Medicine, Indianapolis, IN
| | - Laura Levin
- Ann and Robert H. Lurie Children Hospital, Chicago, IL
| | | | - Donna Eng
- Helen DeVos Children’s Hospital, Grand Rapids, MI
| | - Joyce M. Lee
- C.S. Mott Children’s Hospital, University of Michigan, Ann Arbor, MI
| | | | | | - Elizabeth Mann
- UW Health Kids, University of Wisconsin School of Medicine and Public Health, Madison, WI
| | | | | | | | | | | | - Osagie Ebekozien
- T1D Exchange, Boston, MA
- University of Mississippi School of Population Health, Jackson, MS
| |
Collapse
|
5
|
Implementation and Evaluation of Quality Improvement Training in Surgery: A Systematic Review. Ann Surg 2021; 274:e489-e506. [PMID: 34784666 DOI: 10.1097/sla.0000000000004751] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/16/2022]
Abstract
OBJECTIVE The aim of this study was to review and appraise how quality improvement (QI) skills are taught to surgeons and surgical residents. BACKGROUND There is a global drive to deliver capacity in undertaking QI within surgical services. However, there are currently no specifications regarding optimal QI content or delivery. METHODS We reviewed QI educational intervention studies targeting surgeons or surgical trainees/residents published until 2017. Primary outcomes included teaching methods and training materials. Secondary outcomes were implementation frameworks and strategies used to deliver QI training successfully. RESULTS There were 20,590 hits across 10 databases, of which 11,563 were screened following de-duplication. Seventeen studies were included in the final synthesis. Variable QI techniques (eg, combined QI models, process mapping, and "lean" principles) and assessment methods were found. Delivery was more consistent, typically combining didactic teaching blended with QI project delivery. Implementation of QI training was poorly reported and appears supported by collaborative approaches (including building learning collaboratives, and coalitions). Study designs were typically pre-/post-training without controls. Studies generally lacked clarity on the underpinning framework (59%), setting description (59%), content (47%), and conclusions (47%), whereas 88% scored low on psychometrics reporting. CONCLUSIONS The evidence suggests that surgical QI training can focus on any well-established QI technique, provided it is done through a combination of didactic teaching and practical application. True effectiveness and extent of impact of QI training remain unclear, due to methodological weaknesses and inconsistent reporting. Conduct of larger-scale educational QI studies across multiple institutions can advance the field.
Collapse
|
6
|
Brown A, Lafreniere K, Freedman D, Nidumolu A, Mancuso M, Hecker K, Kassam A. A realist synthesis of quality improvement curricula in undergraduate and postgraduate medical education: what works, for whom, and in what contexts? BMJ Qual Saf 2020; 30:337-352. [PMID: 33023936 DOI: 10.1136/bmjqs-2020-010887] [Citation(s) in RCA: 26] [Impact Index Per Article: 6.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/15/2020] [Revised: 08/11/2020] [Accepted: 08/29/2020] [Indexed: 11/03/2022]
Abstract
BACKGROUND With the integration of quality improvement (QI) into competency-based models of physician training, there is an increasing requirement for medical students and residents to demonstrate competence in QI. There may be factors that commonly facilitate or inhibit the desired outcomes of QI curricula in undergraduate and postgraduate medical education. The purpose of this review was to synthesise attributes of QI curricula in undergraduate and postgraduate medical education associated with curricular outcomes. METHODS A realist synthesis of peer-reviewed and grey literature was conducted to identify the common contexts, mechanisms, and outcomes of QI curricula in undergraduate and postgraduate medical education in order to develop a programme theory to articulate what works, for whom, and in what contexts. RESULTS 18854 records underwent title and abstract screening, full texts of 609 records were appraised for eligibility, data were extracted from 358 studies, and 218 studies were included in the development and refinement of the final programme theory. Contexts included curricular strategies, levels of training, clinical settings, and organisational culture. Mechanisms were identified within the overall QI curricula itself (eg, clear expectations and deliverables, and protected time), in the didactic components (ie, content delivery strategies), and within the experiential components (eg, topic selection strategies, working with others, and mentorship). Mechanisms were often associated with certain contexts to promote educational and clinical outcomes. CONCLUSION This research describes the various pedagogical strategies for teaching QI to medical learners and highlights the contexts and mechanisms that could potentially account for differences in educational and clinical outcomes of QI curricula. Educators may benefit from considering these contexts and mechanisms in the design and implementation of QI curricula to optimise the outcomes of training in this competency area.
Collapse
Affiliation(s)
- Allison Brown
- Department of Medicine, University of Calgary Cumming School of Medicine, Calgary, Alberta, Canada .,Department of Community Health Sciences, University of Calgary Cumming School of Medicine, Calgary, Alberta, Canada
| | - Kyle Lafreniere
- Department of Obstetrics and Gynecology, University of Calgary Cumming School of Medicine, Calgary, Alberta, Canada
| | - David Freedman
- Department of Psychiatry, University of Toronto Faculty of Medicine, Toronto, Ontario, Canada
| | - Aditya Nidumolu
- Department of Psychiatry, Dalhousie University Faculty of Medicine, Halifax, Nova Scotia, Canada
| | - Matthew Mancuso
- Undergraduate Medical Education, University of Alberta Faculty of Medicine and Dentistry, Edmonton, Alberta, Canada
| | - Kent Hecker
- Department of Community Health Sciences, University of Calgary Cumming School of Medicine, Calgary, Alberta, Canada.,Faculty of Veterinary Medicine, University of Calgary, Calgary, Alberta, Canada
| | - Aliya Kassam
- Department of Community Health Sciences, University of Calgary Cumming School of Medicine, Calgary, Alberta, Canada.,Department of Postgraduate Medical Education, University of Calgary Cumming School of Medicine, Calgary, Alberta, Canada
| |
Collapse
|
7
|
Brown A, Nidumolu A, McConnell M, Hecker K, Grierson L. Development and psychometric evaluation of an instrument to measure knowledge, skills, and attitudes towards quality improvement in health professions education: The Beliefs, Attitudes, Skills, and Confidence in Quality Improvement (BASiC-QI) Scale. PERSPECTIVES ON MEDICAL EDUCATION 2019; 8:167-176. [PMID: 31098982 PMCID: PMC6565662 DOI: 10.1007/s40037-019-0511-8] [Citation(s) in RCA: 11] [Impact Index Per Article: 2.2] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Indexed: 06/09/2023]
Abstract
INTRODUCTION Health professionals are increasingly expected to foster and lead initiatives to improve the quality and safety of healthcare. Consequently, health professions education has begun to integrate formal quality improvement (QI) training into their curricula. Few instruments exist in the literature that adequately and reliably assess QI-related competencies in learners without the use of multiple, trained raters in the context of healthcare. This paper describes the development and psychometric evaluation of the Beliefs, Attitudes, Skills, and Confidence in Quality Improvement (BASiC-QI) instrument, a 30-item self-assessment tool designed to assess knowledge, skills, and attitudes towards QI. METHODS Sixty first-year medical student participants completed the BASiC-QI and the Quality Improvement Knowledge Application Tool (QIKAT-R) prior to and immediately following a QI program that challenged learners to engage QI concepts in the context of their own medical education. Measurement properties of the BASiC-QI tool were explored through an exploratory factor analysis and generalizability study. Convergent validity was examined through correlations between BASiC-QI and QIKAT-R scores. RESULTS Psychometric evaluation of BASiC-QI indicated reliability and validity evidence based on internal structure. Analyses also revealed that BASiC-QI scores were positively correlated with the scores from the QIKAT-R, which stands an indicator of convergent validity. CONCLUSION BASiC-QI is a multidimensional self-assessment tool that may be used to assess beliefs, attitudes, skills, and confidence towards QI. In comparison with existing instruments, BASiC-QI does not require multiple raters or scoring rubrics, serving as an efficient, reliable assessment instrument for educators to examine the impact of QI curricula on learners.
Collapse
Affiliation(s)
- Allison Brown
- Michael G. DeGroote School of Medicine, McMaster University, Hamilton, ON, Canada.
- Department of Health Research Methods, Evidence, and Impact, McMaster University, Hamilton, ON, Canada.
| | - Aditya Nidumolu
- Michael G. DeGroote School of Medicine, McMaster University, Hamilton, ON, Canada
| | - Meghan McConnell
- Department of Innovation in Medical Education, University of Ottawa, Ottawa, ON, Canada
| | - Kent Hecker
- Department of Community Health Sciences, University of Calgary, Calgary, AB, Canada
| | - Lawrence Grierson
- Michael G. DeGroote School of Medicine, McMaster University, Hamilton, ON, Canada
- Department of Health Research Methods, Evidence, and Impact, McMaster University, Hamilton, ON, Canada
- Department of Family Medicine, McMaster University, Hamilton, ON, Canada
- McMaster Education Research, Innovation & Theory, McMaster University, Hamilton, ON, Canada
| |
Collapse
|
8
|
Kassardjian C, Park YS, Braksick S, Cutsforth-Gregory J, Robertson C, Young N, Leep Hunderfund A. Validity evidence for Quality Improvement Knowledge Application Tool Revised (QIKAT-R) scores: consequences of rater number and type using neurology cases. BMJ Qual Saf 2019; 28:925-933. [PMID: 30996038 DOI: 10.1136/bmjqs-2018-008689] [Citation(s) in RCA: 4] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/07/2018] [Revised: 02/05/2019] [Accepted: 03/31/2019] [Indexed: 11/03/2022]
Abstract
OBJECTIVES To develop neurology scenarios for use with the Quality Improvement Knowledge Application Tool Revised (QIKAT-R), gather and evaluate validity evidence, and project the impact of scenario number, rater number and rater type on score reliability. METHODS Six neurological case scenarios were developed. Residents were randomly assigned three scenarios before and after a quality improvement (QI) course in 2015 and 2016. For each scenario, residents crafted an aim statement, selected a measure and proposed a change to address a quality gap. Responses were scored by six faculty raters (two with and four without QI expertise) using the QIKAT-R. Validity evidence from content, response process, internal structure, relations to other variables and consequences was collected. A generalisability (G) study examined sources of score variability, and decision analyses estimated projected reliability for different numbers of raters and scenarios and raters with and without QI expertise. RESULTS Raters scored 163 responses from 28 residents. The mean QIKAT-R score was 5.69 (SD 1.06). G-coefficient and Phi-coefficient were 0.65 and 0.60, respectively. Interrater reliability was fair for raters without QI expertise (intraclass correlation = 0.53, 95% CI 0.30 to 0.72) and acceptable for raters with QI expertise (intraclass correlation = 0.66, 95% CI 0.02 to 0.88). Postcourse scores were significantly higher than precourse scores (6.05, SD 1.48 vs 5.22, SD 1.5; p < 0.001). Sufficient reliability for formative assessment (G-coefficient > 0.60) could be achieved by three raters scoring six scenarios or two raters scoring eight scenarios, regardless of rater QI expertise. CONCLUSIONS Validity evidence was sufficient to support the use of the QIKAT-R with multiple scenarios and raters to assess resident QI knowledge application for formative or low-stakes summative purposes. The results provide practical information for educators to guide implementation decisions.
Collapse
Affiliation(s)
| | - Yoon Soo Park
- Department of Medical Education, University of Illinois at Chicago College of Medicine, Chicago, Illinois, USA
| | - Sherri Braksick
- Department of Neurology, University of Kansas Medical Center, Kansas City, Kansas, USA
| | | | | | - Nathan Young
- Department of Neurology, Mayo Clinic, Rochester, Minnesota, USA.,Department of Neurology, Mayo Clinic Department of Health Sciences Research, Rochester, Minnesota, USA
| | | |
Collapse
|
9
|
Lee-Park JJ, Ha RD, Kablinger A. Teaching the Concepts of Quality Improvement for Challenging Patient Populations. Am J Med Qual 2018; 33:670. [DOI: 10.1177/1062860618769384] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/16/2022]
|
10
|
The State of Quality Improvement Training in Pediatric Emergency Medicine Programs: A National Survey of Fellowship Program Directors. CLINICAL PEDIATRIC EMERGENCY MEDICINE 2017. [DOI: 10.1016/j.cpem.2017.05.001] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/20/2022]
|