1
|
Arslan IG, Verheij RA, Hek K, Ramerman L. Lessons learned from a pay-for-performance scheme for appropriate prescribing using electronic health records from general practices in the Netherlands. Health Policy 2024; 149:105148. [PMID: 39241501 DOI: 10.1016/j.healthpol.2024.105148] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/22/2024] [Revised: 06/20/2024] [Accepted: 08/09/2024] [Indexed: 09/09/2024]
Abstract
INTRODUCTION A nationwide pay-for-performance (P4P) scheme was introduced in the Netherlands between 2018 and 2023 to incentivize appropriate prescribing in general practice. Appropriate prescribing was operationalised as adherence to prescription formularies and measured based on electronic health records (EHR) data. We evaluated this P4P scheme from a learning health systems perspective. METHODS We conducted semi-structured interviews with 15 participants representing stakeholders of the scheme: general practitioners (GPs), health insurers, pharmacists, EHR suppliers and formulary committees. We used a thematic approach for data analysis. RESULTS Using EHR data showed several benefits, but lack of uniformity of EHR systems hindered consistent measurements. Specific indicators were favoured over general indicators as they allow GPs to have more control over their performance. Most participants emphasized the need for GPs to jointly reflect on their performance. Communication to GPs appeared to be challenging. Partly because of these challenges, impact of the scheme on prescribing behaviour was perceived as limited. However, several unexpected positive effects of the scheme were mentioned, such as better EHR recording habits. CONCLUSIONS This study identified benefits and challenges useful for future P4P schemes in promoting appropriate care with EHR data. Enhancing uniformity in EHR systems is crucial for more consistent quality measurements. Future P4P schemes should focus on high-quality feedback, peer-to-peer learning and establish a single point of communication for healthcare providers.
Collapse
Affiliation(s)
- I G Arslan
- Netherlands Institute for Health Services Research (NIVEL), Utrecht, the Netherlands.
| | - R A Verheij
- Netherlands Institute for Health Services Research (NIVEL), Utrecht, the Netherlands; Tilburg School of Social and Behavioral Sciences, Tilburg University, Tilburg, the Netherlands; Health Care Institute Netherlands, Diemen, the Netherlands
| | - K Hek
- Netherlands Institute for Health Services Research (NIVEL), Utrecht, the Netherlands
| | - L Ramerman
- Netherlands Institute for Health Services Research (NIVEL), Utrecht, the Netherlands
| |
Collapse
|
2
|
Ambasta A, Holroyd-Leduc JM, Pokharel S, Mathura P, Shih AWY, Stelfox HT, Ma I, Harrison M, Manns B, Faris P, Williamson T, Shukalek C, Santana M, Omodon O, McCaughey D, Kassam N, Naugler C. Re-Purposing the Ordering of Routine Laboratory Tests in Hospitalized Medical Patients (RePORT): protocol for a multicenter stepped-wedge cluster randomised trial to evaluate the impact of a multicomponent intervention bundle to reduce laboratory test over-utilization. Implement Sci 2024; 19:45. [PMID: 38956637 PMCID: PMC11221016 DOI: 10.1186/s13012-024-01376-6] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/29/2024] [Accepted: 06/23/2024] [Indexed: 07/04/2024] Open
Abstract
BACKGROUND Laboratory test overuse in hospitals is a form of healthcare waste that also harms patients. Developing and evaluating interventions to reduce this form of healthcare waste is critical. We detail the protocol for our study which aims to implement and evaluate the impact of an evidence-based, multicomponent intervention bundle on repetitive use of routine laboratory testing in hospitalized medical patients across adult hospitals in the province of British Columbia, Canada. METHODS We have designed a stepped-wedge cluster randomized trial to assess the impact of a multicomponent intervention bundle across 16 hospitals in the province of British Columbia in Canada. We will use the Knowledge to Action cycle to guide implementation and the RE-AIM framework to guide evaluation of the intervention bundle. The primary outcome will be the number of routine laboratory tests ordered per patient-day in the intervention versus control periods. Secondary outcome measures will assess implementation fidelity, number of all common laboratory tests used, impact on healthcare costs, and safety outcomes. The study will include patients admitted to adult medical wards (internal medicine or family medicine) and healthcare providers working in these wards within the participating hospitals. After a baseline period of 24 weeks, we will conduct a 16-week pilot at one hospital site. A new cluster (containing approximately 2-3 hospitals) will receive the intervention every 12 weeks. We will evaluate the sustainability of implementation at 24 weeks post implementation of the final cluster. Using intention to treat, we will use generalized linear mixed models for analysis to evaluate the impact of the intervention on outcomes. DISCUSSION The study builds upon a multicomponent intervention bundle that has previously demonstrated effectiveness. The elements of the intervention bundle are easily adaptable to other settings, facilitating future adoption in wider contexts. The study outputs are expected to have a positive impact as they will reduce usage of repetitive laboratory tests and provide empirically supported measures and tools for accomplishing this work. TRIAL REGISTRATION This study was prospectively registered on April 8, 2024, via ClinicalTrials.gov Protocols Registration and Results System (NCT06359587). https://classic. CLINICALTRIALS gov/ct2/show/NCT06359587?term=NCT06359587&recrs=ab&draw=2&rank=1.
Collapse
Affiliation(s)
- Anshula Ambasta
- Department of Medicine, Cumming School of Medicine, University of Calgary, 3330 Hospital Dr NW, Calgary, AB, T2N 4N1, Canada.
- Department of Anesthesia, Pharmacology and Therapeutics, Therapeutics Initiative, University of British Columbia, Vancouver, V6T 1Z4, Canada.
| | - Jayna M Holroyd-Leduc
- Department of Medicine, Cumming School of Medicine, University of Calgary, 3330 Hospital Dr NW, Calgary, AB, T2N 4N1, Canada
| | - Surakshya Pokharel
- Department of Medicine, Cumming School of Medicine, University of Calgary, 3330 Hospital Dr NW, Calgary, AB, T2N 4N1, Canada
- Ward of the 21st Century, University of Calgary, GD01, CWPH,Building 3280 Hospital Drive NW, Calgary, AB, T2N 4Z6, Canada
| | - Pamela Mathura
- Department of Medicine, University of Alberta, 116 St & 85 Ave, Edmonton, AB, T6G 2R3, Canada
| | - Andrew Wei-Yeh Shih
- Department of Pathology and Laboratory Medicine, University of British Columbia, Vancouver, V6T 1Z4, Canada
| | - Henry T Stelfox
- Faculty of Medicine and Dentistry, University of Alberta, 116 St & 85 Ave, Edmonton, AB, T6G 2R3, Canada
| | - Irene Ma
- Department of Medicine, Cumming School of Medicine, University of Calgary, 3330 Hospital Dr NW, Calgary, AB, T2N 4N1, Canada
| | - Mark Harrison
- Faculty of Pharmaceutical Sciences, University of British Columbia, Vancouver, V6T 1Z4, Canada
| | - Braden Manns
- Department of Medicine, Cumming School of Medicine, University of Calgary, 3330 Hospital Dr NW, Calgary, AB, T2N 4N1, Canada
- Department of Community Health Sciences, Cumming School of Medicine, University of Calgary, 3330 Hospital Dr NW, Calgary, AB, T2N 4N1, Canada
| | - Peter Faris
- Department of Community Health Sciences, Cumming School of Medicine, University of Calgary, 3330 Hospital Dr NW, Calgary, AB, T2N 4N1, Canada
| | - Tyler Williamson
- Department of Community Health Sciences, Cumming School of Medicine, University of Calgary, 3330 Hospital Dr NW, Calgary, AB, T2N 4N1, Canada
| | - Caley Shukalek
- Department of Medicine, Cumming School of Medicine, University of Calgary, 3330 Hospital Dr NW, Calgary, AB, T2N 4N1, Canada
- Department of Community Health Sciences, Cumming School of Medicine, University of Calgary, 3330 Hospital Dr NW, Calgary, AB, T2N 4N1, Canada
| | - Maria Santana
- Department of Community Health Sciences, Cumming School of Medicine, University of Calgary, 3330 Hospital Dr NW, Calgary, AB, T2N 4N1, Canada
- Department of Pediatrics, Cumming School of Medicine, University of Calgary, 3330 Hospital Dr NW, Calgary, AB, T2N 4N1, Canada
| | - Onyebuchi Omodon
- Department of Medicine, Cumming School of Medicine, University of Calgary, 3330 Hospital Dr NW, Calgary, AB, T2N 4N1, Canada
- Ward of the 21st Century, University of Calgary, GD01, CWPH,Building 3280 Hospital Drive NW, Calgary, AB, T2N 4Z6, Canada
| | - Deirdre McCaughey
- Department of Community Health Sciences, Cumming School of Medicine, University of Calgary, 3330 Hospital Dr NW, Calgary, AB, T2N 4N1, Canada
| | - Narmin Kassam
- Department of Medicine, University of Alberta, 116 St & 85 Ave, Edmonton, AB, T6G 2R3, Canada
| | - Chris Naugler
- Department of Pathology and Laboratory Medicine, Cumming School of Medicine, University of Calgary, 3330 Hospital Dr NW, Calgary, AB, T2N 4N1, Canada
| |
Collapse
|
3
|
Sykes M, Rosenberg-Yunger ZRS, Quigley M, Gupta L, Thomas O, Robinson L, Caulfield K, Ivers N, Alderson S. Exploring the content and delivery of feedback facilitation co-interventions: a systematic review. Implement Sci 2024; 19:37. [PMID: 38807219 PMCID: PMC11134935 DOI: 10.1186/s13012-024-01365-9] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/04/2023] [Accepted: 05/13/2024] [Indexed: 05/30/2024] Open
Abstract
BACKGROUND Policymakers and researchers recommend supporting the capabilities of feedback recipients to increase the quality of care. There are different ways to support capabilities. We aimed to describe the content and delivery of feedback facilitation interventions delivered alongside audit and feedback within randomised controlled trials. METHODS We included papers describing feedback facilitation identified by the latest Cochrane review of audit and feedback. The piloted extraction proforma was based upon a framework to describe intervention content, with additional prompts relating to the identification of influences, selection of improvement actions and consideration of priorities and implications. We describe the content and delivery graphically, statistically and narratively. RESULTS We reviewed 146 papers describing 104 feedback facilitation interventions. Across included studies, feedback facilitation contained 26 different implementation strategies. There was a median of three implementation strategies per intervention and evidence that the number of strategies per intervention is increasing. Theory was used in 35 trials, although the precise role of theory was poorly described. Ten studies provided a logic model and six of these described their mechanisms of action. Both the exploration of influences and the selection of improvement actions were described in 46 of the feedback facilitation interventions; we describe who undertook this tailoring work. Exploring dose, there was large variation in duration (15-1800 min), frequency (1 to 42 times) and number of recipients per site (1 to 135). There were important gaps in reporting, but some evidence that reporting is improving over time. CONCLUSIONS Heterogeneity in the design of feedback facilitation needs to be considered when assessing the intervention's effectiveness. We describe explicit feedback facilitation choices for future intervention developers based upon choices made to date. We found the Expert Recommendations for Implementing Change to be valuable when describing intervention components, with the potential for some minor clarifications in terms and for greater specificity by intervention providers. Reporting demonstrated extensive gaps which hinder both replication and learning. Feedback facilitation providers are recommended to close reporting gaps that hinder replication. Future work should seek to address the 'opportunity' for improvement activity, defined as factors that lie outside the individual that make care or improvement behaviour possible. REVIEW REGISTRATION The study protocol was published at: https://www.protocols.io/private/4DA5DE33B68E11ED9EF70A58A9FEAC02 .
Collapse
Affiliation(s)
| | | | | | | | | | - Lisa Robinson
- Newcastle Upon Tyne NHS Foundation Trust, Newcastle Upon Tyne, UK
| | - Karen Caulfield
- Newcastle Upon Tyne NHS Foundation Trust, Newcastle Upon Tyne, UK
| | | | | |
Collapse
|
4
|
Melville J, Carr T, Goodridge D, Muhajarine N, Groot G. Sepsis screening protocol implementation: a clinician-validated rapid realist review. BMJ Open Qual 2024; 13:e002593. [PMID: 38684345 PMCID: PMC11086359 DOI: 10.1136/bmjoq-2023-002593] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/06/2023] [Accepted: 03/12/2024] [Indexed: 05/02/2024] Open
Abstract
INTRODUCTION The failed or partial implementation of clinical practices negatively impacts patient safety and increases systemic inefficiencies. Implementation of sepsis screening guidelines has been undertaken in many settings with mixed results. Without a theoretical understanding of what leads to successful implementation, improving implementation will continue to be ad hoc or intuitive. This study proposes a programme theory for how and why the successful implementation of sepsis screening guidelines can occur. METHODS A rapid realist review was conducted to develop a focused programme theory for the implementation of sepsis screening guidelines. An independent two-reviewer approach was used to iteratively extract and synthesise context and mechanism data. Theoretical context-mechanism-outcome propositions were refined and validated by clinicians using a focus group and individual realist interviews. Implementation resources and clinical reasoning were differentiated in articulating mechanisms. RESULTS Eighteen articles were included in the rapid review. The theoretical domains framework was identified as the salient substantive theory informing the programme theory. The theory consisted of five main middle-range propositions. Three promoting mechanisms included positive belief about the benefits of the protocol, belief in the legitimacy of using the protocol and trust within the clinical team. Two inhibiting mechanisms included pessimism about the protocol being beneficial and pessimism about the team. Successful implementation was defined as achieving fidelity and sustained use of the intervention. Two intermediate outcomes, acceptability and feasibility of the intervention, and adoption, were necessary to achieve before successful implementation. CONCLUSION This rapid realist review synthesised key information from the literature and clinician feedback to develop a theory-based approach to clinical implementation of sepsis screening. The programme theory presents knowledge users with an outline of how and why clinical interventions lead to successful implementation and could be applied in other clinical areas to improve quality and safety.
Collapse
Affiliation(s)
- Jonathan Melville
- Department of Community Health and Epidemiology, University of Saskatchewan, Saskatoon, Saskatchewan, Canada
| | - Tracey Carr
- Department of Community Health and Epidemiology, University of Saskatchewan, Saskatoon, Saskatchewan, Canada
| | - Donna Goodridge
- College of Medicine, University of Saskatchewan, Saskatoon, Saskatchewan, Canada
| | - Nazeem Muhajarine
- Department of Community Health and Epidemiology, University of Saskatchewan, Saskatoon, Saskatchewan, Canada
| | - Gary Groot
- Department of Community Health and Epidemiology, University of Saskatchewan, Saskatoon, Saskatchewan, Canada
| |
Collapse
|
5
|
Anders A. Reconsidering performance management to support innovative changes in health care services. J Health Organ Manag 2024; 38:125-142. [PMID: 38546186 PMCID: PMC10988776 DOI: 10.1108/jhom-12-2022-0379] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/02/2023] [Revised: 12/15/2023] [Accepted: 02/18/2024] [Indexed: 04/05/2024]
Abstract
PURPOSE A large number of studies indicate that coercive forms of organizational control and performance management in health care services often backfire and initiate dysfunctional consequences. The purpose of this article is to discuss new approaches to performance management in health care services when the purpose is to support innovative changes in the delivery of services. DESIGN/METHODOLOGY/APPROACH The article represents cross-boundary work as the theoretical and empirical material used to discuss and reconsider performance management comes from several relevant research disciplines, including systematic reviews of audit and feedback interventions in health care and extant theories of human motivation and organizational control. FINDINGS An enabling approach to performance management in health care services can potentially contribute to innovative changes. Key design elements to operationalize such an approach are a formative and learning-oriented use of performance measures, an appeal to self- and social-approval mechanisms when providing feedback and support for local goals and action plans that fit specific conditions and challenges. ORIGINALITY/VALUE The article suggests how to operationalize an enabling approach to performance management in health care services. The framework is consistent with new governance and managerial approaches emerging in public sector organizations more generally, supporting a higher degree of professional autonomy and the use of nonfinancial incentives.
Collapse
Affiliation(s)
- Anell Anders
- Department of Business Administration, Lund University
School of Economics and Management, Lund, Sweden
| |
Collapse
|
6
|
Anell A, Arvidsson E, Dackehag M, Ellegård LM, Glenngård AH. Access to automated comparative feedback reports in primary care - a study of intensity of use and relationship with clinical performance among Swedish primary care practices. BMC Health Serv Res 2024; 24:33. [PMID: 38178188 PMCID: PMC10768433 DOI: 10.1186/s12913-023-10407-9] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/16/2023] [Accepted: 11/30/2023] [Indexed: 01/06/2024] Open
Abstract
BACKGROUND Digital applications that automatically extract information from electronic medical records and provide comparative visualizations of the data in the form of quality indicators to primary care practices may facilitate local quality improvement (QI). A necessary condition for such QI to work is that practices actively access the data. The purpose of this study was to explore the use of an application that visualizes quality indicators in Swedish primary care, developed by a profession-led QI initiative ("Primärvårdskvalitet"). We also describe the characteristics of practices that used the application more or less extensively, and the relationships between the intensity of use and changes in selected performance indicators. METHODS We studied longitudinal data on 122 primary care practices' visits to pages (page views) in the application over a period up to 5 years. We compared high and low users, classified by the average number of monthly page views, with respect to practice and patient characteristics as well as baseline measurements of a subset of the performance indicators. We estimated linear associations between visits to pages with diabetes-related indicators and the change in measurements of selected diabetes indicators over 1.5 years. RESULTS Less than half of all practices accessed the data in a given month, although most practices accessed the data during at least one third of the observed months. High and low users were similar in terms of most studied characteristics. We found statistically significant positive associations between use of the diabetes indicators and changes in measurements of three diabetes indicators. CONCLUSIONS Although most practices in this study indicated an interest in the automated feedback reports, the intensity of use can be described as varying and on average limited. The positive associations between the use and changes in performance suggest that policymakers should increase their support of practices' QI efforts. Such support may include providing a formalized structure for peer group discussions of data, facilitating both understanding of the data and possible action points to improve performance, while maintaining a profession-led use of applications.
Collapse
Affiliation(s)
- Anders Anell
- Lund University School of Economics & Management, Lund, Sweden
| | - Eva Arvidsson
- Futurum, Region Jönköping County, Jönköping, Sweden
- School of Health and Welfare, Jönköping University, Jönköping, Sweden
| | | | - Lina Maria Ellegård
- Lund University School of Economics & Management, Lund, Sweden.
- Faculty of Business, Kristianstad University, Kristianstad, Sweden.
| | | |
Collapse
|
7
|
Anell A, Glenngård A. Better with GPs as managers? - Variation in perceptions of feedback messages, goal-clarity and performance across manager´s in Swedish primary care. BMC Health Serv Res 2023; 23:639. [PMID: 37316811 DOI: 10.1186/s12913-023-09586-2] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/20/2023] [Accepted: 05/20/2023] [Indexed: 06/16/2023] Open
Abstract
BACKGROUND Primary care in several countries is developing towards team-based and multi-professional care, requiring leadership and management capabilities at the primary care practice level. This article reports findings from a study of primary care managers in Sweden, focusing variation in performance and perceptions of feedback messages and goal-clarity, depending on managers' professional background. METHODS The study was designed as a cross-sectional analysis of primary care practice managers' perceptions combined with registered data on patient-reported performance. Managers perceptions was collected through a survey to all 1 327 primary care practice managers in Sweden. Data about patient-reported performance was collected from the 2021 National Patient Survey in primary care. We used bivariate (Pearson correlation) and multivariate (ordinary least square regression analysis) statistical methods to describe and analyse the possible association between managers' background, responses to survey statements and patient-reported performance. RESULTS Both GP and non-GP managers had positive perceptions of the quality and support of feedback messages from professional committees focusing medical quality indicators, although managers perceived that the feedback facilitated improvement work to a lower degree. Feedback from the regions as payers scored consistently lower in all dimensions, especially among GP-managers. Results from regression analysis indicate that GP-managers correlate with better patient-reported performance when controlling for selected primary care practice and managerial characteristics. A significant positive relationship with patient-reported performance was also found for female managers, a smaller size of the primary care practice and a good staffing situation of GPs. CONCLUSIONS Both GP and non-GP managers rated the quality and support of feedback messages from professional committees higher than feedback from regions as payers. Differences in perceptions were especially striking among GP-managers. Patient-reported performance was significantly better in primary care practices managed by GPs and female managers. Variables reflecting structural and organizational, rather than managerial, characteristics contributed with additional explanations behind the variation in patient-reported performance across primary care practices. As we cannot exclude reversed causality, the findings may reflect that GPs are more likely to accept being a manager of a primary care practice with favourable characteristics.
Collapse
Affiliation(s)
- Anders Anell
- Department of Business Administration, Lund University School of Economics and Management, Lund, Sweden.
| | - Anna Glenngård
- Department of Business Administration, Lund University School of Economics and Management, Lund, Sweden
| |
Collapse
|
8
|
Porritt K, McArthur A, Lockwood C, Munn Z. JBI's approach to evidence implementation: a 7-phase process model to support and guide getting evidence into practice. JBI Evid Implement 2023; 21:3-13. [PMID: 36545902 DOI: 10.1097/xeb.0000000000000361] [Citation(s) in RCA: 30] [Impact Index Per Article: 30.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/24/2022]
Abstract
ABSTRACT In this paper, we provide an overview of JBI's approach to evidence implementation and describe the supporting process model that aligns with this approach. The central tenets of JBI's approach to implementing evidence into practice include the use of evidence-based audit and feedback, identification of the context in which evidence is being implemented, facilitation of any change, and an evaluation process. A pragmatic and practical seven-phased approach is outlined to assist with the 'planning' and 'doing' of getting evidence into practice, focusing on clinicians as change agents for implementing evidence in clinical and policy settings. Further research and development is required to formally evaluate the robustness of the approach to better understand the complex nature of evidence implementation.
Collapse
Affiliation(s)
- Kylie Porritt
- JBI, Faculty of Health and Medical Sciences, The University of Adelaide, Adelaide, Australia
| | | | | | | |
Collapse
|
9
|
Desveaux L, Rosenberg-Yunger ZRS, Ivers N. You can lead clinicians to water, but you can't make them drink: the role of tailoring in clinical performance feedback to improve care quality. BMJ Qual Saf 2023; 32:76-80. [PMID: 36597998 DOI: 10.1136/bmjqs-2022-015149] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/09/2022] [Accepted: 11/29/2022] [Indexed: 12/14/2022]
Affiliation(s)
- Laura Desveaux
- Institute for Better Health, Trillium Health Partners, Mississauga, Ontario, Canada .,Institute for Health Policy, Management, and Evaluation, University of Toronto Dalla Lana School of Public Health, Toronto, Ontario, Canada
| | - Zahava R S Rosenberg-Yunger
- Institute for Better Health, Trillium Health Partners, Mississauga, Ontario, Canada.,Toronto Metropolitan University, Toronto, Ontario, Canada
| | - Noah Ivers
- Women's College Hospital Institute for Health System Solutions and Virtual Care, Toronto, Ontario, Canada.,Department of Family and Community Medicine, University of Toronto, Toronto, Ontario, Canada
| |
Collapse
|
10
|
van der Winden D, van Dijk N, Visser MRM, Bont J. Walking the line between assessment, improvement and learning: a qualitative study on opportunities and risks of incorporating peer discussion of audit and feedback within quality improvement in general practice. BMJ Open 2023; 13:e066793. [PMID: 36720571 PMCID: PMC9890762 DOI: 10.1136/bmjopen-2022-066793] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 02/02/2023] Open
Abstract
OBJECTIVES There is a broad call for change towards 'new era' quality systems in healthcare, in which the focus lies on learning and improving. A promising way to establish this in general practice care is to combine audit and feedback with peer group discussion. However, it is not known what different stakeholders think of this type of quality improvement. The aim of this research was to explore the opinions of different stakeholders in general practice on peer discussion of audit and feedback and on its opportunities and risks. Second, their thoughts on transparency versus accountability, regarding this system, were studied. DESIGN An exploratory qualitative study within a constructivist paradigm. Semistructured interviews and focus group discussions were held and coded using thematic analysis. Included stakeholders were general practitioners (GP), patients, professional organisations and insurance companies. SETTING General practice in the Netherlands. PARTICIPANTS 22 participants were purposively sampled for eight interviews and two focus group discussions. RESULTS Three main opportunities of peer discussion of audit and feedback were identified: deeper levels of reflection on data, adding context to numbers and more ownership; and three main risks: handling of unwilling colleagues, lacking a safe group and the necessity of patient involvement. An additional theme concerned disagreement on the amount of transparency to be offered: insurance companies and patients advocated for complete transparency on data and improvement of outcomes, while GPs and professional organisations urged to restrict transparency to giving insight into the process. CONCLUSIONS Peer discussion of audit and feedback could be part of a change movement, towards a quality system based on learning and trust, that is initiated by the profession. Creating a safe learning environment and involving patients is key herein. Caution is needed when complete transparency is asked, since it could jeopardise practitioners' reflection and learning in safety.
Collapse
Affiliation(s)
- Dorien van der Winden
- Department of General Practice, Amsterdam UMC Locatie AMC, Amsterdam, Netherlands
- Methodology and Quality of Care, Amsterdam Public Health, Amsterdam, The Netherlands
| | - Nynke van Dijk
- Department of General Practice, Amsterdam UMC Locatie AMC, Amsterdam, Netherlands
- Personalized Medicine and Quality of Care, Amsterdam Public Health, Amsterdam, The Netherlands
- Faculty of Health and Faculty of Sports and Nutrition, Center of Expertise Urban Vitality, Amsterdam University of Applied Sciences, Amsterdam, The Netherlands
| | - Mechteld R M Visser
- Department of General Practice, Amsterdam UMC Locatie AMC, Amsterdam, Netherlands
| | - Jettie Bont
- Department of General Practice, Amsterdam UMC Locatie AMC, Amsterdam, Netherlands
| |
Collapse
|
11
|
Desveaux L, Nguyen MD, Ivers NM, Devotta K, Upshaw T, Ramji N, Weyman K, Kiran T. Snakes and ladders: A qualitative study understanding the active ingredients of social interaction around the use of audit and feedback. Transl Behav Med 2023; 13:316-326. [PMID: 36694357 PMCID: PMC10182419 DOI: 10.1093/tbm/ibac114] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/26/2023] Open
Abstract
Explore characteristics of the facilitator, group, and interaction that influence whether a group discussion about data leads to the identification of a clearly specified action plan. Peer-facilitated group discussions among primary care physicians were carried out and recorded. A follow-up focus group was conducted with peer facilitators to explore which aspects of the discussion promoted action planning. Qualitative data was analyzed using an inductive-deductive thematic analysis approach using the conceptual model developed by Cooke et al. Group discussions were coded case-specifically and then analyzed to identify which themes influenced action planning as it relates to performance improvement. Physicians were more likely to interact with practice-level data and explore actions for performance improvement when the group facilitator focused the discussion on action planning. Only one of the three sites (Site C) converged on an action plan following the peer-facilitated group discussion. At Site A, physicians shared skepticism of the data, were defensive about performance, and explained performance as a product of factors beyond their control. Site B identified several potential actions but had trouble focusing on a single indicator or deciding between physician- and group-level actions. None of the groups discussed variation in physician-level performance indicators, or how physician actions might contribute to the reported outcomes. Peer facilitators can support data interpretation and practice change; however their success depends on their personal beliefs about the data and their ability to identify and leverage change cues that arise in conversation. Further research is needed to understand how to create a psychologically safe environment that welcomes open discussion of physician variation.
Collapse
Affiliation(s)
- Laura Desveaux
- Institute for Better Health, Trillium Health Partners, 100 Queensway West, Mississauga, OntarioCanada.,Women's College Hospital Institute for Health Systems Solutions and Virtual Care, 76 Grenville Ave Toronto, Toronto, Ontario, Canada.,Institute for Health Policy, Management & Evaluation, University of Toronto, 155 College St, Toronto, Ontario, Canada
| | - Marlena Dang Nguyen
- Women's College Hospital Institute for Health Systems Solutions and Virtual Care, 76 Grenville Ave Toronto, Toronto, Ontario, Canada
| | - Noah Michael Ivers
- Women's College Hospital Institute for Health Systems Solutions and Virtual Care, 76 Grenville Ave Toronto, Toronto, Ontario, Canada.,Department of Family and Community Medicine, University of Toronto, 500 University Avenue, Toronto, Canada
| | - Kimberly Devotta
- MAP Centre for Urban Health Solutions, Li Ka Shing Knowledge Institute of St. Michael's Hospital, 30 Bond Street, Toronto, Canada.,Dalla Lana School of Public Health, University of Toronto, 155 College St, Toronto, Canada
| | - Tara Upshaw
- MAP Centre for Urban Health Solutions, Li Ka Shing Knowledge Institute of St. Michael's Hospital, 30 Bond Street, Toronto, Canada.,Dalla Lana School of Public Health, University of Toronto, 155 College St, Toronto, Canada
| | - Noor Ramji
- Department of Family and Community Medicine, University of Toronto, 500 University Avenue, Toronto, Canada.,Department of Family and Community Medicine, St. Michael's Hospital, 30 Bond Street, Toronto, Canada
| | - Karen Weyman
- Department of Family and Community Medicine, University of Toronto, 500 University Avenue, Toronto, Canada.,Department of Family and Community Medicine, St. Michael's Hospital, 30 Bond Street, Toronto, Canada
| | - Tara Kiran
- Institute for Health Policy, Management & Evaluation, University of Toronto, 155 College St, Toronto, Ontario, Canada.,Department of Family and Community Medicine, University of Toronto, 500 University Avenue, Toronto, Canada.,MAP Centre for Urban Health Solutions, Li Ka Shing Knowledge Institute of St. Michael's Hospital, 30 Bond Street, Toronto, Canada.,Department of Family and Community Medicine, St. Michael's Hospital, 30 Bond Street, Toronto, Canada
| |
Collapse
|
12
|
Kushniruk A, Reis C, Ivers N, Desveaux L. Characterizing the Gaps Between Best-Practice Implementation Strategies and Real-world Implementation: Qualitative Study Among Family Physicians Who Engaged With Audit and Feedback Reports. JMIR Hum Factors 2023; 10:e38736. [PMID: 36607715 PMCID: PMC9947922 DOI: 10.2196/38736] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/18/2022] [Revised: 07/28/2022] [Accepted: 11/10/2022] [Indexed: 11/12/2022] Open
Abstract
BACKGROUND In Ontario, Canada, a government agency known as Ontario Health is responsible for making audit and feedback reports available to all family physicians to encourage ongoing quality improvement. The confidential report provides summary data on 3 key areas of practice: safe prescription, cancer screening, and diabetes management. OBJECTIVE This report was redesigned to improve its usability in line with evidence. The objective of this study was to explore how the redesign was perceived, with an emphasis on recipients' understanding of the report and their engagement with it. METHODS We conducted qualitative semistructured interviews with family physicians who had experience with both versions of the report recruited through purposeful and snowball sampling. We analyzed the transcripts following an emergent and iterative approach. RESULTS Saturation was reached after 17 family physicians participated. In total, 2 key themes emerged as factors that affected the perceived usability of the report: alignment between the report and the recipients' expectations and capacity to engage in quality improvement. Family physicians expected the report and its quality indicators to reflect best practices and to be valid and accurate. They also expected the report to offer feedback on the clinical activities they perceived to be within their control to change. Furthermore, family physicians expected the goal of the report to be aligned with their perspective on feasible quality improvement activities. Most of these expectations were not met, limiting the perceived usability of the report. The capacity to engage with audit and feedback was hindered by several organizational and physician-level barriers, including the lack of fit with the existing workflow, competing priorities, time constraints, and insufficient skills for bridging the gaps between their data and the corresponding desired actions. CONCLUSIONS Despite recognized improvements in the design of the report to better align with best practices, it was not perceived as highly usable. Improvements in the presentation of the data could not overcome misalignment with family physicians' expectations or the limited capacity to engage with the report. Integrating iterative evaluations informed by user-centered design can complement evidence-based guidance for implementation strategies. Creating a space for bringing together audit and feedback designers and recipients may help improve usability and effectiveness.
Collapse
Affiliation(s)
| | - Catherine Reis
- Institute for Health System Solutions and Virtual Care, Women's College Hospital, Toronto, ON, Canada
| | - Noah Ivers
- Institute for Health System Solutions and Virtual Care, Women's College Hospital, Toronto, ON, Canada.,Institute of Health Policy, Management and Evaluation, University of Toronto, Toronto, ON, Canada.,Department of Family and Community Medicine, University of Toronto, Toronto, ON, Canada
| | - Laura Desveaux
- Institute for Health System Solutions and Virtual Care, Women's College Hospital, Toronto, ON, Canada.,Institute of Health Policy, Management and Evaluation, University of Toronto, Toronto, ON, Canada.,Institute for Better Health, Trillium Health Partners, Mississauga, ON, Canada
| |
Collapse
|
13
|
Wiljer D, Tavares W, Charow R, Williams S, Campbell C, Davis D, Jeyakumar T, Mylopoulos M, Okrainec A, Silver I, Sockalingam S. A Qualitative Study to Understand the Cultural Factors That Influence Clinical Data Use for Continuing Professional Development. THE JOURNAL OF CONTINUING EDUCATION IN THE HEALTH PROFESSIONS 2023; 43:34-41. [PMID: 35443251 DOI: 10.1097/ceh.0000000000000423] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/14/2023]
Abstract
INTRODUCTION The use of data to inform lifelong learning has become increasingly important in continuing professional development (CPD) practice. Despite the potential benefits of data-driven learning, our understanding of how physicians engage in data-informed learning activities, particularly for CPD, remains unclear and warrants further study. The purpose of this study was to explore how physicians perceive cultural factors (individual, organizational, and systemic) that influence the use of clinical data to inform lifelong learning and self-initiated CPD activities. METHODS This qualitative study is part of an explanatory sequential mixed-methods study examining data-informed learning. Participants were psychiatrists and general surgeons from Canada and the United States. Recruitment occurred between April 2019 and November 2019, and the authors conducted semistructured telephone interviews between May 2019 and November 2019. The authors performed thematic analysis using an iterative, inductive method of constant comparative analysis. RESULTS The authors interviewed 28 physicians: 17 psychiatrists (61%) and 11 general surgeons (39%). Three major themes emerged from the continuous, iterative analysis of interview transcripts: (1) a strong relationship between data and trust, (2) a team-based approach to data-informed learning for practice improvement, and (3) a need for organizational support and advocacy to put data into practice. CONCLUSION Building trust, taking a team-based approach, and engaging multiple stakeholders, such as data specialists and organizational leadership, may significantly improve the use of data-informed learning. The results are situated in the existing literature, and opportunities for future research are summarized.
Collapse
Affiliation(s)
- David Wiljer
- Dr. Wiljer: Executive Director, Education Technology and Innovation, University Health Network, and Professor, Institute of Health Policy, Management and Evaluation, Dalla Lana School of Public Health, and Department of Psychiatry, University of Toronto, Toronto, Ontario, Canada. Dr. Tavares: Scientist, Wilson Centre, University Health Network, and Assistant Professor, Temerty Faculty of Medicine, and Institute of Health Policy, Management and Evaluation, Dalla Lana School of Public Health, University of Toronto, Toronto, Ontario, Canada. Ms. Charow: Research Associate, Education, Technology and Innovation, University Health Network, and PhD Student, Institute of Health Policy, Management and Evaluation, Dalla Lana School of Public Health, University of Toronto, Toronto, Ontario, Canada. Mr. Williams: Research Analyst, Education, Technology and Innovation, University Health Network, Toronto, Ontario, Canada. Dr. Campbell : Director, Curriculum, UGME, and Associate Professor, Faculty of Medicine, University of Ottawa, Ottawa, Ontario, Canada. Dr. Davis: Professor Emeritus, Department of Family and Community Medicine, University of Toronto, and Adjunct Professor, Medical Education, Mohammed Bin Rashid University of Medicine and Health Sciences, Dubai, United Arab Emirites. Ms. Jeyakumar: Education Specialist, Digital Education, University Health Network, Toronto, Ontario, Canada. Dr. Mylopoulos: Scientist and Associate Director of Training Programs, Wilson Centre, University Health Network, and Program Director, Health Professions Education Research, Institute of Health Policy, Management and Evaluation, Dalla Lana School of Public Health, and Associate Professor, Department of Paediatrics, University of Toronto, Toronto, Ontario, Canada. Dr. Okrainec: Head, Division of General Survey, Peter A. Crossgrove Chair in General Surgery and Director, Temerty/Chang Telesimulation Centre, University Health Network, and Associate Professor, Department of Surgery, University of Toronto, Toronto, Ontario, Canada. Dr. Silver : Staff Psychiatrist, Centre for Addiction and Mental Health, and Professor, Department of Psychiatry, University of Toronto, Toronto, Ontario, Canada. Dr. Sockalingam: Vice President of Education and Clinician Scientist, Centre for Addiction and Mental Health, and Professor, Department of Psychiatry, University of Toronto; Wilson Centre Researcher, University Health Network, Toronto, Ontario, Canada
| | | | | | | | | | | | | | | | | | | | | |
Collapse
|
14
|
de Bekker PJGM, de Weerdt V, Vink MDH, van der Kolk AB, Donker MH, van der Hijden EJE. 'Give me something meaningful': GPs perspectives on how to improve an audit and feedback report provided by health insurers - an exploratory qualitative study. BMJ Open Qual 2022; 11:bmjoq-2022-002006. [PMID: 36375859 PMCID: PMC9664288 DOI: 10.1136/bmjoq-2022-002006] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/31/2022] [Accepted: 10/24/2022] [Indexed: 11/16/2022] Open
Abstract
BACKGROUND Audit and feedback (A&F) is a valuable quality improvement strategy, which can contribute to de-implementation of low-value care. In the Netherlands, all health insurers collaboratively provide A&F to general practitioners (GPs), the 'Primary Care Practice Report' (PCPR). Unfortunately, the use of this report by GPs is limited. This study examined the thoughts of GPs on the usability of the PCPR and GPs recommendations for improving the PCPR. METHOD We used an interpretative qualitative design, with think-aloud tasks to uncover thoughts of GPs on the usability of the PCPR and semistructured interview questions to ask GPs' recommendations for improvement of the PCPR. Interviews were audiorecorded and transcribed ad verbatim. Data were analysed using thematic content analysis. RESULTS We identified two main themes: 'poor usability of the PCPR', and 'minimal motivation to change based on the PCPR'. The GPs found the usability of the PCPR poor due to the feedback not being clinically meaningful, the data not being recent, individual and reliable, the performance comparators offer insufficient guidance to assess clinical performance, the results are not discussed with peers and the definitions and visuals are unclear. The GPs recommended improving these issues. The GPs motivation to change based on the PCPR was minimal. CONCLUSIONS The GPs evaluated the PCPR as poorly usable and were minimally motivated to change. The PCPR seems developed from the perspective of the reports' commissioners, health insurers, and does not meet known criteria for effective A&F design and user-centred design. Importantly, the GPs did state that well-designed feedback could contribute to their motivation to improve clinical performance.Furthermore, the GPs stated that they receive a multitude of A&F reports, which they hardly use. Thus, we see a need for policy makers to invest in less, but more usable A&F reports.
Collapse
Affiliation(s)
- P J G M de Bekker
- Department of Health Economics & Talma Institute, Vrije Universiteit Amsterdam, Amsterdam, Netherlands .,Zorgvuldig Advies, Utrecht, Netherlands
| | - V de Weerdt
- Department of Health Economics & Talma Institute, Vrije Universiteit Amsterdam, Amsterdam, Netherlands.,Amsterdam University Medical Centres, Holendrecht, Netherlands
| | - M D H Vink
- Department of Health Economics & Talma Institute, Vrije Universiteit Amsterdam, Amsterdam, Netherlands.,Gynaecology, Amsterdam Universitair Medische Centra, Duivendrecht, Netherlands
| | - A B van der Kolk
- Talma Institute, Vrije Universiteit Amsterdam, Amsterdam, Netherlands
| | - M H Donker
- Department of Health Sciences, Vrije Universiteit Amsterdam, Amsterdam, Netherlands
| | - E J E van der Hijden
- Department of Health Economics & Talma Institute, Vrije Universiteit Amsterdam, Amsterdam, Netherlands.,Zilveren Kruis Health Insurance, Zeist, Netherlands
| |
Collapse
|
15
|
Whitelock-Wainwright E, Koh JW, Whitelock-Wainwright A, Talic S, Rankin D, Gašević D. An exploration into physician and surgeon data sensemaking: a qualitative systematic review using thematic synthesis. BMC Med Inform Decis Mak 2022; 22:256. [PMID: 36171583 PMCID: PMC9520820 DOI: 10.1186/s12911-022-01997-1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/15/2022] [Accepted: 09/12/2022] [Indexed: 11/10/2022] Open
Abstract
Providing electronic health data to medical practitioners to reflect on their performance can lead to improved clinical performance and quality of care. Understanding the sensemaking process that is enacted when practitioners are presented with such data is vital to ensure an improvement in performance. Thus, the primary objective of this research was to explore physician and surgeon sensemaking when presented with electronic health data associated with their clinical performance. A systematic literature review was conducted to analyse qualitative research that explored physicians and surgeons experiences with electronic health data associated with their clinical performance published between January 2010 and March 2022. Included articles were assessed for quality, thematically synthesised, and discussed from the perspective of sensemaking. The initial search strategy for this review returned 8,829 articles that were screened at title and abstract level. Subsequent screening found 11 articles that met the eligibility criteria and were retained for analyses. Two articles met all of the standards within the chosen quality assessment (Standards for Reporting Qualitative Research, SRQR). Thematic synthesis generated five overarching themes: data communication, performance reflection, infrastructure, data quality, and risks. The confidence of such findings is reported using CERQual (Confidence in the Evidence from Reviews of Qualitative research). The way the data is communicated can impact sensemaking which has implications on what is learned and has impact on future performance. Many factors including data accuracy, validity, infrastructure, culture can also impact sensemaking and have ramifications on future practice. Providing data in order to support performance reflection is not without risks, both behavioural and affective. The latter of which can impact the practitioner's ability to effectively make sense of the data. An important consideration when data is presented with the intent to improve performance.Registration This systematic review was registered with Prospero, registration number: CRD42020197392.
Collapse
Affiliation(s)
- Emma Whitelock-Wainwright
- Faculty of Information Technology, Centre for Learning Analytics (CoLAM), Monash University, Melbourne, Australia
- Practice Analytics, DHCRC, Sydney, Australia
| | - Jia Wei Koh
- Faculty of Information Technology, Centre for Learning Analytics (CoLAM), Monash University, Melbourne, Australia
- Practice Analytics, DHCRC, Sydney, Australia
| | | | - Stella Talic
- School of Public Health and Preventative Medicine, Monash University, Melbourne, Australia
- Practice Analytics, DHCRC, Sydney, Australia
| | - David Rankin
- Clinical Governance and Informatics, Cabrini Health, Melbourne, Australia
- Practice Analytics, DHCRC, Sydney, Australia
| | - Dragan Gašević
- Faculty of Information Technology, Centre for Learning Analytics (CoLAM), Monash University, Melbourne, Australia
- Practice Analytics, DHCRC, Sydney, Australia
| |
Collapse
|
16
|
Chan TM, Dowling S, Tastad K, Chin A, Thoma B. Integrating training, practice, and reflection within a new model for Canadian medical licensure: a concept paper prepared for the Medical Council of Canada. CANADIAN MEDICAL EDUCATION JOURNAL 2022; 13:68-81. [PMID: 36091730 PMCID: PMC9441128 DOI: 10.36834/cmej.73717] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Indexed: 06/15/2023]
Abstract
In 2020 the Medical Council of Canada created a task force to make recommendations on the modernization of its practices for granting licensure to medical trainees. This task force solicited papers on this topic from subject matter experts. As outlined within this Concept Paper, our proposal would shift licensure away from the traditional focus on high-stakes summative exams in a way that integrates training, clinical practice, and reflection. Specifically, we propose a model of graduated licensure that would have three stages including: a trainee license for trainees that have demonstrated adequate medical knowledge to begin training as a closely supervised resident, a transition to practice license for trainees that have compiled a reflective educational portfolio demonstrating the clinical competence required to begin independent practice with limitations and support, and a fully independent license for unsupervised practice for attendings that have demonstrated competence through a reflective portfolio of clinical analytics. This proposal was reviewed by a diverse group of 30 trainees, practitioners, and administrators in medical education. Their feedback was analyzed and summarized to provide an overview of the likely reception that this proposal would receive from the medical education community.
Collapse
Affiliation(s)
- Teresa M Chan
- Division of Emergency Medicine, Department of Medicine, Faculty of Health Sciences, McMaster University
- Division of Education & Innovation, Department of Medicine
- Faculty of Health Sciences, McMaster University; McMaster Education Research, Innovation, and Theory (MERIT) program
- Office of Continuing Professional Development; Faculty of Health Sciences, McMaster University
| | - Shawn Dowling
- Department of Emergency Medicine, Cumming School of Medicine, University of Calgary
| | - Kara Tastad
- Royal College Emergency Medicine Training Program, University of Toronto
| | - Alvin Chin
- Division of Emergency Medicine, Department of Medicine, Faculty of Health Sciences, McMaster University
| | - Brent Thoma
- Department of Emergency Medicine, University of Saskatchewan
| |
Collapse
|
17
|
Roy M, Lockyer J, Touchie C. Family Physician Quality Improvement Plans: A Realist Inquiry Into What Works, for Whom, Under What Circumstances. THE JOURNAL OF CONTINUING EDUCATION IN THE HEALTH PROFESSIONS 2022; 43:155-163. [PMID: 37638679 DOI: 10.1097/ceh.0000000000000454] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 08/29/2023]
Abstract
INTRODUCTION Evaluation of quality improvement programs shows variable impact on physician performance often neglecting to examine how implementation varies across contexts and mechanisms that affect uptake. Realist evaluation enables the generation, refinement, and testing theories of change by unpacking what works for whom under what circumstances and why. This study used realist methods to explore relationships between outcomes, mechanisms (resources and reasoning), and context factors of a national multisource feedback (MSF) program. METHODS Linked data for 50 physicians were examined to determine relationships between action plan completion status (outcomes), MSF ratings, MSF comments and prescribing data (resource mechanisms), a report summarizing the conversation between a facilitator and physician (reasoning mechanism), and practice risk factors (context). Working backward from outcomes enabled exploration of similarities and differences in mechanisms and context. RESULTS The derived model showed that the completion status of plans was influenced by interaction of resource and reasoning mechanisms with context mediating the relationships. Two patterns were emerged. Physicians who implemented all their plans within six months received feedback with consistent messaging, reviewed data ahead of facilitation, coconstructed plan(s) with the facilitator, and had fewer risks to competence (dyscompetence). Physicians who were unable to implement any plans had data with fewer repeated messages and did not incorporate these into plans, had difficult plans, or needed to involve others and were physician-led, and were at higher risk for dyscompetence. DISCUSSION Evaluation of quality improvement initiatives should examine program outcomes taking into consideration the interplay of resources, reasoning, and risk factors for dyscompetence.
Collapse
Affiliation(s)
- Marguerite Roy
- Roy: Adjunct Professor, Department of Innovation in Medical Education, University of Ottawa, Ottawa, Canada. Lockyer: Professor, Department of Community Health Sciences, Cumming School of Medicine, University of Calgary, Calgary, Canada. Touchie: Professor of Medicine, Department of Medicine, University of Ottawa and the Ottawa Hospital, Ottawa, Canada, and Department of Innovation in Medical Education, Medical Council of Canada, Ottawa, Canada
| | | | | |
Collapse
|
18
|
Gottlieb M, Chan TM, Zaver F, Ellaway R. Confidence-competence alignment and the role of self-confidence in medical education: A conceptual review. MEDICAL EDUCATION 2022; 56:37-47. [PMID: 34176144 DOI: 10.1111/medu.14592] [Citation(s) in RCA: 24] [Impact Index Per Article: 12.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 02/02/2021] [Revised: 05/20/2021] [Accepted: 06/21/2021] [Indexed: 06/13/2023]
Abstract
CONTEXT There have been significant advances in competency-based medical education (CBME) within health professions education. While most of the efforts have focused on competency, less attention has been paid to the role of confidence as a factor in preparing for practice. This paper seeks to address this deficit by exploring the role of confidence and the calibration of confidence with regard to competence. METHODS This paper presents a conceptual review of confidence and the calibration of confidence in different medical education contexts. Building from an initial literature review, the authors engaged in iterative discussions exploring divergent and convergent perspectives, which were then supplemented with targeted literature reviews. Finally, a stakeholder consultation was conducted to situate and validate the provisional findings. RESULTS A series of axioms were developed to guide perceptions and responses to different states of confidence in health professionals: (a) confidence can shape how we act and is optimised when it closely corresponds to reality; (b) self-confidence is task-specific, but also inextricably influenced by the individual self-conceptualisation, the surrounding system and society; (c) confidence is shaped by many external factors and the context of the situation; (d) confidence must be considered in conjunction with competence and (e) the confidence-competence ratio (CCR) changes over time. It is important to track learners' CCRs and work with them to maintain balance. CONCLUSION Confidence is expressed in different ways and is shaped by a variety of modifiers. While CBME primarily focuses on competency, proportional confidence is an integral component in ensuring safe and professional practice. As such, it is important to consider both confidence and competence, as well as their relationship in CBME. The CCR can serve as a key construct in developing mindful and capable health professionals. Future research should evaluate strategies for assessing CCR, identify best practices for teaching confidence and guiding self-calibration of CCR and explore the role of CCR in continuing professional development for individuals and teams.
Collapse
Affiliation(s)
- Michael Gottlieb
- Department of Emergency Medicine, Rush University Medical Center, Chicago, IL, USA
| | - Teresa M Chan
- Division of Emergency Medicine, Department of Medicine, McMaster University, Hamilton, ON, Canada
| | - Fareen Zaver
- Department of Emergency Medicine, University of Calgary, Calgary, AB, Canada
| | - Rachel Ellaway
- Department of Community Health Sciences and Director of the Office of Health and Medical Education Scholarship, Cumming School of Medicine, University of Calgary, Calgary, AB, Canada
| |
Collapse
|
19
|
Kennedy W, Dowling S, Lonergan K, Rich T, Patocka C. A Realist Evaluation of a 72-Hour Readmission Audit and Feedback (A&F) Intervention in Emergency Medicine. Cureus 2021; 13:e18402. [PMID: 34729279 PMCID: PMC8556763 DOI: 10.7759/cureus.18402] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 09/30/2021] [Indexed: 11/08/2022] Open
Abstract
Introduction Audit and feedback (A&F) interventions are intended to increase accountability and improve the quality of care; however, their impact can vary significantly. As performance feedback is implemented in healthcare, there is a growing need to determine how users interact with the data and how systems can achieve more consistent performance outcomes. This study aimed to understand the contexts, mechanisms, and outcomes of an emergency department 72-hour readmission A&F intervention. Methods Semi-structured interviews with key stakeholders were conducted and analyzed using thematic and template analysis techniques specifically aimed at identifying context, mechanism, and outcome configurations. Results Seventeen (17) physician interviews were conducted. We identified five outcomes of the intervention and the contexts and mechanisms contributing to them. Importantly, we identified that this A&F strategy could potentially have positive (improved follow-up of cases, improved discharge communication) and negative impacts (increased physician anxiety, potentially increased resource use) on physicians and departmental efficiency. Conclusion The 72-hour readmission alert A&F intervention generates a number of distinct outcome patterns that result from a variety of mechanisms acting in different contexts. Knowledge of these context-mechanism-outcome relationships may help implementers design and tailor performance feedback strategies.
Collapse
Affiliation(s)
| | - Shawn Dowling
- Emergency Medicine, University of Calgary, Calgary, CAN
| | - Kevin Lonergan
- Analytics, Data Integration, Measurement and Reporting Program, Alberta Health Services, Calgary, CAN
| | - Tom Rich
- Emergency Medicine, University of Calgary, Calgary, CAN
| | | |
Collapse
|
20
|
Goodrich N, Dolter S, Snow J, Stoolman S, Kerns E, McCulloh R. Quality Improvement of Primary Care Provider Communication on Hospital Discharge. Hosp Pediatr 2021; 11:1050-1056. [PMID: 34531302 DOI: 10.1542/hpeds.2020-004804] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 06/13/2023]
Abstract
BACKGROUND AND OBJECTIVES Effective communication between inpatient and primary care providers (PCPs) is important for safe transition of care for hospitalized patients. In 2017, communication with PCPs was prioritized for the pediatric hospital medicine division. Our primary aim was to improve documented attempted communication with PCPs within 72 hours of discharge from 41% to at least 60% by January 1, 2018, and maintain this performance through 2019. METHODS This study included all inpatient encounters discharged by a pediatric hospital medicine provider from March 2017 to April 2020. An electronic health record phrase debuted March 2017. Successful documentation was defined as any attempt to contact the PCP, regardless of whether actual communication occurred. Group and individual audit and feedback occurred in July 2017 to April 2020. Provider communication was financially incentivized in July 2018 to June 2019. An annotated P-chart for the proportion of encounters with documented PCP communication occurring within 72 hours was established. Special-cause variation was determined by using Shewhart rules. RESULTS The mean proportion of encounters with documented PCP communication increased from 41% at baseline (March 2017 through July 2017) to 60% in August 2017 and 66% in December 2017. After the financial incentive was removed in July 2019, documentation decreased to 54%. Phone calls with clinic staff were the most common communication method (40% to 71%). Direct conversations with the PCP occurred rarely (0% to 3%). CONCLUSIONS Even when coupled with audit and feedback with EHR interventions, our work suggests that shifting to external financial motivation may hinder sustainability of behavior change to improve attempted documented PCP communication.
Collapse
Affiliation(s)
- Nathaniel Goodrich
- Department of Pediatrics, Division of Pediatric Hospital Medicine, University of Nebraska Medical Center, Children's Hospital & Medical Center, Omaha, Nebraska
| | - Stephen Dolter
- Department of Pediatrics, Division of Pediatric Hospital Medicine, University of Nebraska Medical Center, Children's Hospital & Medical Center, Omaha, Nebraska
| | - Joseph Snow
- Department of Pediatrics, Division of Pediatric Hospital Medicine, University of Nebraska Medical Center, Children's Hospital & Medical Center, Omaha, Nebraska
| | - Sharon Stoolman
- Department of Pediatrics, Division of Pediatric Hospital Medicine, University of Nebraska Medical Center, Children's Hospital & Medical Center, Omaha, Nebraska
| | - Ellen Kerns
- Department of Pediatrics, Division of Pediatric Hospital Medicine, University of Nebraska Medical Center, Children's Hospital & Medical Center, Omaha, Nebraska
| | - Russell McCulloh
- Department of Pediatrics, Division of Pediatric Hospital Medicine, University of Nebraska Medical Center, Children's Hospital & Medical Center, Omaha, Nebraska
| |
Collapse
|
21
|
Audit and feedback for individual practitioners in the emergency department: an evidence-based and practical approach. CAN J EMERG MED 2021; 22:528-533. [PMID: 32419681 DOI: 10.1017/cem.2020.28] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/06/2022]
|
22
|
Kamhawy R, Chan TM, Mondoux S. Enabling positive practice improvement through data-driven feedback: A model for understanding how data and self-perception lead to practice change. J Eval Clin Pract 2021; 27:917-925. [PMID: 33124754 DOI: 10.1111/jep.13504] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 06/25/2020] [Revised: 09/28/2020] [Accepted: 10/02/2020] [Indexed: 11/26/2022]
Abstract
PURPOSE This article aims to identify the factors that affect physicians' experiences of receiving practice data and to use these data to develop a model describing how individuals interact with the data. METHODS We designed an interview guide to study physicians' perspectives on audit and feedback. By intentional sampling, we recruited 15 physicians amongst gender groups, types of practice (academic vs community), and durations of practice. The interviews were conducted by a single author and transcribed without identifiers. We then began with an open coding analysis for all of the transcripts, and thereafter conducted axial coding to group the data into larger themes. RESULTS Several attributes were identified as either enabling or counterproductive attributes for participant improvement. The final proposed model identifies different zones of engagement on the basis of both the individual practitioner's growth mindset and the quality of the existing data system. In the highest engagement zone, the mindset of the collective leadership is one of growth. Systemic supports are in place, which potentiates learning that may come from an individual motivated to use their own data. CONCLUSION Our novel model depicts the relationship between data feedback systems and individuals' mindsets interact to augment or hinder clinical practice improvement. This model may provide leaders with a framework to examine their academic and administrative structures and how they might interface with performance feedback systems with clinicians.
Collapse
Affiliation(s)
- Rana Kamhawy
- Faculty of Health Sciences, McMaster University, Hamilton, Ontario, Canada
| | - Teresa M Chan
- Division of Emergency Medicine, Department of Medicine, McMaster University, Hamilton, Ontario, Canada
| | - Shawn Mondoux
- Division of Emergency Medicine, Department of Medicine, McMaster University, Hamilton, Ontario, Canada
| |
Collapse
|
23
|
Thoma B, Caretta-Weyer H, Schumacher DJ, Warm E, Hall AK, Hamstra SJ, Cavalcanti R, Chan TM. Becoming a deliberately developmental organization: Using competency based assessment data for organizational development. MEDICAL TEACHER 2021; 43:801-809. [PMID: 34033512 DOI: 10.1080/0142159x.2021.1925100] [Citation(s) in RCA: 22] [Impact Index Per Article: 7.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/12/2023]
Abstract
Medical education is situated within health care and educational organizations that frequently lag in their use of data to learn, develop, and improve performance. How might we leverage competency-based medical education (CBME) assessment data at the individual, program, and system levels, with the goal of redefining CBME from an initiative that supports the development of physicians to one that also fosters the development of the faculty, administrators, and programs within our organizations? In this paper we review the Deliberately Developmental Organization (DDO) framework proposed by Robert Kegan and Lisa Lahey, a theoretical framework that explains how organizations can foster the development of their people. We then describe the DDO's conceptual alignment with CBME and outline how CBME assessment data could be used to spur the transformation of health care and educational organizations into digitally integrated DDOs. A DDO-oriented use of CBME assessment data will require intentional investment into both the digitalization of assessment data and the development of the people within our organizations. By reframing CBME in this light, we hope that educational and health care leaders will see their investments in CBME as an opportunity to spur the evolution of a developmental culture.
Collapse
Affiliation(s)
- Brent Thoma
- Department of Emergency Medicine, University of Saskatchewan, Saskatoon, Canada
- Royal College of Physicians and Surgeons of Canada, Saskatoon, Canada
| | - Holly Caretta-Weyer
- Department of Emergency Medicine, Stanford University School of Medicine, Palo Alto, CA, USA
| | - Daniel J Schumacher
- Department of Pediatrics, Cincinnati Children's Hospital Medical Center/University of Cincinnati College of Medicine, Cincinnati, OH, USA
| | - Eric Warm
- Department of Internal Medicine, University of Cincinnati College of Medicine, Cincinnati, OH, USA
| | - Andrew K Hall
- Department of Emergency Medicine, University of Saskatchewan, Saskatoon, Canada
- Royal College of Physicians and Surgeons of Canada, Saskatoon, Canada
| | - Stanley J Hamstra
- Milestones Research and Evaluation, Accreditation Council for Graduate Medical Education, Chicago, IL, USA
- Faculty of Education, University of Ottawa, Ottawa, Canada
- Department of Medical Education, Feinberg School of Medicine, Northwestern University, Chicago, IL, USA
| | - Rodrigo Cavalcanti
- Department of Medicine, University of Toronto, Toronto, Canada
- HoPingKong Centre for Excellence in Education and Practice, UHN, Toronto, Canada
| | - Teresa M Chan
- Program for Faculty Development, Faculty of Health Sciences, McMaster University, Hamilton, Canada
- Division of Emergency Medicine, Department of Medicine, McMaster University, Hamilton, Canada
- McMaster program for Education Research, Innovation, and Theory (MERIT), Hamilton, Canada
| |
Collapse
|
24
|
Ramly E, Lauver DR, Gilmore-Bykovskyi A, Bartels CM. Interactive and Participatory Audit and Feedback (IPAF): theory-based development and multi-site implementation outcomes with specialty clinic staff. Implement Sci Commun 2021; 2:58. [PMID: 34059154 PMCID: PMC8167954 DOI: 10.1186/s43058-021-00155-4] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/25/2020] [Accepted: 05/04/2021] [Indexed: 01/14/2023] Open
Abstract
BACKGROUND Theory-based implementation strategies, such as audit and feedback (A&F), can improve the adoption of evidence-based practices. However, few strategies have been developed and tested to meet the needs of specialty clinics. In particular, frontline staff can execute cardiovascular disease (CVD) risk reduction protocols, but A&F strategies to support them are not well examined. Our objective was to develop and evaluate a theory-based approach to A&F, Interactive and Participatory A&F (IPAF). METHODS We developed IPAF informed by two complementary theories, self-regulation theory (SRT) and self-determination theory (SDT). IPAF applies concepts from these theories to inform (1) what to address with staff to improve rates of best practices (SRT) and (2) how to interact with staff to improve behaviors aligned with best practices (SDT). We promoted IPAF fidelity by developing a semi-structured guide to facilitate staff discussion of target behaviors, perceived barriers, goals, and action plans. We evaluated IPAF in the context of eight quasi-experimental implementations in specialty clinics across two health systems. Following a hybrid type 2 effectiveness-implementation design, we reported intervention outcomes for CVD risk reduction elsewhere. This paper reports implementation outcomes associated with IPAF, focusing on feasibility, appropriateness, acceptability, fidelity, and adoption. We evaluated implementation outcomes using mixed-methods data including electronic health record (EHR) data, team records, and staff questionnaire responses. RESULTS Eighteen staff participated in 99 monthly, individual, synchronous (face-to-face or phone) IPAF sessions during the first 6 months of implementation. Subsequently, we provided over 375 monthly feedback emails. Feasibility data revealed high staff attendance (90-93%) and engagement in IPAF sessions. Staff highly rated questionnaire items about IPAF acceptability. Team records and staff responses demonstrated fidelity of IPAF delivery and receipt. Adoption of target behaviors increased significantly (all P values < 0.05), and adoption or behaviors were maintained for over 24 months. CONCLUSIONS We developed and evaluated a theory-based approach to A&F with frontline staff in specialty clinics to improve the implementation of evidence-based interventions. The findings support feasibility, appropriateness, acceptability, and fidelity of IPAF, and staff adoption and maintenance of target behaviors. By evaluating multi-site implementation outcomes, we extended prior research on clinic protocols and A&F beyond primary care settings and providers.
Collapse
Affiliation(s)
- Edmond Ramly
- Department of Family and Community Medicine, School of Medicine & Public Health, University of Wisconsin-Madison, Madison, WI, USA.,Department of Industrial and Systems Engineering, College of Engineering, University of Wisconsin-Madison, Madison, WI, USA
| | - Diane R Lauver
- School of Nursing, University of Wisconsin-Madison, Madison, WI, USA
| | | | - Christie M Bartels
- Department of Medicine, School of Medicine & Public Health, University of Wisconsin-Madison, 1685 Highland Ave, Rm 4132, Madison, WI, 53705-2281, USA.
| |
Collapse
|
25
|
Kandasamy S, Vanstone M, Colvin E, Chan T, Sherbino J, Monteiro S. "I made a mistake!": A narrative analysis of experienced physicians' stories of preventable error. J Eval Clin Pract 2021; 27:236-245. [PMID: 33399266 DOI: 10.1111/jep.13531] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 08/08/2020] [Revised: 12/09/2020] [Accepted: 12/14/2020] [Indexed: 01/01/2023]
Abstract
RATIONALE, AIMS, AND OBJECTIVES The complexity of healthcare systems makes errors unavoidable. To strengthen the dialogue around how physicians experience and share medical errors, the objective of this study was to understand how generalist physicians make meaning of and grow from their medical errors. METHODS This study used a narrative inquiry approach to conduct and analyse in-depth interviews from 26 physicians from the generalist specialties of emergency, internal, and family medicine. We gathered stories via individual interview, analysed them for key components, and rewrote a "meta-story" in a chronological sequence. We conceptualized the findings into a metaphor to draw similarities, learn from, and apply new principles from other fields of practice. RESULTS Through analysis we interpreted the story of a physician who is required to make numerous decisions in a short period of time in different clinical environments among the patient's family and whilst abiding by existing rules and regulations. Through sharing stories of success and failure, the clinical supervisor can help optimize the physician's emotional growth and professional development. Similarly, through sharing and learning from stories, colleagues and trainees can also contribute to the growth of the protagonist's character and the development of clinic, hospital, and healthcare system. CONCLUSION We draw parallels between the clinical setting and a generalist physician's experiences of a medical error with the environment and practices within professional sports. Using this comparison, we discuss the potential for meaningful coaching in medical education.
Collapse
Affiliation(s)
- Sujane Kandasamy
- Department of Health Research Methods, Evidence & Impact, Health Research Methodology PhD Program, McMaster University, Hamilton, Ontario, Canada
| | - Meredith Vanstone
- Department of Family Medicine, McMaster University, Hamilton, Ontario, Canada.,Centre for Health Economics and Policy Analysis, McMaster University, Hamilton, Ontario, Canada.,McMaster Education Research, Innovation & Theory (MERIT) Program, McMaster University, Hamilton, Ontario, Canada
| | - Eamon Colvin
- School of Psychology, Clinical Psychology PhD Program, University of Ottawa, Ottawa, Ontario, Canada
| | - Teresa Chan
- McMaster Education Research, Innovation & Theory (MERIT) Program, McMaster University, Hamilton, Ontario, Canada.,Department of Medicine, Division of Emergency Medicine, McMaster University, Hamilton, Ontario, Canada.,Program for Faculty Development, Faculty of Health Sciences, McMaster University, Hamilton, Ontario, Canada
| | - Jonathan Sherbino
- McMaster Education Research, Innovation & Theory (MERIT) Program, McMaster University, Hamilton, Ontario, Canada.,Department of Medicine, Division of Emergency Medicine, McMaster University, Hamilton, Ontario, Canada
| | - Sandra Monteiro
- Department of Health Research Methods, Evidence & Impact, Health Research Methodology PhD Program, McMaster University, Hamilton, Ontario, Canada.,McMaster Education Research, Innovation & Theory (MERIT) Program, McMaster University, Hamilton, Ontario, Canada
| |
Collapse
|
26
|
Peer Learning Through Multi-Institutional Case Conferences: Abdominal and Cardiothoracic Radiology Experience. Acad Radiol 2021; 28:255-260. [PMID: 32061469 DOI: 10.1016/j.acra.2020.01.015] [Citation(s) in RCA: 15] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/30/2019] [Revised: 12/31/2019] [Accepted: 01/02/2020] [Indexed: 11/23/2022]
Abstract
RATIONALE AND OBJECTIVES We describe a model of multi-institutional, multisociety, online case conferences that is a case-based group discussion of selected (nonrandom) cases which are subsequently hosted on social media and online platforms (e.g., YouTube, websites) to be available for a wider audience. MATERIALS AND METHODS Using online conferencing software (Zoom, GoToMeeting), participants from both abdominal and cardiothoracic radiologists engage in separate, subspecialty one-hour meetings discussing a variety of meaningful cases. Participants take turns presenting their cases to the group and discuss significant findings, interpretations, differential diagnoses, and any other teaching points. All of the case conferences for both societies are recorded and edited to be uploaded on YouTube and their respective websites. RESULTS Participants from these conferences log in from 14 institutions in 7 states across the United States. The YouTube videos reach thousands of people around the world. The abdominal case conference on YouTube has received almost 1,300 views with 90 videos uploaded. The thoracic (the Society of Thoracic Radiology) case conference has been running for over 7 years, with 226 videos uploaded to YouTube and 38,200 views, 1426 subscribers, and a total watch time of over 525,800 minutes. Twitter has been utilized by both groups to promote online viewership. CONCLUSION Our model is feasible and effective compared to traditional peer review. The cases selected are deliberate and focused on quality improvement and/or education. We harness online engagement, specifically social media presence, which has opened new opportunities to educate our peers and reach a global audience, including the nonradiologic community, to learn about radiology and unique practices.
Collapse
|
27
|
Barber CEH, Mosher D, Dowling S, Bohm V, Solbak NM, MacMullan P, Pan B, Barnabe C, Hazlewood GS, Then KL, Marshall DA, Rankin JA, Li LC, Tsui K, English K, Homik J, Spencer N, Hall M, Lacaille D. Implementation and Evaluation of Audit and Feedback for Monitoring Treat-to-Target (T2T) Strategies in Rheumatoid Arthritis Using Performance Measures. Rheumatol Ther 2020; 7:909-925. [PMID: 33034861 PMCID: PMC7695654 DOI: 10.1007/s40744-020-00237-0] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/13/2020] [Accepted: 09/16/2020] [Indexed: 11/11/2022] Open
Abstract
INTRODUCTION In collaboration with the Alberta Medical Association's Physician Learning Program we developed individualized physician reports and held a group feedback session on rheumatoid arthritis (RA) performance measures (PM) to facilitate treat-to-target (T2T) strategies and evaluated physician experiences with this process. METHODS 5 PMs addressing T2T concepts from an established Canadian quality framework were operationalized for physician practice reports at 2 university-affiliated rheumatology clinics. Rheum4U, a quality improvement and research platform, was the data source. The audit results were reviewed in a facilitated group feedback session. Rheumatologists provided experiential feedback on the process through survey and/or an interview. Transcripts from interviews were analyzed using a 6-step thematic analysis. RESULTS 11 of 12 eligible rheumatologists consented to receive practice reports and provided feedback through surveys (n = 5) and interviews (n = 6). The practice reports from Rheum4U (n = 448 patients) revealed high rates of yearly follow-up (> 85%, PM1) and 100% performance on documentation of disease activity at ≥ 50% of visits (PM2). Only 34% of patients were seen within 3 months if not in remission (PM3) with 62% (2017) and 69% (2018) of those with active RA achieving a LDA state within 6 months (PM4). Approximately 70% of patients were in remission at any time point (PM5). All survey respondents agreed or strongly agreed comparison to peers was valuable and helped them reflect on their practice. Several strategies for improvement were identified, including but not limited to, leveraging of electronic records for future audit and feedback reports, providing additional granularity of results, additional stratification of results, and using high-performing peers as the comparator rather than the group mean. CONCLUSIONS Audit and feedback was perceived by clinicians as a useful strategy for evaluating T2T efforts in RA. Future work will focus on longitudinal evaluation of the clinical impact of this quality improvement initiative.
Collapse
Affiliation(s)
- Claire E H Barber
- Department of Medicine, Cumming School of Medicine, University of Calgary, Calgary, AB, Canada.
- Department of Community Health Sciences, Cumming School of Medicine, University of Calgary, Calgary, AB, Canada.
- Arthritis Research Canada, Richmond, BC, Canada.
| | - Dianne Mosher
- Department of Medicine, Cumming School of Medicine, University of Calgary, Calgary, AB, Canada
| | - Shawn Dowling
- Department of Medicine, Cumming School of Medicine, University of Calgary, Calgary, AB, Canada
| | - Victoria Bohm
- Department of Medicine, Cumming School of Medicine, University of Calgary, Calgary, AB, Canada
| | - Nathan M Solbak
- Department of Medicine, Cumming School of Medicine, University of Calgary, Calgary, AB, Canada
| | - Paul MacMullan
- Department of Medicine, Cumming School of Medicine, University of Calgary, Calgary, AB, Canada
| | - Bo Pan
- Epidemiology Coordinating and Research Centre (EPICORE), Edmonton, AB, Canada
| | - Cheryl Barnabe
- Department of Medicine, Cumming School of Medicine, University of Calgary, Calgary, AB, Canada
- Department of Community Health Sciences, Cumming School of Medicine, University of Calgary, Calgary, AB, Canada
- Arthritis Research Canada, Richmond, BC, Canada
| | - Glen S Hazlewood
- Department of Medicine, Cumming School of Medicine, University of Calgary, Calgary, AB, Canada
- Department of Community Health Sciences, Cumming School of Medicine, University of Calgary, Calgary, AB, Canada
- Arthritis Research Canada, Richmond, BC, Canada
| | - Karen L Then
- Faculty of Nursing, University of Calgary, Calgary, AB, Canada
| | - Deborah A Marshall
- Department of Medicine, Cumming School of Medicine, University of Calgary, Calgary, AB, Canada
- Department of Community Health Sciences, Cumming School of Medicine, University of Calgary, Calgary, AB, Canada
- Arthritis Research Canada, Richmond, BC, Canada
| | - James A Rankin
- Faculty of Nursing, University of Calgary, Calgary, AB, Canada
| | - Linda C Li
- Arthritis Research Canada, Richmond, BC, Canada
- Department of Physical Therapy, University of British Columbia, Vancouver, Canada
| | - Karen Tsui
- Arthritis Patient Advisory Board, Richmond, BC, Canada
| | - Kelly English
- Arthritis Patient Advisory Board, Richmond, BC, Canada
| | - Joanne Homik
- Department of Medicine, University of Alberta, Edmonton, AB, Canada
| | - Nicole Spencer
- Department of Medicine, Cumming School of Medicine, University of Calgary, Calgary, AB, Canada
| | - Marc Hall
- Faculty of Nursing, University of Calgary, Calgary, AB, Canada
| | - Diane Lacaille
- Arthritis Research Canada, Richmond, BC, Canada
- Department of Medicine, University of British Columbia, Vancouver, Canada
| |
Collapse
|
28
|
Yilmaz Y, Lal S, Tong XC, Howard M, Bal S, Bayer I, Monteiro S, Chan TM. Technology-Enhanced Faculty Development: Future Trends and Possibilities for Health Sciences Education. MEDICAL SCIENCE EDUCATOR 2020; 30:1787-1796. [PMID: 33078084 PMCID: PMC7556766 DOI: 10.1007/s40670-020-01100-1] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Accepted: 10/01/2020] [Indexed: 05/25/2023]
Affiliation(s)
- Yusuf Yilmaz
- McMaster University Faculty of Health Sciences McMaster Education Research, Innovation and Theory (MERIT) Program, Hamilton, Ontario Canada
- Department of Medical Education, Faculty of Medicine, Ege University, Izmir, Turkey
| | - Sarrah Lal
- McMaster University Faculty of Health Sciences McMaster Education Research, Innovation and Theory (MERIT) Program, Hamilton, Ontario Canada
- MacPherson Leadership in Teaching and Learning Fellowship, McMaster University, Hamilton, Ontarion Canada
- Michael G. DeGroote Initiative for Innovation in Healthcare and Division of Education & Innovation, Department of Medicine, McMaster University, Hamilton, Ontario Canada
| | - X. Catherine Tong
- Department of Family Medicine, McMaster University, Hamilton, Ontario Canada
| | - Michelle Howard
- Department of Family Medicine, McMaster University, Hamilton, Ontario Canada
| | - Sharon Bal
- Department of Family Medicine, McMaster University, Hamilton, Ontario Canada
| | - Ilana Bayer
- MacPherson Leadership in Teaching and Learning Fellowship, McMaster University, Hamilton, Ontarion Canada
- Department of Pathology and Molecular Medicine, Faculty of Health Sciences Learning Technologies Lab, McMaster University, Hamilton, Ontario Canada
| | - Sandra Monteiro
- McMaster University Faculty of Health Sciences McMaster Education Research, Innovation and Theory (MERIT) Program, Hamilton, Ontario Canada
- Department of Health Research Methods, Evidence and Impact, McMaster University, Hamilton, Ontario Canada
| | - Teresa M. Chan
- McMaster University Faculty of Health Sciences McMaster Education Research, Innovation and Theory (MERIT) Program, Hamilton, Ontario Canada
- Program for Faculty Development, Faculty of Health Sciences, McMaster University, Hamilton, Ontario Canada
- Division of Emergency Medicine, Department of Medicine, McMaster University, Hamilton, Ontario Canada
| |
Collapse
|
29
|
Armstrong V, Tan N, Sekhar A, Richardson ML, Kanne JP, Sai V, Chernyak V, Godwin JD, Tammisetti VS, Eberhardt SC, Henry TS. Peer Learning Through Multi-Institutional Web-based Case Conferences: Perceived Value (and Challenges) From Abdominal, Cardiothoracic, and Musculoskeletal Radiology Case Conference Participants. Acad Radiol 2020; 27:1641-1646. [PMID: 31848074 DOI: 10.1016/j.acra.2019.11.009] [Citation(s) in RCA: 9] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/17/2019] [Revised: 11/02/2019] [Accepted: 11/12/2019] [Indexed: 11/19/2022]
Abstract
RATIONALE AND OBJECTIVES Peer learning is a case-based group-learning model intended to improve performance. In this descriptive paper, we describe multi-institutional, multi-subspecialty, web-based radiology case conferences and summarize the participants' experiences. MATERIALS AND METHODS A semi-structured, 27-question survey was administered to radiologists participating in abdominal, cardiothoracic, and musculoskeletal case conferences. Survey questions included demographics, perceived educational value and challenges experienced. Survey question formats were continuous, binary, five-point Likert scale or text-based. The measures of central tendencies, proportions of responses and patterns were tabulated. RESULTS From 57 responders, 12/57 (21.1%) were abdominal, 16/57 (28.1%) were cardiothoracic, and 29/57 (50.8%) were musculoskeletal conference participants; 50/56 (89.3%) represented academic practice. Median age was 45 years (range 35-74); 43/57 (75.4%) were male. Geographically, 16/52 (30.8%) of participants were from the East Coast, 16/52 (30.8%) Midwest, 18/52 (34.6%) West Coast, and 2/52 (3.8%) International. The median reported educational value was 5/5 (interquartile range 5-5). Benefits of the case conference included education (50/95, 52.6%) and networking (39/95, 41.1%). Participants reported presenting the following cases: "great call" 32/48 (66.7%), learning opportunity 32/48 (66.7%), new knowledge 41/49 (83.7%), "zebras" 46/49 (93.9%), and procedural-based 16/46 (34.8%). All 51/51 (100%) of responders reportedly gained new knowledge, 49/51 (96.1%) became more open to group discussion, 34/51 (66.7%) changed search patterns, and 50/51 (98%) would continue to participate. Reported challenges included time zone differences and support from departments for a protected time to participate. CONCLUSION Peer learning through multi-institutional case conferences provides educational and networking opportunities. Current challenges and desires include having department-supported protected time and ability to receive continuing medical education credit.
Collapse
Affiliation(s)
| | - Nelly Tan
- Loma Linda University Medical Center, Loma Linda, California.
| | - Aarti Sekhar
- Emory University School of Medicine, Atlanta, Georgia
| | | | | | | | | | | | | | | | | |
Collapse
|
30
|
Larson DB, Broder JC, Bhargavan-Chatfield M, Donnelly LF, Kadom N, Khorasani R, Sharpe RE, Pahade JK, Moriarity AK, Tan N, Siewert B, Kruskal JB. Transitioning From Peer Review to Peer Learning: Report of the 2020 Peer Learning Summit. J Am Coll Radiol 2020; 17:1499-1508. [DOI: 10.1016/j.jacr.2020.07.016] [Citation(s) in RCA: 11] [Impact Index Per Article: 2.8] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/10/2020] [Revised: 07/05/2020] [Accepted: 07/15/2020] [Indexed: 10/23/2022]
|
31
|
Liao M, Tan N. Collective Intelligence of Peer Learning: Promoting Culture of Learning and Improvement Among Radiologists. Curr Probl Diagn Radiol 2020; 50:761-763. [PMID: 33032854 DOI: 10.1067/j.cpradiol.2020.09.017] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/30/2020] [Revised: 08/31/2020] [Accepted: 09/15/2020] [Indexed: 11/22/2022]
Abstract
The traditional Scoring-Based Peer Review system has been the predominant radiology performance quality assurance model, which can become a condemning, ineffective process. In 2015, the Institute of Medicine called for "policies and practices that promote a non-punitive culture that values open discussion and feedback on diagnostic performance." The development of Peer Learning (PL), a process that encompasses peer feedback, learning, and improvement, has positively impacted radiology through the recognition of success, identification of mistakes as learning opportunities, and development of a professional culture of trust. Furthermore, collective intelligence advances the PL process within the learning community, optimizing the abilities of a group effort that outperform that of a single individual, especially in the setting of complex medical and diagnostic imaging decision-making. The objective of the review article is to highlight the collective intelligence aspect of PL program, which allows PL to be more effective than established peer review model.
Collapse
Affiliation(s)
- Millie Liao
- Department of Radiology, Loma Linda University Medical Center, Loma Linda, CA
| | - Nelly Tan
- Department of Radiology, Mayo Clinic - Arizona, Phoenix, AZ.
| |
Collapse
|
32
|
Campbell C, Lockyer J. Categorising and enhancing the impacts of continuing professional development to improve performance and health outcomes. MEDICAL EDUCATION 2019; 53:1066-1069. [PMID: 31650584 DOI: 10.1111/medu.13995] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/10/2023]
Affiliation(s)
- Craig Campbell
- Principal Senior Advisor, Competency-based Continuing Professional Development, Office of Specialty Education, Royal College of Physicians and Surgeons of Canada, Ottawa, Ontario, Canada
| | - Jocelyn Lockyer
- Community Health Sciences, Cumming School of Medicine, University of Calgary, Calgary, Alberta, Canada
| |
Collapse
|
33
|
MacKenzie C, Chan TM, Mondoux S. Clinical Improvement Interventions for Residents and Practicing Physicians: A Scoping Review of Coaching and Mentoring for Practice Improvement. AEM EDUCATION AND TRAINING 2019; 3:353-364. [PMID: 31637353 PMCID: PMC6795351 DOI: 10.1002/aet2.10345] [Citation(s) in RCA: 9] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 01/15/2019] [Revised: 03/12/2019] [Accepted: 03/18/2019] [Indexed: 05/25/2023]
Abstract
BACKGROUND Graduate medical education (GME) bodies are beginning to mandate coaching as an integral part of the learning process, in addition to current requirements for mentorship. Once an emergency medicine physician transitions beyond graduate training, there is no requirement and little focus on coaching as a method of improving or maintaining clinical practice. Our objective was to understand and describe the current state of the published literature with regard to the use of coaching and mentorship for both GME and practicing physicians. METHODS We conducted a structured review of the literature through PubMed and Google Scholar and included all articles applying coaching or mentorship modalities to GME trainees or practicing physicians. A Google Form was used for standardized data abstraction. Data were collected pertaining to the settings of intervention, the nature of the intervention, its effect, and its resource requirements. RESULTS A total of 3,546 papers were isolated during the literature review. After exclusion, 186 underwent full-text review by the authors of which 126 articles were included in the final data analysis. Eighty-two articles (65%) pertained to mentorship and 14 (11%) to coaching; the remainder of the articles discussed a combination or variation of these two concepts. Fifty-three (42%) articles were descriptive studies and 35 (28%) were narrative reviews or commentaries. Forty-seven (37%) articles originated from within surgical specialties and coaching was most commonly applied to procedural or manual skills with 22 (17%) instances among all studies. CONCLUSIONS Most literature on coaching and mentorship is descriptive or narrative, and few papers are in the specialty of emergency medicine. Most interventions are limited to single instances of coaching or mentorship without longitudinal application of the intervention. There is an important need to study and publish further evidence on coaching interventions.
Collapse
Affiliation(s)
- Casey MacKenzie
- Michael G. DeGroote School of MedicineDepartment of MedicineMcMaster UniversityHamiltonOntario
| | - Teresa M. Chan
- Division of Emergency MedicineDepartment of MedicineMcMaster UniversityHamiltonOntario
- McMaster program for Education Research, Innovation, and Theory (MERIT)HamiltonOntario
| | - Shawn Mondoux
- Institute of Health Policy, Management and Evaluation (IHPME)University of TorontoTorontoOntarioCanada
| |
Collapse
|
34
|
van Braak M, Visser M, Holtrop M, Statius Muller I, Bont J, van Dijk N. What motivates general practitioners to change practice behaviour? A qualitative study of audit and feedback group sessions in Dutch general practice. BMJ Open 2019; 9:e025286. [PMID: 31154299 PMCID: PMC6549704 DOI: 10.1136/bmjopen-2018-025286] [Citation(s) in RCA: 13] [Impact Index Per Article: 2.6] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 11/12/2022] Open
Abstract
OBJECTIVES Adopting an attributional perspective, the current article investigates how audit and feedback group sessions contribute to general practitioners' (GPs) motivation to change their practice behaviour to improve care. We focus on the contributions of the audit and feedback itself (content) and the group discussion (process). METHODS Four focus groups, comprising a total of 39 participating Dutch GPs, discussed and compared audit and feedback of their practices. The focus groups were analysed thematically. RESULTS Audit and feedback contributed to GPs' motivation to change in two ways: by raising awareness about aspects of their current care practice and by providing indications of the possible impact of change. For these contributions to play out, the audit and feedback should be reliable and valid, specific, recent and recurrent and concern GPs' own practices or practices within their own influence sphere. Care behaviour attributed to external, uncontrollable or unstable causes would not induce change. The added value of the group is twofold as well: group discussion contributed to GPs' motivation to change by providing a frame of reference and by affording insights that participants would not have been able to achieve on their own. CONCLUSIONS In audit and feedback group sessions, both audit and feedback information and group discussion can valuably contribute to GPs' motivation to change care practice behaviour. Peer interaction can positively contribute to explore alternative practices and avenues for improvement. Local or regional peer meetings would be beneficial in facilitating reflection and discussion. An important avenue for future studies is to explore the contribution of audit and feedback and small-group discussion to actual practice change.
Collapse
Affiliation(s)
- Marije van Braak
- General Practice, Erasmus Medical Centre, Rotterdam, The Netherlands
| | - Mechteld Visser
- General Practice, Amsterdam University Medical Centre, Amsterdam, The Netherlands
| | - Marije Holtrop
- General Practitioners Holtrop and Sieben, Amsterdam, The Netherlands
| | | | - Jettie Bont
- General Practice, Amsterdam University Medical Centre, Amsterdam, The Netherlands
| | - Nynke van Dijk
- General Practice, Amsterdam University Medical Centre, Amsterdam, The Netherlands
| |
Collapse
|
35
|
Gotlib Conn L, Hoeft C, Neal M, Nathens A. Use of performance reports among trauma medical directors and programme managers in the American College of Surgeons' Trauma Quality Improvement Program: a qualitative analysis. BMJ Qual Saf 2019; 28:721-728. [PMID: 30923180 DOI: 10.1136/bmjqs-2018-008797] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.6] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/06/2018] [Revised: 02/27/2019] [Accepted: 03/11/2019] [Indexed: 11/04/2022]
Abstract
BACKGROUND The American College of Surgeons' Trauma Quality Improvement Program (TQIP) provides trauma centres with performance reports on their processes and outcomes of care relative to their peers. This study explored how performance reports are used by trauma centre leaders to engage in performance improvement and perceived barriers to use. STUDY DESIGN Qualitative focus group study with trauma medical directors (TMDs) and trauma programme managers (TPMs) in US trauma centres. Consistent with qualitative descriptive analysis, data collection and interpretation were inductively and iteratively completed. Major themes were derived using a constant comparative technique. RESULTS Six focus groups were conducted involving 22 TMDs and 22 TPMs. Three major themes were captured: (1) technical uses of performance reports; (2) cultural uses of performance reports; (3) opportunities to enhance the role and value of TQIP. First, technical uses included using reports to assess data collection procedures, data quality and areas of poor performance relative to peers. In this domain, barriers to report use included not trusting others' data quality and challenges with report interpretation. Second, reports were used to influence practice change by fostering inter-specialty discussions, leveraging resources for quality improvement, community engagement and regional collaboratives. Perceived lack of specialist engagement was viewed as an impediment in this domain. Lastly, identified opportunities for TQIP to support report use involved clarifying the relationship between verification and performance reports, and increasing partnerships with nursing associations. CONCLUSION Trauma centre improvement leaders indicated practical and social uses of performance reports that can affect intention and ability to change. Recommendations to optimise programme participation include a focus on data quality, adequate resource provision and enhanced support for regional collaboratives.
Collapse
Affiliation(s)
- Lesley Gotlib Conn
- Evaluative Clinical Sciences, Sunnybrook Research Institute, Toronto, Ontario, Canada
| | - Christopher Hoeft
- Trauma Quality Improvement Program, American College of Surgeons, Chicago, IL, USA
| | - Melanie Neal
- Trauma Quality Improvement Program, American College of Surgeons, Chicago, IL, USA
| | - Avery Nathens
- Evaluative Clinical Sciences, Sunnybrook Research Institute, Toronto, Ontario, Canada
| |
Collapse
|
36
|
Sockalingam S, Tavares W, Charow R, Youssef A, Campbell C, Davis D, Giuliani M, Okrainec A, Papadakos J, Silver I, Wiljer D. Examining Associations Between Physician Data Utilization for Practice Improvement and Lifelong Learning. THE JOURNAL OF CONTINUING EDUCATION IN THE HEALTH PROFESSIONS 2019; 39:236-242. [PMID: 31688154 DOI: 10.1097/ceh.0000000000000268] [Citation(s) in RCA: 9] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/23/2023]
Abstract
INTRODUCTION Practice data can inform the selection of educational strategies; however, it is not widely used, even when available. This study's purpose was to determine factors that influence physician engagement with practice data to advance competence and drive practice change. METHODS A practice-based, pan-Canadian survey was administered to three physician subspecialties: psychiatrists (Psy), radiation oncologists (RO), and general surgeons (GS). The survey was distributed through national specialty society membership lists. The survey assessed factors that influence the use of data for practice improvement and orientation to lifelong learning, using the Jefferson Scale of Physician Lifelong Learning (JeffSPLL). Linear regression was used to model the relationship between the outcome variable frequency of data use and independent predictors of continuous learning to improving practice. RESULTS A total of 305 practicing physicians (Psy = 203, RO = 53, GS = 49) participated in this study. Most respondents used data for practice improvement (n = 177, 61.7%; Psy = 115, 40.1%; RO = 35; 12.2%; GS = 27, 9.4%) and had high orientation to lifelong learning (JeffSPLL mean scores: Psy = 47.4; RO = 43.5; GS = 45.1; Max = 56). Linear regression analysis identified significant predictors of data use in practice being: frequency of assessing learning needs, helpfulness of data to improve practice, and frequency to develop learning plans. Together, these predictors explained 42.9% of the variance in physicians' orientation toward integrating accessible data into practice (R = 0.426, P < .001). DISCUSSION This study demonstrates an association between practice data use and perceived data utility, reflection on learning needs and learning plan development. Implications for this work include process development for data-informed action planning for practice improvement for physicians.
Collapse
Affiliation(s)
- Sanjeev Sockalingam
- Dr. Sockalingam: Vice President of Education, Center for Addiction and Mental Health, Associate Professor, Department of Psychiatry, University of Toronto, and Staff Psychiatrist, Center of Mental Health & Wilson Center Researcher, University Health Network, Toronto, Ontario Canada. Dr. Tavares: Scientist, Wilson Center, University Health Network, Paramedic Services, Community and Health Services, Regional Municipality of York, and Assistant Professor, Post MD Education, Faculty of Medicine, Institute of Health Policy, Management and Evaluation and Dalla Lana School of Public Health, University of Toronto, Toronto, Ontario, Canada. Ms. Charow: Research Associate, Education Technology and Innovation, and Cancer Health Literacy Research Center, University Health Network, Toronto, Ontario, Canada. Ms. Youssef: PhD Candidate, Institute of Medical Sciences and Department of Psychiatry, University of Toronto, and Research Fellow, Wilson Center, University Health Network, Toronto, Ontario, Canada. Dr. Campbell: Principal Senior Advisor, Competency-based Continuing Professional Development, Office of Specialty Education, Royal College of Physicians and Surgeons of Canada, and Associate Professor, Faculty of Medicine, University of Ottawa, Ottawa, Ontario, Canada. Dr. Davis: Professor Emeritus, Department of Family and Community Medicine, University of Toronto, Toronto, Ontario, Canada, and Senior Director-CORE & Visiting Professor, Mohammed Bin Rashid University of Medicine and Health Sciences, Dubai Healthcare City, United Arab Emirates. Dr. Giuliani: Director, Cancer Education Program, Co-Director, Cancer Health Literacy Research Center and Staff Radiation Oncologist, Princess Margaret Cancer Center, and Assistant Professor, Department of Radiation Oncology, Faculty of Medicine, University of Toronto, Toronto, Ontario, Canada. Dr. Okrainec: Head, Division of General Survey, Peter A. Crossgrove Chair in General Surgery and Director, Temerty/Chang Telesimulation Center, University Health Network, and Associate Professor, Department of Surgery, University of Toronto, Toronto, Ontario, Canada. Dr. Papadakos: Senior Manager, Cancer Education Program, Co-Director, Cancer Health Literacy Research Center and Associate Director, ELLICSR: Health, Wellness & Cancer Survivorship Center, Princess Margaret Cancer Center, University Health Network; Provincial Head, Patient Education, Cancer Care Ontario, Toronto, Ontario, Canada. Dr. Silver: Staff Psychiatrist, Center for Addiction and Mental Health, and Professor, Department of Psychiatry, University of Toronto, Toronto, Ontario Canada. Dr. Wiljer: Executive Director, Education Technology and Innovation, University Health Network, Associate Professor, Institute of Health Policy, Management and Evaluation and Department of Psychiatry, University of Toronto, Toronto, Ontario, Canada
| | | | | | | | | | | | | | | | | | | | | |
Collapse
|
37
|
Cooke LJ, Duncan D, Rivera L, Dowling SK, Symonds C, Armson H. The Calgary Audit and Feedback Framework: a practical, evidence-informed approach for the design and implementation of socially constructed learning interventions using audit and group feedback. Implement Sci 2018; 13:136. [PMID: 30376848 PMCID: PMC6208022 DOI: 10.1186/s13012-018-0829-3] [Citation(s) in RCA: 35] [Impact Index Per Article: 5.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/07/2018] [Accepted: 10/18/2018] [Indexed: 11/10/2022] Open
Abstract
Background Audit and feedback interventions may be strengthened using social interaction. The Calgary office of the Alberta Physician Learning Program (CPLP) developed a process for audit and group feedback for physicians. This paper extends previous work in which we developed a conceptual model of physician responses to audit and group feedback based on a qualitative analysis of six audit and group feedback sessions. The present study explored the mediating factors for successfully engaging physician groups in change planning through audit and group feedback. Methods To understand why some groups were more interactive than others, we completed a comparative case analysis of the six audit and group feedback projects from the prior study. We used framework analysis to build the case studies, triangulated our observations across data sources to validate findings, compared the case studies for similarities and differences that influenced social interaction (mediating factors), and thematically categorized mediating factors into an organizing framework. Results Mediating factors for socially interactive AGFS were a pre-existing relationship between the program team and the physician group, projects addressing important, actionable questions, easily interpretable data visualization in the reports, and facilitation of the groups that included reflective questioning. When these factors were in place (cases 1, 2A, 3), the audit and group feedback sessions were dynamic, with physicians sharing and comparing practices, and raising change cues (such as declaring commitments to de-prescribing, planning educational interventions, and improving documentation). In cases 2C–D, the mediating factors were less well established and in these cases, the sessions showed little physician reflection or change planning. We organized the mediating factors into a framework linking the factors for successful sessions to the conceptual model of physician behaviors which these mediating factors drive. Conclusions We propose the Calgary Audit and Feedback Framework as a practical tool to help foster socially constructed learning in audit and group feedback sessions. Ensuring that the four factors, relationship, question choice, data visualization, and facilitation, are considered for design and implementation of audit and group feedback will help physicians move from reactions to their data towards planning for change.
Collapse
Affiliation(s)
- Lara J Cooke
- Department of Clinical Neurosciences, Cumming School of Medicine, University of Calgary, UCMC Area 3, 3350 Hospital Drive NW, Calgary, AB, T2N 4N1, Canada.
| | - Diane Duncan
- Cumming School of Medicine, University of Calgary, 3330 Hospital Dr NW, Calgary, AB, T2N 4N1, Canada
| | - Laura Rivera
- Cumming School of Medicine, University of Calgary, 3330 Hospital Dr NW, Calgary, AB, T2N 4N1, Canada
| | - Shawn K Dowling
- Physician Learning Program, Cumming School of Medicine, University of Calgary, HSC G302, 3330 Hospital Dr. NW, Calgary, AB, T2N 4N1, Canada
| | - Christopher Symonds
- Division of Endocrinology & Metabolism, Cumming School of Medicine, University of Calgary, 3330 Hospital Dr NW, Calgary, AB, T2N 4N1, Canada
| | - Heather Armson
- Department of Family Medicine, Cumming School of Medicine, University of Calgary, 3330 Hospital Dr NW, Calgary, AB, T2N 4N1, Canada
| |
Collapse
|