1
|
Haase CB, Bearman M, Brodersen JB, Risor T, Hoeyer K. Data driven or data informed? How general practitioners use data to evaluate their own and colleagues' clinical work in clusters. SOCIOLOGY OF HEALTH & ILLNESS 2024; 46:948-965. [PMID: 38156947 DOI: 10.1111/1467-9566.13743] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 03/01/2023] [Accepted: 11/24/2023] [Indexed: 01/03/2024]
Abstract
In contemporary policy discourses, data are presented as key assets for improving health-care quality: policymakers want health care to become 'data driven'. In this article, we focus on a particular example of this ambition, namely a new Danish national quality development program for general practitioners (GPs) where doctors are placed in so-called 'clusters'. In these clusters, GPs are obliged to assess their own and colleagues' clinical quality with data derived from their own clinics-using comparisons, averages and benchmarks. Based on semi-structured interviews with Danish GPs and drawing on Science and Technology Studies, we explore how GPs understand these data, and what makes them trust-or question-a data analysis. The GPs describe how they change clinical practices based on these discussions of data. So, when and how do data for quality assurance come to influence their perceptions of quality? By exploring these issues, we carve out a role for a sociological engagement with evidence in everyday medical practices. In conclusion, we suggest a need to move from the aim of being data driven to one of being data informed.
Collapse
Affiliation(s)
- Christoffer Bjerre Haase
- Department of Public Health, Section for Health Services Research, University of Copenhagen, Copenhagen, Denmark
- Centre for Research in Assessment and Digital Learning (CRADLE), Deakin University, Melbourne, Victoria, Australia
- Department of Public Health, Centre of General Practice, University of Copenhagen, Copenhagen, Denmark
| | - Margaret Bearman
- Centre for Research in Assessment and Digital Learning (CRADLE), Deakin University, Melbourne, Victoria, Australia
| | - John Brandt Brodersen
- Department of Public Health, Centre of General Practice, University of Copenhagen, Copenhagen, Denmark
- Research Unit for General Practice, Region Zealand, Denmark
- Research Unit for General Practice, UiT The Arctic University of Norway, Tromsø, Norway
| | - Torsten Risor
- Department of Public Health, Centre of General Practice, University of Copenhagen, Copenhagen, Denmark
- Section for General Practice, Department of Community Medicine, UiT The Arctic University of Norway, Tromsø, Norway
| | - Klaus Hoeyer
- Department of Public Health, Section for Health Services Research, University of Copenhagen, Copenhagen, Denmark
| |
Collapse
|
2
|
Keshmiri F, Javadi A. Feedback-based learning from viewpoints of surgical nursing students: A mixed-method study. J Eval Clin Pract 2024. [PMID: 38818690 DOI: 10.1111/jep.14024] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 01/06/2024] [Revised: 04/30/2024] [Accepted: 05/13/2024] [Indexed: 06/01/2024]
Abstract
BACKGROUND Feedback-based learning (FBL) focuses on guiding the learning process according to educational objectives and the student's needs. This study aimed to investigate surgical nursing students' perceptions and explore their experiences of FBL. METHOD The present study used a mixed-methods sequential explanatory design that was conducted in the quantitative and qualitative phases. Surgical nursing students participated in the quantitative phase (n = 105). In the first phase, students completed two questionnaires about FBL and clinical feedback. Semi-structured face-to-face interviews were used to collect qualitative data in the second phase. Graneheim and Lundman's inductive approaches were used to analyse the qualitative data. RESULTS The mean (SD) score for students' perception of FBL was 3.99 ± 0.70. The qualitative results were explored in two themes, "motivational support for improvement" and "unpleasant learning". CONCLUSION In this study, the positive and negative aspects of FBL were explained. FBL is perceived as a motivational support mechanism to improve students' capabilities during their academic courses and also prepare them for future careers. Conversely, FBL may experience unpleasant learning due to negative feedback and negative emotions.
Collapse
Affiliation(s)
- Fatemeh Keshmiri
- Department of Medical Education, Education Development Center, Shahid Sadoughi University of Medical Sciences, Yazd, Iran
| | - Alireza Javadi
- Department of Surgical Technology, Student Research Committee, Shahid Sadoughi University of Medical Sciences, Yazd, Iran
| |
Collapse
|
3
|
Tanaka P, Soo Park Y, Chen CY, Yumul R, Macario A. Domains Influencing Faculty Decisions on the Level of Supervision Required for Anesthesiology EPAs with Analysis of Feedback Comments. JOURNAL OF SURGICAL EDUCATION 2024; 81:741-752. [PMID: 38553368 DOI: 10.1016/j.jsurg.2024.02.003] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 10/19/2023] [Revised: 12/30/2023] [Accepted: 02/02/2024] [Indexed: 04/26/2024]
Abstract
OBJECTIVE The purpose of this qualitative study was to examine responses related to entrustment and feedback comments from an assessment tool. DESIGN Qualitative analyses using semi-structured interviews and analysis of narrative comments. SETTING Main hospital OR suite at a large academic medical center. PARTICIPANTS faculty, and residents who work in the OR suite. RESULTS Seven of the 14 theoretical domains from the Theoretical Domains Framework were identified as influencing faculty decision on entrustment: knowledge, skills, intention, memory/attention/decision processes, environmental context, and resources, beliefs of capabilities, and reinforcement. The majority (651/1116 (58.4%)) of faculty comments were critical/modest praise and relevant, consistent across all 6 EPAs. The written in feedback comments for all 1,116 Web App EPA assessments yielded a total of 1,599 sub-competency specific responses. These responses were mapped to core competencies, and at least once to 13 of the 23 ACGME subcompetencies. CONCLUSIONS Domains identified as influencing faculty decision on entrustment were knowledge, skills, intention, memory/attention/decision processes, environmental context, and resources, beliefs of capabilities, and reinforcement. Most narrative feedback comments were critical/modest praise and relevant, consistent across each of the EPAs.
Collapse
Affiliation(s)
- Pedro Tanaka
- Department of Anesthesiology, Perioperative and Pain Medicine, Stanford University School of Medicine, Stanford, California.
| | - Yoon Soo Park
- Associate Professor, Department of Medical Education, University of Illinois at Chicago College of Medicine, Chicago, Illinois
| | - Chien-Yu Chen
- Department of Anesthesiology, Taipei Medical University Hospital, Taipei, Taiwan; Department of Humanities in Medicine, School of Medicine, College of Medicine, Taipei
| | - Roya Yumul
- Professor, Cedars-Sinai Medical Center, Los Angeles, California
| | - Alex Macario
- Department of Anesthesiology, Perioperative and Pain Medicine, Stanford University School of Medicine, Stanford, California
| |
Collapse
|
4
|
Bearman M, Ajjawi R, Castanelli D, Denniston C, Molloy E, Ward N, Watling C. Meaning making about performance: A comparison of two specialty feedback cultures. MEDICAL EDUCATION 2023; 57:1010-1019. [PMID: 37142553 DOI: 10.1111/medu.15118] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 09/13/2022] [Revised: 04/20/2023] [Accepted: 04/24/2023] [Indexed: 05/06/2023]
Abstract
INTRODUCTION Specialty trainees often struggle to understand how well they are performing, and feedback is commonly seen as a solution to this problem. However, medical education tends to approach feedback as acontextual rather than located in a specialty-specific cultural world. This study therefore compares how specialty trainees in surgery and intensive care medicine (ICM) make meaning about the quality of their performance and the role of feedback conversations in this process. METHODS We conducted a qualitative interview study in the constructivist grounded theory tradition. We interviewed 17 trainees from across Australia in 2020, eight from ICM and nine from surgery, and iterated between data collection and analytic discussions. We employed open, focused, axial and theoretical coding. FINDINGS There were significant divergences between specialties. Surgical trainees had more opportunity to work directly with supervisors, and there was a strong link between patient outcome and quality of care, with a focus on performance information about operative skills. ICM was a highly uncertain practice environment, where patient outcome could not be relied upon as a source of performance information; valued performance information was diffuse and included tacit emotional support. These different 'specialty feedback cultures' strongly influenced how trainees orchestrated opportunities for feedback, made meaning of their performance in their day-to-day patient care tasks and 'patched together' experiences and inputs into an evolving sense of overall progress. DISCUSSION We identified two types of meaning-making about performance: first, trainees' understanding of an immediate performance in a patient-care task and, second, a 'patched together' sense of overall progress from incomplete performance information. This study suggests approaches to feedback should attend to both, but also take account of the cultural worlds of specialty practice, with their attendant complexities. In particular, feedback conversations could better acknowledge the variable quality of performance information and specialty specific levels of uncertainty.
Collapse
Affiliation(s)
- Margaret Bearman
- Centre for Research in Assessment and Digital Learning, Deakin University, Melbourne, Australia
| | - Rola Ajjawi
- Centre for Research in Assessment and Digital Learning, Deakin University, Melbourne, Australia
| | - Damian Castanelli
- School of Clinical Sciences at Monash Health, Monash University, Clayton, Victoria, Australia
- Department of Anaesthesia and Perioperative Medicine, Monash Health, Clayton, Victoria, Australia
| | - Charlotte Denniston
- Department of Medical Education, Melbourne Medical School, University of Melbourne, Melbourne, Australia
| | - Elizabeth Molloy
- Department of Medical Education, Melbourne Medical School, University of Melbourne, Melbourne, Australia
| | - Natalie Ward
- Centre for Research in Assessment and Digital Learning, Deakin University, Melbourne, Australia
| | - Chris Watling
- Centre for Education Research and Innovation, Schulich School of Medicine and Dentistry, Western University, London, Canada
| |
Collapse
|
5
|
Ajjawi R, Bearman M, Molloy E, Noble C. The role of feedback in supporting trainees who underperform in clinical environments. Front Med (Lausanne) 2023; 10:1121602. [PMID: 37181376 PMCID: PMC10167016 DOI: 10.3389/fmed.2023.1121602] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/12/2022] [Accepted: 04/03/2023] [Indexed: 05/16/2023] Open
Abstract
Introduction Underperformance in clinical environments can be costly and emotional for all stakeholders. Feedback is an important pedagogical strategy for working with underperformance - both formal and informal strategies can make a difference. Feedback is a typical feature of remediation programs, and yet there is little consensus on how feedback should unfold in the context of underperformance. Methods This narrative review synthesises literature at the intersections of feedback and underperformance in clinical environments where service, learning and safety need to be considered. We do so with a critical eye towards generating insights for working with underperformance in the clinical environment. Synthesis and discussion There are compounding and multi-level factors that contribute to underperformance and subsequent failure. This complexity overwrites simplistic notions of 'earned' failure through individual traits and deficit. Working with such complexity requires feedback that goes beyond educator input or 'telling'. When we shift beyond feedback as input to process, we recognise that these processes are fundamentally relational, where trust and safety are necessary for trainees to share their weaknesses and doubts. Emotions are always present and they signal action. Feedback literacy might help us consider how to engage trainees with feedback so that they take an active (autonomous) role in developing their evaluative judgements. Finally, feedback cultures can be influential and take effort to shift if at all. A key mechanism running through all these considerations of feedback is enabling internal motivation, and creating conditions for trainees to feel relatedness, competence and autonomy. Broadening our perceptions of feedback, beyond telling, might help create environments for learning to flourish.
Collapse
Affiliation(s)
- Rola Ajjawi
- Centre for Research in Assessment and Digital Learning, Deakin University, Melbourne, VIC, Australia
| | - Margaret Bearman
- Centre for Research in Assessment and Digital Learning, Deakin University, Melbourne, VIC, Australia
| | - Elizabeth Molloy
- Department of Medical Education, Faculty of Medicine, Dentistry and Health Sciences, University of Melbourne, Melbourne, VIC, Australia
| | - Christy Noble
- Academy for Medical Education, Medical School, The University of Queensland, Herston, QLD, Australia
| |
Collapse
|
6
|
Koutra K, Burns C, Sinko L, Kita S, Bilgin H, Arnault DS. Trauma Recovery Rubric: A Mixed-Method Analysis of Trauma Recovery Pathways in Four Countries. INTERNATIONAL JOURNAL OF ENVIRONMENTAL RESEARCH AND PUBLIC HEALTH 2022; 19:10310. [PMID: 36011945 PMCID: PMC9408383 DOI: 10.3390/ijerph191610310] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 07/16/2022] [Revised: 08/15/2022] [Accepted: 08/16/2022] [Indexed: 06/15/2023]
Abstract
Research is beginning to examine gender-based violence (GBV) survivors' recovery, but little is known about diverse recovery trajectories or their relationships with other distress and recovery variables. This interdisciplinary, international multisite mixed-method study developed and used the TRR to identify and classify survivors' trauma pathways. This study describes the phases of the initial development of the preliminary TRR (Phase 1), refines and calibrates the TRR (Phase 2), and then integrates the TRR into quantitative data from four countries (Phase 3). Seven recovery pathways with six domains emerged: normalizing, minimizing, consumed/trapped; shutdown or frozen, surviving, seeking and fighting for integration; finding integration/equanimity. Depression scores were related to most recovery domains, and TRR scores had large effect sizes. At the same time, PTSD was not statistically related to TRR scores, but TRR had a medium effect size. Our study found that the TRR can be implemented in diverse cultural settings and promises a reliable cross-cultural tool. The TRR is a survivor-centered, trauma-informed way to understand different survivorship pathways and how different pathways impact health outcomes. Overall, this rubric provides a foundation for future study on differences in survivor healing and the drivers of these differences. This tool can potentially improve survivor care delivery and our understanding of how to meet best the needs of the survivor populations we intend to serve.
Collapse
Affiliation(s)
- Kleio Koutra
- Social Work Department, Hellenic Mediterranean University, 71401 Crete, Greece
| | - Courtney Burns
- University of Michigan Medical School, Ann Arbor, MI 48109, USA
| | - Laura Sinko
- Department of Nursing, Temple University College of Public Health, Philadelphia, PA 19122, USA
| | - Sachiko Kita
- Department of Family Nursing, Division of Health Science and Nursing, Graduate School of Medicine, The University of Tokyo, Tokyo 1130003, Japan
| | - Hülya Bilgin
- Mental Health and Psychiatric Nursing Department, Florence Nightingale Nursing Faculty, Istanbul University-Cerrahpasa, Istanbul 34381, Turkey
| | | |
Collapse
|
7
|
Bearman M, Dracup M, Garth B, Johnson C, Wearne E. Learning to recognise what good practice looks like: how general practice trainees develop evaluative judgement. ADVANCES IN HEALTH SCIENCES EDUCATION : THEORY AND PRACTICE 2022; 27:215-228. [PMID: 34859317 DOI: 10.1007/s10459-021-10086-3] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 04/30/2021] [Accepted: 11/14/2021] [Indexed: 06/13/2023]
Abstract
The nature of healthcare means doctors must continually calibrate the quality of their work within constantly changing standards of practice. As trainees move into working as fully qualified professionals, they can struggle to know how well they are practising in the absence of formal oversight. They therefore need to build their evaluative judgement: their capability to interpret cues and messages from the clinical environment, allowing them to judge quality of practice. This paper explores how Australian general practice (GP) trainees develop their evaluative judgement. We interviewed 16 GPs, who had recently completed certification requirements, asking them how they managed complex learning challenges across their training trajectory. A thematic analysis was sensitised by conceptualisations of evaluative judgement and feedback for future practice. Findings are reported via three themes: sources of performance relevant information; sense-making about progress within complex learning challenges; and changing practice as evaluative judgement develops. Trainees actively sought to understand what quality practice looked like within complex and ambiguous circumstances but often found it difficult to calibrate their performance. While reflective practice was key to developing evaluative judgment, feedback conversations could provide significant opportunities for trainees and supervisors to co-construct meaning. A 'feedback community' was available for frequent instances where supervisors were absent or not regarded as entirely credible, although feedback conversations in themselves did not necessarily assist trainees to develop evaluative judgement. There is room for a more active role for supervisors in assisting trainees to consider how to independently make sense of learning cues.
Collapse
Affiliation(s)
- Margaret Bearman
- Centre for Research in Assessment and Digital Learning (CRADLE), Deakin University, Tower 2, Level 12, 727 Collins St, Docklands Melbourne, 3008, Australia.
| | - Mary Dracup
- Centre for Research in Assessment and Digital Learning (CRADLE), Deakin University, Tower 2, Level 12, 727 Collins St, Docklands Melbourne, 3008, Australia
| | - Belinda Garth
- Faculty of Medicine, Nursing and Health Sciences, Monash University, Melbourne, Australia
| | - Caroline Johnson
- Eastern Victoria General Practice Training, Melbourne/Churchill, Australia
- Department of General Practice, University of Melbourne, Melbourne, Australia
| | - Elisabeth Wearne
- Eastern Victoria General Practice Training, Melbourne/Churchill, Australia
| |
Collapse
|
8
|
Abstract
If used thoughtfully and with intent, feedback and coaching will promote learning and growth as well as personal and professional development in our learners. Feedback is an educational tool as well as a social interaction between learner and supervisor, in the context of a respectful and trusting relationship. It challenges the learner's thinking and supports the learner's growth. Coaching is an educational philosophy dedicated to supporting learners' personal and professional development and growth and supporting them to reach their potential. In clinical education, feedback is most effective when it is explicitly distinguished from summative assessment. Importantly, feedback should be about firsthand observed behaviors (which can be direct or indirect) and not about information which comes from a third party. Learners are more receptive to feedback if it comes from a source that they perceive as credible, and with whom they have developed rapport. The coaching relationship between learner and supervisor should also be built on mutual trust and respect. Coaching can be provided in the moment (feedback on everyday clinical activities that leads to performance improvement, even with short interaction with a supervisor) and over time (a longer term relationship with a supervisor in which there is reflection on the learner's development and co-creation of new learning goals). Feedback and coaching are most valuable when the learner and teacher exhibit a growth mindset. At the organizational level, it is important that both the structures and training are in place to ensure a culture of effective feedback and coaching in the clinical workplace.Conclusions: Having a thoughtful and intentional approach to feedback and coaching with learners, as well as applying evidence-based principles, will not only contribute in a significant way to their developmental progression, but will also provide them with the tools they need to have the best chance of achieving competence throughout their training. What is Known: • Feedback and coaching are key to advancing the developmental progression of trainees as they work towards achieving competence. • Feedback is not a one-way delivery of specific information from supervisor to trainee, but rather a social interaction between two individuals in which trust and respect play a key role. • Provision of effective feedback may be hampered by confusing formative (supporting trainee learning and development) and summative (the judgment that is made about a trainee's level of competence) purposes. What is New: • Approaches to both the provision of feedback/coaching and the assessment of competence must be developed in parallel to ensure success in clinical training programs. • Faculty development is essential to provide clinical teachers with the skills to provide effective feedback and coaching. • Coaching's effectiveness relies on nurturing strong trainee-supervisor relationships, ensuring high-quality feedback, nourishing a growth mindset, and encouraging an institutional culture that embraces feedback and coaching.
Collapse
|
9
|
Roberts LW. Emerging Issues in Assessment in Medical Education: A Collection. ACADEMIC MEDICINE : JOURNAL OF THE ASSOCIATION OF AMERICAN MEDICAL COLLEGES 2021; 96:159-160. [PMID: 33492817 DOI: 10.1097/acm.0000000000003855] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/12/2023]
|