51
|
Berger S, Stalmeijer RE, Marty AP, Berendonk C. Exploring the Impact of Entrustable Professional Activities on Feedback Culture: A Qualitative Study of Anesthesiology Residents and Attendings. ACADEMIC MEDICINE : JOURNAL OF THE ASSOCIATION OF AMERICAN MEDICAL COLLEGES 2023; 98:836-843. [PMID: 36812061 DOI: 10.1097/acm.0000000000005188] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/18/2023]
Abstract
PURPOSE Entrustable professional activities (EPAs) were introduced as a potential way to optimize workplace-based assessments. Yet, recent studies suggest that EPAs have not yet overcome all of the challenges to implementing meaningful feedback. The aim of this study was to explore the extent to which the introduction of EPAs via mobile app impacts feedback culture as experienced by anesthesiology residents and attending physicians. METHOD Using a constructivist grounded theory approach, the authors interviewed a purposive and theoretical sample of residents (n = 11) and attendings (n = 11) at the Institute of Anaesthesiology, University Hospital of Zurich, where EPAs had recently been implemented. Interviews took place between February and December 2021. Data collection and analysis were conducted iteratively. The authors used open, axial, and selective coding to gain knowledge and understanding on the interplay of EPAs and feedback culture. RESULTS Participants reflected on a number of changes in their day-to-day experience of feedback culture with the implementation of EPAs. Three main mechanisms were instrumental in this process: lowering the feedback threshold, change in feedback focus, and gamification. Participants felt a lower threshold to feedback seeking and giving and that the frequency of feedback conversations increased and tended to be more focused on a specific topic and shorter, while feedback content tended to focus more on technical skills and more attention was given to average performances. Residents indicated that the app-based approach fostered a game-like motivation to "climb levels," while attendings did not perceive a game-like experience. CONCLUSIONS EPAs may offer a solution to problems of infrequent occurrence of feedback and invite attention to average performances and technical competencies, but may come at the expense of feedback on nontechnical skills. This study suggests that feedback culture and feedback instruments have a mutually interacting influence on each other.
Collapse
Affiliation(s)
- Sabine Berger
- S. Berger is a third-year medical resident, Internal Medicine Training Program, St. Claraspital, Basel, Switzerland
| | - Renee E Stalmeijer
- R.E. Stalmeijer is associate professor, Department of Educational Development and Research, Faculty of Health, Medicine and Life Sciences, Maastricht University, Maastricht, The Netherlands
| | - Adrian P Marty
- A.P. Marty is currently senior attending physician and team lead for education, Institute of Anaesthesiology, Intensive Care and Pain Medicine, Orthopedic University Hospital Balgrist, Zurich, Switzerland. At the time of writing, he was attending physician, Institute of Anaesthesiology, University of Zurich, University Hospital of Zurich, Zurich, Switzerland
| | - Christoph Berendonk
- C. Berendonk is senior lecturer in medical education, Institute for Medical Education, University of Bern, Bern, Switzerland
| |
Collapse
|
52
|
Gauthier S, Braund H, Dalgarno N, Taylor D. Assessment-Seeking Strategies: Navigating the Decision to Initiate Workplace-Based Assessment. TEACHING AND LEARNING IN MEDICINE 2023:1-10. [PMID: 37384570 DOI: 10.1080/10401334.2023.2229803] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Received: 07/29/2022] [Revised: 04/13/2023] [Accepted: 06/01/2023] [Indexed: 07/01/2023]
Abstract
Phenomenon: Competency-based medical education (CBME) relies on workplace-based assessment (WBA) to generate formative feedback (assessment for learning-AfL) and make inferences about competence (assessment of learning-AoL). When approaches to CBME rely on residents to initiate WBA, learners experience tension between seeking WBA for learning and for establishing competence. How learners resolve this tension may lead to unintended consequences for both AfL and AoL. We sought to explore the factors that impact both decisions to seek and not to seek WBA and use the findings to build a model of assessment-seeking strategy used by residents. In building this model we consider how the link between WBA and promotion or progression within a program impacts an individual's assessment-seeking strategy. Approach: We conducted 20 semi-structured interviews with internal medicine residents at Queen's University about the factors that influence their decision to seek or avoid WBA. Using grounded theory methodology, we applied a constant comparative analysis to collect data iteratively and identify themes. A conceptual model was developed to describe the interaction of factors impacting the decision to seek and initiate WBA. Findings: Participants identified two main motivations when deciding to seek assessments: the need to fulfill program requirements and the desire to receive feedback for learning. Analysis suggested that these motivations are often at odds with each other. Participants also described several moderating factors that impact the decision to initiate assessments, irrespective of the primary underlying motivation. These included resident performance, assessor factors, training program expectations, and clinical context. A conceptual framework was developed to describe the factors that lead to strategic assessment-seeking behaviors. Insights: Faced with the dual purpose of WBA in CBME, resident behavior in initiating assessment is guided by specific assessment-seeking strategies. Strategies reflect individual underlying motivations, influenced by four moderating factors. These findings have broad implications for programmatic assessment in a CBME context including validity considerations for assessment data used in summative decision-making including readiness for unsupervised practice.
Collapse
Affiliation(s)
- Stephen Gauthier
- Department of Medicine, Queen's University, Kingston, Ontario, Canada
| | - Heather Braund
- Office of Professional Development and Educational Scholarship, Faculty of Health Sciences, Queen's University, Kingston, Ontario, Canada
| | - Nancy Dalgarno
- Office of Professional Development and Educational Scholarship, Faculty of Health Sciences, Queen's University, Kingston, Ontario, Canada
| | - David Taylor
- Department of Medicine, Queen's University, Kingston, Ontario, Canada
| |
Collapse
|
53
|
Lip A, Watling CJ, Ginsburg S. What does "Timely" Mean to Residents? Challenging Feedback Assumptions in Postgraduate Education. PERSPECTIVES ON MEDICAL EDUCATION 2023; 12:218-227. [PMID: 37334109 PMCID: PMC10275343 DOI: 10.5334/pme.1052] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 05/11/2023] [Accepted: 05/29/2023] [Indexed: 06/20/2023]
Abstract
Introduction Current orthodoxy states that feedback should be timely and face-to-face, yet the optimal timing and mode of delivery for feedback is unclear. We explored what "optimal timing" means from residents' points of view as feedback providers and receivers, to ultimately inform strategies to optimize feedback in training. Methods As near-peers who have dual roles in both providing and receiving feedback, 16 subspecialty (PGY4 and 5) internal medicine residents were interviewed about their perceptions of the optimal timing and format of feedback. Using constructivist grounded theory, interviews were conducted and analyzed iteratively. Results Drawing on their experiences as providers and recipients, residents described simultaneously considering and weighing multiple factors when deciding on when and how to provide feedback. These included their own readiness to engage in providing meaningful feedback, the perceived receptiveness of the learner and the apparent urgency of feedback delivery (e.g., if patient safety was at stake). Face-to-face verbal feedback was valued for encouraging dialogue but could be uncomfortable and limited by time constraints. Written feedback could be more honest and concise, and the possibility of asynchronous delivery had potential to overcome issues with timing and discomfort. Discussion Participants' perceptions of the optimal timing of feedback challenge current assumptions about the benefits of "immediate" versus "delayed". The concept of "optimal timing" for feedback was found to be complex and context-dependent, defying a formulaic approach. There may be a role for asynchronous and/or written feedback, which has potential to address unique issues identified issues in near-peer relationships.
Collapse
Affiliation(s)
- Alyssa Lip
- Department of Medicine, Temerty Faculty of Medicine, University of Toronto, CA
| | - Christopher J. Watling
- Centre for Education Research and Innovation, Schulich School of Medicine & Dentistry, Western University, London, Ontario, CA
| | - Shiphra Ginsburg
- Department of Medicine, Sinai Health System and Faculty of Medicine, University of Toronto, scientist, Wilson Centre for Research in Education, University of Toronto, Toronto, Ontario, CA
- Canada Research Chair in Health Professions Education, CA
| |
Collapse
|
54
|
Kozato A, Shikino K, Matsuyama Y, Hayashi M, Kondo S, Uchida S, Stanyon M, Ito S. A qualitative study examining the critical differences in the experience of and response to formative feedback by undergraduate medical students in Japan and the UK. BMC MEDICAL EDUCATION 2023; 23:408. [PMID: 37277728 PMCID: PMC10240445 DOI: 10.1186/s12909-023-04257-6] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 07/29/2022] [Accepted: 04/13/2023] [Indexed: 06/07/2023]
Abstract
BACKGROUND Formative feedback plays a critical role in guiding learners to gain competence, serving as an opportunity for reflection and feedback on their learning progress and needs. Medical education in Japan has historically been dominated by a summative paradigm within assessment, as opposed to countries such as the UK where there are greater opportunities for formative feedback. How this difference affects students' interaction with feedback has not been studied. We aim to explore the difference in students' perception of feedback in Japan and the UK. METHODS The study is designed and analysed with a constructivist grounded theory lens. Medical students in Japan and the UK were interviewed on the topic of formative assessment and feedback they received during clinical placements. We undertook purposeful sampling and concurrent data collection. Data analysis through open and axial coding with iterative discussion among research group members was conducted to develop a theoretical framework. RESULTS Japanese students perceived feedback as a model answer provided by tutors which they should not critically question, which contrasted with the views of UK students. Japanese students viewed formative assessment as an opportunity to gauge whether they are achieving the pass mark, while UK students used the experience for reflective learning. CONCLUSIONS The Japanese student experience of formative assessment and feedback supports the view that medical education and examination systems in Japan are focused on summative assessment, which operates alongside culturally derived social pressures including the expectation to correct mistakes. These findings provide new insights in supporting students to learn from formative feedback in both Japanese and UK contexts.
Collapse
Affiliation(s)
- An Kozato
- Postgraduate Education Centre, Ipswich Hospital NHS Trust, Ipswich, UK.
- Ipswich Hospital, Heath Rd, IP4 5PD, Ipswich, UK.
| | - Kiyoshi Shikino
- Health Professional Development Center, Chiba University Hospital, Chiba, Japan
- Department of General Medicine, Chiba University Hospital, Chiba, Japan
| | | | - Mikio Hayashi
- Center for Medical Education, Kansai Medical University, Osaka, Japan
- Master of Medical Sciences in Medical Education, Harvard Medical School, Boston, MA, USA
| | - Satoshi Kondo
- Department of Medical Education Studies, Graduate School of Medicine, International Research Center for Medical Education, The University of Tokyo, Tokyo, Japan
- Center for Medical Education and Career Development, Graduate School of Medicine, University of Toyama, Toyama, Japan
| | - Shun Uchida
- Health Professional Development Center, Chiba University Hospital, Chiba, Japan
| | - Maham Stanyon
- Center for Medical Education and Career Development, Fukushima Medical University, Fukushima, Japan
| | - Shoichi Ito
- Health Professional Development Center, Chiba University Hospital, Chiba, Japan
- Department of Medical Education, Graduate School of Medicine, Chiba University, Chiba, Japan
| |
Collapse
|
55
|
Peng X, Jiang L, Cao D, Chen G, Li D, Teng P, Li J. Teacher feedback-based collaborative testing improves students' knowledge gaps of parasitology. ADVANCES IN PHYSIOLOGY EDUCATION 2023; 47:259-264. [PMID: 36861765 DOI: 10.1152/advan.00230.2022] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 11/04/2022] [Revised: 02/22/2023] [Accepted: 02/23/2023] [Indexed: 06/18/2023]
Abstract
Collaborative testing has been demonstrated the ability to improve students' performance, enhance students' learning, and aid in knowledge retention in many different courses. However, this examination mode lacks the process of teacher feedback. Herein, a short teacher feedback from was added immediately after the collaborative testing to improve the students' performance. A parasitology class of 121 undergraduates was randomized into two groups: group A and group B. Collaborative testing was carried out at the end of theoretical teaching. During the test, students would first answer questions as individuals for 20 minutes. Then, students from group A answered the same questions in groups (5 students in each group) for 20 minutes, while the group-testing duration was only 15 minutes in group B. Immediately after the group testing, teachers conducted a 5-minute feedback about the morphology identification according to the analysis of the answers by group B. Four weeks later, a final test was conducted in an individual test. The total scores and scores for each examination content were analyzed. The results showed that there was no significant difference in the final exam scores between both groups (t = -1.278, P = 0.204). However, the morphological and diagnostic test results of the final examination in group B were significantly higher than those of the midterm examination, while there was no significant change in group A (t = 4.333, P = 0.051). The results confirmed that the teacher feedback after the collaborative testing can effectively make up for the students' knowledge gaps.NEW & NOTEWORTHY This study found that collaborative group testing is helpful for teachers to grasp students' knowledge gaps more easily and the teacher feedback after the collaborative group testing can effectively make up for the knowledge gaps of students.
Collapse
Affiliation(s)
- Xiaohong Peng
- Department of Parasitology, Guilin Medical University, Guangxi, People's Republic of China
- Office of Academic Affairs, Guilin Medical University, Guangxi, People's Republic of China
| | - Liping Jiang
- Department of Parasitology, Guilin Medical University, Guangxi, People's Republic of China
| | - Deping Cao
- Department of Parasitology, Guilin Medical University, Guangxi, People's Republic of China
| | - Gen Chen
- Department of Parasitology, Guilin Medical University, Guangxi, People's Republic of China
| | - Dayu Li
- Department of Parasitology, Guilin Medical University, Guangxi, People's Republic of China
| | - Pingying Teng
- Department of Parasitology, Guilin Medical University, Guangxi, People's Republic of China
| | - Jianzhou Li
- Office of Academic Affairs, Guilin Medical University, Guangxi, People's Republic of China
| |
Collapse
|
56
|
Martin L, Blissett S, Johnston B, Tsang M, Gauthier S, Ahmed Z, Sibbald M. How workplace-based assessments guide learning in postgraduate education: A scoping review. MEDICAL EDUCATION 2023; 57:394-405. [PMID: 36286100 DOI: 10.1111/medu.14960] [Citation(s) in RCA: 6] [Impact Index Per Article: 6.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 05/30/2022] [Revised: 09/16/2022] [Accepted: 10/21/2022] [Indexed: 06/16/2023]
Abstract
INTRODUCTION Competency-based medical education (CBME) led to the widespread adoption of workplace-based assessment (WBA) with the promise of achieving assessment for learning. Despite this, studies have illustrated tensions between the summative and formative role of WBA which undermine learning goals. Models of workplace-based learning (WBL) provide insight, however, these models excluded WBA. This scoping review synthesizes the primary literature addressing the role of WBA to guide learning in postgraduate medical education, with the goal of identifying gaps to address in future studies. METHODS The search was applied to OVID Medline, Web of Science, ERIC and CINAHL databases, articles up to September 2020 were included. Titles and abstracts were screened by two reviewers, followed by a full text review. Two members independently extracted and analysed quantitative and qualitative data using a descriptive-analytic technique rooted in Billett's four premises of WBL. Themes were synthesized and discussed until consensus. RESULTS All 33 papers focused on the perception of learning through WBA. The majority applied qualitative methodology (70%), and 12 studies (36%) made explicit reference to theory. Aligning with Billett's first premise, results reinforce that learning always occurs in the workplace. WBA helped guide learning goals and enhanced feedback frequency and specificity. Billett's remaining premises provided an important lens to understand how tensions that existed in WBL have been exacerbated with frequent WBA. As individuals engage in both work and WBA, they are slowly transforming the workplace. Culture and context frame individual experiences and the perceived authenticity of WBA. Finally, individuals will have different goals, and learn different things, from the same experience. CONCLUSION Analysing WBA literature through the lens of WBL theory allows us to reframe previously described tensions. We propose that future studies attend to learning theory, and demonstrate alignment with philosophical position, to advance our understanding of assessment-for-learning in the workplace.
Collapse
Affiliation(s)
- Leslie Martin
- Department of Medicine, McMaster University, Hamilton, Ontario, Canada
| | - Sarah Blissett
- Department of Medicine, Western University, London, Ontario, Canada
| | - Bronte Johnston
- McMaster Education Research, Innovation, and Theory Program, McMaster University, Hamilton, Ontario, Canada
| | - Michael Tsang
- Department of Medicine, McMaster University, Hamilton, Ontario, Canada
| | - Stephen Gauthier
- Department of Medicine, Queens University, Kingston, Ontario, Canada
| | - Zeeshan Ahmed
- Department of Medicine, Ottawa University, Ottawa, Ontario, Canada
| | - Matthew Sibbald
- Department of Medicine, McMaster University, Hamilton, Ontario, Canada
| |
Collapse
|
57
|
Chin M, Pack R, Cristancho S. "A whole other competence story": exploring faculty perspectives on the process of workplace-based assessment of entrustable professional activities. ADVANCES IN HEALTH SCIENCES EDUCATION : THEORY AND PRACTICE 2023; 28:369-385. [PMID: 35997910 DOI: 10.1007/s10459-022-10156-0] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Received: 02/24/2022] [Accepted: 08/07/2022] [Indexed: 05/11/2023]
Abstract
The centrality of entrustable professional activities (EPAs) in competency-based medical education (CBME) is predicated on the assumption that low-stakes, high-frequency workplace-based assessments used in a programmatic approach will result in accurate and defensible judgments of competence. While there have been conversations in the literature regarding the potential of this approach, only recently has the conversation begun to explore the actual experiences of clinical faculty in this process. The purpose of this qualitative study was to explore the process of EPA assessment for faculty in everyday practice. We conducted 18 semi-structured interviews with Anesthesia faculty at a Canadian academic center. Participants were asked to describe how they engage in EPA assessment in daily practice and the factors they considered. Interviews were audio-recorded, transcribed, and analysed using the constant comparative method of grounded theory. Participants in this study perceived two sources of tension in the EPA assessment process that influenced their scoring on official forms: the potential constraints of the assessment forms and the potential consequences of their assessment outcome. This was particularly salient in circumstances of uncertainty regarding the learner's level of competence. Ultimately, EPA assessment in CBME may be experienced as higher-stakes by faculty than officially recognized due to these tensions, suggesting a layer of discomfort and burden in the process that may potentially interfere with the goal of assessment for learning. Acknowledging and understanding the nature of this burden and identifying strategies to mitigate it are critical to achieving the assessment goals of CBME.
Collapse
Affiliation(s)
- Melissa Chin
- Department of Anesthesia and Perioperative Medicine, London Health Sciences Centre, Schulich School of Medicine and Dentistry, University of Western Ontario, London, ON, Canada.
| | - Rachael Pack
- Center for Education Research and Innovation, University of Western Ontario, London, ON, Canada
| | - Sayra Cristancho
- Center for Education Research and Innovation, University of Western Ontario, London, ON, Canada
| |
Collapse
|
58
|
Nilsson T, Masiello I, Broberger E, Lindström V. Digital feedback during clinical education in the emergency medical services: a qualitative study. BMC MEDICAL EDUCATION 2023; 23:156. [PMID: 36918851 PMCID: PMC10015958 DOI: 10.1186/s12909-023-04138-y] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 01/05/2023] [Accepted: 03/03/2023] [Indexed: 06/18/2023]
Abstract
BACKGROUND Clinical education is essential for students' progress towards becoming registered nurses (RN) in Sweden. Assessment of caring skills in the Emergency Medical Services (EMS) is complex due to the ever-changing scenarios and the fact that multiple supervisors are involved in the student's education. Currently, assessments of student's skills are summative and occur twice during the six weeks of clinical education. A digitalized assessment tool (DAT) with an adaptation for formative assessment is a new approach to assessment of nursing skills in the EMS. Since new technologies and changes in procedures are likely to affect both students and supervisors, our aim in this study is to describe students' and clinical supervisors' experience of formative assessments using DAT in the EMS. METHOD This study is qualitative, using semi-structured group interviews (N = 2) with students and semi-structured individual telephone interviews (N = 13) with supervisors. The data was analysed according to Graneheim and Landman's method for content analysis. This analysis generated 221 codes organized into 10 categories within which three themes were identified. The students in this study were nursing students in their last semester and all supervisors were experienced RNs. RESULTS The results showed that students and supervisors had mainly positive views of the DAT and the formative assessment stating that the information they provided while using the DAT offered opportunities for reflection. The DAT supported the students' learning by visualizing strengths and areas of improvement, as well as displaying progress using a Likert scale. The application improved communication, but additional features linking the assessment tool with the university were requested. The application contributed to transparency in the assessments and was seen as preferable to the traditional 'pen and paper' method. CONCLUSION A digital system was described in a positive manner, and the assessment using the DAT facilitated reflection and formative assessment. The use of a Likert scale was considered positive in order to demonstrate progression which with advantage could be demonstrated visually.
Collapse
Affiliation(s)
| | - I. Masiello
- Department of Computer Science and Media Technology, Linnaeus University, Växjö, Sweden
- Department for Neurobiology, Care Sciences, and Society, Division of Nursing, Karolinska Institutet, Huddinge, Sweden
| | - E. Broberger
- Department for Neurobiology, Care Sciences, and Society, Division of Nursing, Karolinska Institutet, Huddinge, Sweden
| | - V. Lindström
- Department of Nursing, Umeå University, Umeå, Sweden
- Department of Ambulance Service, Region Västerbotten, Umeå, Sweden
- Department of Health Promotion Science, Sophiahemmet University, Stockholm, Sweden
| |
Collapse
|
59
|
Sasek CA. Implications of Entrustable Professional Activities for Motivation and Learning. J Physician Assist Educ 2023; 34:15-19. [PMID: 36723431 DOI: 10.1097/jpa.0000000000000477] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 06/18/2023]
Abstract
INTRODUCTION This study sought to explore the experience of entrustable professional activity-based assessment and its effect on self-determination, a well-supported measure of motivation. The goal was to understand EPA-based assessment's implications for student motivation, learning, and wellbeing. METHODS A qualitative case study design was utilized through a standardized simulation activity with EPA-based assessment. An analysis of interviews and reflection journals was completed by the coding of emergent themes and significant statements with a subsequent thematic analysis framed in self-determination theory (SDT). RESULTS Entrustment was found to be an affectively and interpersonally impactful metric that provided the learner with a clear understanding of clinical performance. Quality entrustment feedback supported participants' sense of competence and increased their confidence in their ability to be autonomous as well as a valuable part of the healthcare team. DISCUSSION EPAs appeared to support learners' sense of competence and to provide a bridge to support autonomy and the ability to relate to others. To the extent that entrustment supported learners' sense of competence, other SDT propositions could be subsequently bolstered as well. Self-determination and intrinsic motivation were uniquely supported by EPA-based assessment, if implemented effectively. Further research is needed to better define effective implementation, faculty development, and feedback strategies.
Collapse
Affiliation(s)
- Cody A Sasek
- Cody A. Sasek, PhD, PA-C, is program director and assistant professor for the Butler University DMS Bridge Program in Indianapolis, Indiana
| |
Collapse
|
60
|
Rietmeijer CBT, van Esch SCM, Blankenstein AH, van der Horst HE, Veen M, Scheele F, Teunissen PW. A phenomenology of direct observation in residency: Is Miller's 'does' level observable? MEDICAL EDUCATION 2023; 57:272-279. [PMID: 36515981 PMCID: PMC10107098 DOI: 10.1111/medu.15004] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Download PDF] [Subscribe] [Scholar Register] [Received: 04/11/2022] [Revised: 12/08/2022] [Accepted: 12/10/2022] [Indexed: 06/17/2023]
Abstract
INTRODUCTION Guidelines on direct observation (DO) present DO as an assessment of Miller's 'does' level, that is, the learner's ability to function independently in clinical situations. The literature, however, indicates that residents may behave 'inauthentically' when observed. To minimise this 'observer effect', learners are encouraged to 'do what they would normally do' so that they can receive feedback on their actual work behaviour. Recent phenomenological research on patients' experiences with DO challenges this approach; patients needed-and caused-some participation of the observing supervisor. Although guidelines advise supervisors to minimise their presence, we are poorly informed on how some deliberate supervisor participation affects residents' experience in DO situations. Therefore, we investigated what residents essentially experienced in DO situations. METHODS We performed an interpretive phenomenological interview study, including six general practice (GP) residents. We collected and analysed our data, using the four phenomenological lenses of lived body, lived space, lived time and lived relationship. We grouped our open codes by interpreting what they revealed about common structures of residents' pre-reflective experiences. RESULTS Residents experienced the observing supervisor not just as an observer or assessor. They also experienced them as both a senior colleague and as the patient's familiar GP, which led to many additional interactions. When residents tried to act as if the supervisor was not there, they could feel insecure and handicapped because the supervisor was there, changing the situation. DISCUSSION Our results indicate that the 'observer effect' is much more material than was previously understood. Consequently, observing residents' 'authentic' behaviour at Miller's 'does' level, as if the supervisor was not there, seems impossible and a misleading concept: misleading, because it may frustrate residents and cause supervisors to neglect patients' and residents' needs in DO situations. We suggest that one-way DO is better replaced by bi-directional DO in working-and-learning-together sessions.
Collapse
Affiliation(s)
- Chris B. T. Rietmeijer
- Department of General PracticeAmsterdam UMC, location Vrije Universiteit AmsterdamAmsterdamThe Netherlands
| | - Suzanne C. M. van Esch
- Department of General PracticeAmsterdam UMC, location University of AmsterdamAmsterdamThe Netherlands
| | - Annette H. Blankenstein
- Department of General PracticeAmsterdam UMC, location Vrije Universiteit AmsterdamAmsterdamThe Netherlands
| | - Henriëtte E. van der Horst
- Department of General PracticeAmsterdam UMC, location Vrije Universiteit AmsterdamAmsterdamThe Netherlands
| | - Mario Veen
- Department of General PracticeErasmus Medical CenterRotterdamThe Netherlands
| | - Fedde Scheele
- School of Medical Sciences, Athena Institute for Transdisciplinary ResearchAmsterdam UMC, location Vrije Universiteit AmsterdamAmsterdamThe Netherlands
| | - Pim W. Teunissen
- School of Health Professions EducationMaastricht UniversityMaastrichtThe Netherlands
| |
Collapse
|
61
|
Westein MPD, Koster AS, Daelmans HEM, Bouvy ML, Kusurkar RA. How progress evaluations are used in postgraduate education with longitudinal supervisor-trainee relationships: a mixed method study. ADVANCES IN HEALTH SCIENCES EDUCATION : THEORY AND PRACTICE 2023; 28:205-222. [PMID: 36094680 PMCID: PMC9992254 DOI: 10.1007/s10459-022-10153-3] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 04/30/2021] [Accepted: 08/07/2022] [Indexed: 06/15/2023]
Abstract
The combination of measuring performance and giving feedback creates tension between formative and summative purposes of progress evaluations and can be challenging for supervisors. There are conflicting perspectives and evidence on the effects supervisor-trainee relationships have on assessing performance. The aim of this study was to learn how progress evaluations are used in postgraduate education with longitudinal supervisor-trainee relationships. Progress evaluations in a two-year community-pharmacy specialization program were studied with a mixed-method approach. An adapted version of the Canadian Medical Education Directives for Specialists (CanMEDS) framework was used. Validity of the performance evaluation scores of 342 trainees was analyzed using repeated measures ANOVA. Semi-structured interviews were held with fifteen supervisors to investigate their response processes, the utility of the progress evaluations, and the influence of supervisor-trainee relationships. Time and CanMEDS roles affected the three-monthly progress evaluation scores. Interviews revealed that supervisors varied in their response processes. They were more committed to stimulating development than to scoring actual performance. Progress evaluations were utilized to discuss and give feedback on trainee development and to add structure to the learning process. A positive supervisor-trainee relationship was seen as the foundation for feedback and supervisors preferred the roles of educator, mentor, and coach over the role of assessor. We found that progress evaluations are a good method for directing feedback in longitudinal supervisor-trainee relationships. The reliability of scoring performance was low. We recommend progress evaluations to be independent of formal assessments in order to minimize roles-conflicts of supervisors.
Collapse
Affiliation(s)
- Marnix P D Westein
- Department of Pharmaceutical Sciences, Utrecht University, Universiteitsweg 99, 3584, CG, Utrecht, The Netherlands.
- Research in Education, Faculty of Medicine Vrije Universiteit, Amsterdam, The Netherlands.
- The Royal Dutch Pharmacists Association (KNMP), The Hague, The Netherlands.
| | - A S Koster
- Department of Pharmaceutical Sciences, Utrecht University, Universiteitsweg 99, 3584, CG, Utrecht, The Netherlands
| | - H E M Daelmans
- Programme Director Master of Medicine, Faculty of Medicine Vrije Universiteit, Amsterdam, The Netherlands
| | - M L Bouvy
- Department of Pharmaceutical Sciences, Utrecht University, Universiteitsweg 99, 3584, CG, Utrecht, The Netherlands
| | - R A Kusurkar
- Research in Education, Faculty of Medicine Vrije Universiteit, Amsterdam, The Netherlands
| |
Collapse
|
62
|
Toubassi D, Schenker C, Roberts M, Forte M. Professional identity formation: linking meaning to well-being. ADVANCES IN HEALTH SCIENCES EDUCATION : THEORY AND PRACTICE 2023; 28:305-318. [PMID: 35913664 PMCID: PMC9341156 DOI: 10.1007/s10459-022-10146-2] [Citation(s) in RCA: 9] [Impact Index Per Article: 9.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 01/12/2022] [Accepted: 07/10/2022] [Indexed: 06/15/2023]
Abstract
Trainee distress and burnout continue to be serious concerns for educational programs in medicine, prompting the implementation of numerous interventions. Although an expansive body of literature suggests that the experience of meaning at work is critical to professional wellbeing, relatively little attention has been paid to how this might be leveraged in the educational milieu. We propose that professional identity formation (PIF), the process by which trainees come to not only attain competence, but additionally to "think, act and feel" like physicians, affords us a unique opportunity to ground trainees in the meaningfulness of their work. Using the widely accepted tri-partite model of meaning, we outline how this process can contribute to wellbeing. We suggest strategies to optimize the influence of PIF on wellbeing, offering curricular suggestions, as well as ideas regarding the respective roles of communities of practice, teachers, and formative educational experiences. Collectively, these encourage trainees to act as intentional agents in the making of their novel professional selves, anchoring them to the meaningfulness of their work, and supporting their short and long-term wellbeing.
Collapse
Affiliation(s)
- Diana Toubassi
- Department of Family and Community Medicine, University of Toronto, Toronto, ON, Canada.
- University Health Network - Toronto Western FHT, 440 Bathurst Street - Suite 300, Toronto, ON, M5T 2S6, Canada.
| | - Carly Schenker
- Department of Family and Community Medicine, University of Toronto, Toronto, ON, Canada
| | - Michael Roberts
- Department of Family and Community Medicine, University of Toronto, Toronto, ON, Canada
| | - Milena Forte
- Department of Family and Community Medicine, University of Toronto, Toronto, ON, Canada
| |
Collapse
|
63
|
Watling C, Shaw J, Field E, Ginsburg S. 'For the most part it works': Exploring how authors navigate peer review feedback. MEDICAL EDUCATION 2023; 57:151-160. [PMID: 36031758 DOI: 10.1111/medu.14932] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 05/23/2022] [Revised: 08/15/2022] [Accepted: 08/25/2022] [Indexed: 06/15/2023]
Abstract
BACKGROUND Peer review aims to provide meaningful feedback to research authors so that they may improve their work, and yet it constitutes a particularly challenging context for the exchange of feedback. We explore how research authors navigate the process of interpreting and responding to peer review feedback, in order to elaborate how feedback functions when some of the conditions thought to be necessary for it to be effective are not met. METHODS Using constructivist grounded theory methodology, we interviewed 17 recently published health professions education researchers about their experiences with the peer review process. Data collection and analysis were concurrent and iterative. We used constant comparison to identify themes and to develop a conceptual model of how feedback functions in this setting. RESULTS Although participants expressed faith in peer review, they acknowledged that the process was emotionally trying and raised concerns about its consistency and credibility. These potential threats were mitigated by factors including time, team support, experience and the exercise of autonomy. Additionally, the perceived engagement of reviewers and the cultural norms and expectations surrounding the process strengthened authors' willingness and capacity to respond productively. Our analysis suggests a model of feedback within which its perceived usefulness turns on the balance of threats and countermeasures. CONCLUSIONS Feedback is a balancing act. Although threats to the productive uptake of peer review feedback abound, these threats may be neutralised by a range of countermeasures. Among these, opportunities for autonomy and cultural normalisation of both the professional responsibility to engage with feedback and the challenge of doing so may be especially influential and may have implications beyond the peer review setting.
Collapse
Affiliation(s)
- Christopher Watling
- Centre for Education Research and Innovation, Schulich School of Medicine and Dentistry, Western University, London, Ontario, Canada
| | - Jennifer Shaw
- Centre for Education Research and Innovation, Schulich School of Medicine and Dentistry, Western University, London, Ontario, Canada
| | - Emily Field
- Centre for Education Research and Innovation, Schulich School of Medicine and Dentistry, Western University, London, Ontario, Canada
| | - Shiphra Ginsburg
- Department of Medicine, University of Toronto, Toronto, Ontario, Canada
| |
Collapse
|
64
|
van der Winden D, van Dijk N, Visser MRM, Bont J. Walking the line between assessment, improvement and learning: a qualitative study on opportunities and risks of incorporating peer discussion of audit and feedback within quality improvement in general practice. BMJ Open 2023; 13:e066793. [PMID: 36720571 PMCID: PMC9890762 DOI: 10.1136/bmjopen-2022-066793] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 02/02/2023] Open
Abstract
OBJECTIVES There is a broad call for change towards 'new era' quality systems in healthcare, in which the focus lies on learning and improving. A promising way to establish this in general practice care is to combine audit and feedback with peer group discussion. However, it is not known what different stakeholders think of this type of quality improvement. The aim of this research was to explore the opinions of different stakeholders in general practice on peer discussion of audit and feedback and on its opportunities and risks. Second, their thoughts on transparency versus accountability, regarding this system, were studied. DESIGN An exploratory qualitative study within a constructivist paradigm. Semistructured interviews and focus group discussions were held and coded using thematic analysis. Included stakeholders were general practitioners (GP), patients, professional organisations and insurance companies. SETTING General practice in the Netherlands. PARTICIPANTS 22 participants were purposively sampled for eight interviews and two focus group discussions. RESULTS Three main opportunities of peer discussion of audit and feedback were identified: deeper levels of reflection on data, adding context to numbers and more ownership; and three main risks: handling of unwilling colleagues, lacking a safe group and the necessity of patient involvement. An additional theme concerned disagreement on the amount of transparency to be offered: insurance companies and patients advocated for complete transparency on data and improvement of outcomes, while GPs and professional organisations urged to restrict transparency to giving insight into the process. CONCLUSIONS Peer discussion of audit and feedback could be part of a change movement, towards a quality system based on learning and trust, that is initiated by the profession. Creating a safe learning environment and involving patients is key herein. Caution is needed when complete transparency is asked, since it could jeopardise practitioners' reflection and learning in safety.
Collapse
Affiliation(s)
- Dorien van der Winden
- Department of General Practice, Amsterdam UMC Locatie AMC, Amsterdam, Netherlands
- Methodology and Quality of Care, Amsterdam Public Health, Amsterdam, The Netherlands
| | - Nynke van Dijk
- Department of General Practice, Amsterdam UMC Locatie AMC, Amsterdam, Netherlands
- Personalized Medicine and Quality of Care, Amsterdam Public Health, Amsterdam, The Netherlands
- Faculty of Health and Faculty of Sports and Nutrition, Center of Expertise Urban Vitality, Amsterdam University of Applied Sciences, Amsterdam, The Netherlands
| | - Mechteld R M Visser
- Department of General Practice, Amsterdam UMC Locatie AMC, Amsterdam, Netherlands
| | - Jettie Bont
- Department of General Practice, Amsterdam UMC Locatie AMC, Amsterdam, Netherlands
| |
Collapse
|
65
|
Kaur D, Negi G, Jain A, Meinia SK, Sidhu TK, Rathore BB. Structured feedback: A teaching and learning tool for the postgraduate medical students in transfusion medicine. Asian J Transfus Sci 2023; 17:13-20. [PMID: 37188008 PMCID: PMC10180804 DOI: 10.4103/ajts.ajts_127_21] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/02/2021] [Revised: 10/22/2021] [Accepted: 11/16/2021] [Indexed: 11/04/2022] Open
Abstract
CONTEXT Structured Feedback is a learning and assessment tool designed to provide feedback to students and educators to adjust learning and teaching during the training. Lack of provision of structured feedback to postgraduate (PG) medical students prompted us to plan a study to introduce a structured feedback module into the existing monthly assessment schedules in the Department of Transfusion Medicine. AIM This study aims to introduce a structured feedback module and evaluate its efficacy after incorporation into the existing monthly assessment schedules for the PG students in the Department of Transfusion Medicine. DESIGN AND SETTING A quasi-experimental study was commenced after obtaining clearance from the Institutional Ethics Committee in the Department of Transfusion Medicine for the students pursuing postgraduation in Transfusion Medicine. METHODOLOGY A peer-validated feedback module was designed and implemented for MD students by the core team faculty. The students underwent the structured feedback sessions after each monthly assessment for of 3 months. One on one, verbal feedback was conducted using Pendleton's method, for monthly online assessment for the learning that happened during study period. DATA COLLECTION AND STATISTICAL ANALYSIS The data were collected from the open-ended and closed-ended questions using Google form-based Student/Faculty perception and students' pre-post self-efficacy questionnaires on 5-point Likert Scale and the quantitative data analysis was done using percentage of Likert scores, median values for each item for pre-and post-responses and comparison using nonparametric test - Wilcoxon signed-rank test. The qualitative data analysis was done using thematic analysis from the open ended questions. RESULTS All (n = 9; 100%) the PG students strongly agreed and agreed (median score of 5 and 4) that the feedback they received made them aware of their learning gaps, enabled them in bridging those gaps and provided ample opportunity to interact with faculty. Both students and faculty agreed that the feedback session should be an ongoing and continuous process in the department. CONCLUSION Both the students as well as faculty were satisfied with the implementation of the feedback module in the department. Students reported awareness about the learning gaps, identification of appropriate study resources, and ample opportunity to interact with faculty, after taking the feedback sessions. The faculty felt satisfied on the acquisition of new skill for delivering structured feedback to students.
Collapse
Affiliation(s)
- Daljit Kaur
- Department of Transfusion Medicine, All India Institute of Medical Sciences, Rishikesh, Uttarakhand, India
| | - Gita Negi
- Department of Transfusion Medicine, All India Institute of Medical Sciences, Rishikesh, Uttarakhand, India
| | - Ashish Jain
- Department of Transfusion Medicine, All India Institute of Medical Sciences, Rishikesh, Uttarakhand, India
| | - Sushant Kumar Meinia
- Department of Transfusion Medicine, All India Institute of Medical Sciences, Rishikesh, Uttarakhand, India
| | - Tanvir Kaur Sidhu
- Department of Community Medicine, Adesh Institute of Medical Sciences and Research, Bathinda, Punjab, India
| | - Bharti Bhandari Rathore
- Department of Physiology, Government Institute of Medical Sciences, Greater Noida, Uttar Pradesh, India
| |
Collapse
|
66
|
Jamil N, Belkacem AN, Lakas A. On enhancing students' cognitive abilities in online learning using brain activity and eye movements. EDUCATION AND INFORMATION TECHNOLOGIES 2023; 28:4363-4397. [PMID: 36277512 PMCID: PMC9574174 DOI: 10.1007/s10639-022-11372-2] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Received: 07/05/2022] [Accepted: 09/19/2022] [Indexed: 05/11/2023]
Abstract
The COVID-19 pandemic has interrupted education institutions in over 150 nations, affecting billions of students. Many governments have forced a transition in higher education from in-person to remote learning. After this abrupt, worldwide transition away from the classroom, some question whether online education will continue to grow in acceptance in post-pandemic times. However, new technology, such as the brain-computer interface and eye-tracking, have the potential to improve the remote learning environment, which currently faces several obstacles and deficiencies. Cognitive brain computer interfaces can help us develop a better understanding of brain functions, allowing for the development of more effective learning methodologies and the enhancement of brain-based skills. We carried out a systematic literature review of research on the use of brain computer interfaces and eye-tracking to measure students' cognitive skills during online learning. We found that, because many experimental tasks depend on recorded rather than real-time video, students don't have direct and real-time interaction with their teacher. Further, we found no evidence in any of the reviewed papers for brain-to-brain synchronization during remote learning. This points to a potentially fruitful future application of brain computer interfaces in education, investigating whether the brains of student-teacher pairs who interact with the same course content have increasingly similar brain patterns.
Collapse
Affiliation(s)
- Nuraini Jamil
- Department of Computer Science and Software Engineering, College of Information Technology, UAE University, P.O. Box 15551 Abu Dhabi Al-Ain, UAE
| | - Abdelkader Nasreddine Belkacem
- Department of Computer and Network Engineering, College of Information Technology, UAE University, P.O. Box 15551 Abu Dhabi Al-Ain, UAE
| | - Abderrahmane Lakas
- Department of Computer and Network Engineering, College of Information Technology, UAE University, P.O. Box 15551 Abu Dhabi Al-Ain, UAE
| |
Collapse
|
67
|
McDonald J, Ryan S, Heeneman S, Hu W. Informed self-assessment during the transition to medical school: a longitudinal narrative study. BMJ Open 2022; 12:e065203. [PMID: 36581430 PMCID: PMC9806099 DOI: 10.1136/bmjopen-2022-065203] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 12/31/2022] Open
Abstract
OBJECTIVES To explore how medical students' narratives of informed self-assessment (ISA) change during their first 18 months of study. DESIGN This longitudinal study used student narratives drawn from qualitative interviews and written reflections during the transition to medical school, to examine changes in ISA. Our analysis was informed by Situated Cognition Theory which recognises the impact and interplay of personal and environmental factors in cognition. SETTING To study medicine, first year students need to adapt their self-regulated learning in the context of a new peer group, study demands and educational culture. During this adaptation, students need to seek and interpret available cues to inform their self-assessment. PARTICIPANTS Longitudinal data were collected at five time points over 18 months from a diverse sample of seven first year medical students in an undergraduate medical programme, including 13.5 hours of interviews and 12 written reflections. RESULTS Before and after starting medical school, the participants' self-assessments were informed by environmental influences (exam results and comparison with peers), and personal influences (fear of failure and anxiety about not belonging). Early uncertainty meant self-assessments were overestimated and underestimated.By the end of first year, an enhanced sense of belonging coincided with less fear of failure, less emphasis on exam performance and reduced competition with peers. Self-assessments became increasingly informed by evidence of clinical skills and knowledge gained related to future professional competence. CONCLUSION Influences on medical students' ISAs change during the transition to studying medicine. A greater sense of belonging, and evidence of progress towards clinical competence became more important to self-assessment than comparison with peers and exam performance. Our findings reinforce the importance of formative assessments, opportunities to study and socialise with peers and early clinical experiences during first year. These experiences enhance ISA skills during the transition to medical school.
Collapse
Affiliation(s)
- Jenny McDonald
- School of Medicine, Western Sydney University, Penrith South, New South Wales, Australia
| | - Samantha Ryan
- School of Medicine, Western Sydney University, Penrith South, New South Wales, Australia
| | - Sylvia Heeneman
- Department of Pathology, Maastricht University, Maastricht, Netherlands
| | - Wendy Hu
- School of Medicine, Western Sydney University, Penrith South, New South Wales, Australia
| |
Collapse
|
68
|
Prescott-Clements L, Soreskog-Turp J, Crawford B, Williams K. The Development and Implementation of a National Veterinary Graduate Development Programme (VetGDP) to Support Veterinarians Entering the UK Workforce. JOURNAL OF VETERINARY MEDICAL EDUCATION 2022:e20220112. [PMID: 36538366 DOI: 10.3138/jvme-2022-0112] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/17/2023]
Abstract
The UK veterinary profession is facing significant challenges, including high rates of veterinarians leaving the profession amidst workforce shortages, alongside high levels of dissatisfaction, stress, and poor mental health. The highest rates of attrition are associated with veterinarians who have recently graduated and are at an early stage in their careers. Although there may be many contributory factors, a lack of adequate support during the transition from vet school into their first professional role following graduation may be one important cause. Consequently, it has never been more important to develop an effective system for supporting new graduates, which is accessible to all. A new Veterinary Graduate Development Programme (VetGDP) has been developed, using a framework of professional activities that are sufficiently flexible to create a bespoke, individualized program for each graduate depending on the role they enter. Each new graduate is assigned a dedicated coach (Adviser) within their workplace, who has been trained to provide effective support and has committed to doing so throughout the program. VetGDP has been implemented on a national scale in the UK; engagement is assured through the RCVS Code of Professional Conduct and quality assurance is in place. VetGDP has been developed using established educational, sociocultural, and behavioral theories, and the latest research in coaching and feedback within medical education. These methods, which aim to ensure there is the best possible impact on graduates' professional development, and the creation of a positive learning culture within the workplace, are taught to all Advisers via an e-learning package.
Collapse
Affiliation(s)
- Linda Prescott-Clements
- Royal College of Veterinary Surgeons (RCVS), The Cursitor, 38 Chancery Lane, London, WC2A 1EN UK
| | - Jenny Soreskog-Turp
- Royal College of Veterinary Surgeons (RCVS), The Cursitor, 38 Chancery Lane, London, WC2A 1EN UK
| | - Britta Crawford
- Royal College of Veterinary Surgeons (RCVS), The Cursitor, 38 Chancery Lane, London, WC2A 1EN UK
| | - Kirsty Williams
- Royal College of Veterinary Surgeons (RCVS), The Cursitor, 38 Chancery Lane, London, WC2A 1EN UK
| |
Collapse
|
69
|
Jamieson J, Gibson S, Hay M, Palermo C. Teacher, Gatekeeper, or Team Member: supervisor positioning in programmatic assessment. ADVANCES IN HEALTH SCIENCES EDUCATION : THEORY AND PRACTICE 2022:10.1007/s10459-022-10193-9. [PMID: 36469231 DOI: 10.1007/s10459-022-10193-9] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Received: 07/18/2022] [Accepted: 11/27/2022] [Indexed: 06/17/2023]
Abstract
Competency-based assessment is undergoing an evolution with the popularisation of programmatic assessment. Fundamental to programmatic assessment are the attributes and buy-in of the people participating in the system. Our previous research revealed unspoken, yet influential, cultural and relationship dynamics that interact with programmatic assessment to influence success. Pulling at this thread, we conducted secondary analysis of focus groups and interviews (n = 44 supervisors) using the critical lens of Positioning Theory to explore how workplace supervisors experienced and perceived their positioning within programmatic assessment. We found that supervisors positioned themselves in two of three ways. First, supervisors universally positioned themselves as a Teacher, describing an inherent duty to educate students. Enactment of this position was dichotomous, with some supervisors ascribing a passive and disempowered position onto students while others empowered students by cultivating an egalitarian teaching relationship. Second, two mutually exclusive positions were described-either Gatekeeper or Team Member. Supervisors positioning themselves as Gatekeepers had a duty to protect the community and were vigilant to the detection of inadequate student performance. Programmatic assessment challenged this positioning by reorientating supervisor rights and duties which diminished their perceived authority and led to frustration and resistance. In contrast, Team Members enacted a right to make a valuable contribution to programmatic assessment and felt liberated from the burden of assessment, enabling them to assent power shifts towards students and the university. Identifying supervisor positions revealed how programmatic assessment challenged traditional structures and ideologies, impeding success, and provides insights into supporting supervisors in programmatic assessment.
Collapse
Affiliation(s)
- Janica Jamieson
- Monash University, Melbourne, Australia.
- School of Medical and Health Sciences, Edith Cowan University, 270 Joondalup Drive, Joondalup, WA, 6027, Australia.
| | | | | | | |
Collapse
|
70
|
Mooney CJ, Pascoe JM, Blatt AE, Lang VJ, Kelly MS, Braun MK, Burch JE, Stone RT. Predictors of faculty narrative evaluation quality in medical school clerkships. MEDICAL EDUCATION 2022; 56:1223-1231. [PMID: 35950329 DOI: 10.1111/medu.14911] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 05/02/2022] [Revised: 08/01/2022] [Accepted: 08/08/2022] [Indexed: 06/15/2023]
Abstract
INTRODUCTION Narrative approaches to assessment provide meaningful and valid representations of trainee performance. Yet, narratives are frequently perceived as vague, nonspecific and low quality. To date, there is little research examining factors associated with narrative evaluation quality, particularly in undergraduate medical education. The purpose of this study was to examine associations of faculty- and student-level characteristics with the quality of faculty member's narrative evaluations of clerkship students. METHODS The authors reviewed faculty narrative evaluations of 50 students' clinical performance in their inpatient medicine and neurology clerkships, resulting in 165 and 87 unique evaluations in the respective clerkships. The authors evaluated narrative quality using the Narrative Evaluation Quality Instrument (NEQI). The authors used linear mixed effects modelling to predict total NEQI score. Explanatory covariates included the following: time to evaluation completion, number of weeks spent with student, faculty total weeks on service per year, total faculty years in clinical education, student gender, faculty gender, and an interaction term between student and faculty gender. RESULTS Significantly higher narrative evaluation quality was associated with a shorter time to evaluation completion, with NEQI scores decreasing by approximately 0.3 points every 10 days following students' rotations (p = .004). Additionally, women faculty had statistically higher quality narrative evaluations with NEQI scores 1.92 points greater than men faculty (p = .012). All other covariates were not significant. CONCLUSIONS The quality of faculty members' narrative evaluations of medical students was associated with time to evaluation completion and faculty gender but not faculty experience in clinical education, faculty weeks on service, or the amount of time spent with students. Findings advance understanding on ways to improve the quality of narrative evaluations which are imperative given assessment models that will increase the volume and reliance on narratives.
Collapse
Affiliation(s)
- Christopher J Mooney
- School of Medicine and Dentistry, University of Rochester, Rochester, New York, USA
| | - Jennifer M Pascoe
- School of Medicine and Dentistry, University of Rochester, Rochester, New York, USA
| | - Amy E Blatt
- School of Medicine and Dentistry, University of Rochester, Rochester, New York, USA
| | - Valerie J Lang
- School of Medicine and Dentistry, University of Rochester, Rochester, New York, USA
| | | | - Melanie K Braun
- School of Medicine and Dentistry, University of Rochester, Rochester, New York, USA
| | - Jaclyn E Burch
- School of Medicine and Dentistry, University of Rochester, Rochester, New York, USA
| | | |
Collapse
|
71
|
Kaleem SZ, Sahni VN, Suresh A, Duke P. Beyond Competency: A Student Perspective on Growth Through Clerkship Feedback. MEDICAL SCIENCE EDUCATOR 2022; 32:1527-1533. [PMID: 36532401 PMCID: PMC9755405 DOI: 10.1007/s40670-022-01628-4] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Accepted: 09/06/2022] [Indexed: 06/17/2023]
Abstract
This article proposes a paradigm shift from the competency-based model of clerkship feedback using checklists to a coaching-based, action plan-oriented process that centers on individualized student-oriented goals. Using a student perspective, the authors examine the feedback literature and put forward a proposal to use an impact model whose emphasis is to improve the learning climate for students. Several techniques are reviewed which include goal generation and creation of dynamic action plans. By intentionally focusing on coaching relationships as a platform for feedback, the learners and mentors share goals and the result of feedback becomes action-based behaviors which may help negate personal attribution and bias in the feedback process.
Collapse
Affiliation(s)
- Syed Z. Kaleem
- Department of Psychiatry and Behavioral Health, Pennsylvania State University College of Medicine, 700 HMC Crescent Road, Hershey, PA 17033 USA
| | - Vikram N. Sahni
- Department of Dermatology, University of Utah School of Medicine, Salt Lake City, UT USA
| | - Aishwarya Suresh
- Department of Otolaryngology, Loma Linda University School of Medicine, Loma Linda, CA USA
| | - Pamela Duke
- Division of Medical Education, Drexel University College of Medicine, Philadelphia, PA USA
| |
Collapse
|
72
|
Branfield Day L, Rassos J, Billick M, Ginsburg S. 'Next steps are…': An exploration of coaching and feedback language in EPA assessment comments. MEDICAL TEACHER 2022; 44:1368-1375. [PMID: 35944554 DOI: 10.1080/0142159x.2022.2098098] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/15/2023]
Abstract
PURPOSE Entrustable Professional Activities (EPA) assessments are intended to facilitate meaningful, low-stakes coaching and feedback, partly through the provision of written comments. We sought to explore EPA assessment comments provided to internal medicine (IM) residents for evidence of feedback and coaching language as well as politeness. METHODS We collected all written comments from EPA assessments of communication from a first-year IM resident cohort at the University of Toronto. Sensitized by politeness theory, we analyzed data using principles of constructivist grounded theory. RESULTS Nearly all EPA assessments (94%) contained written feedback based on focused clinical encounters. The majority of comments demonstrated coaching language, including phrases like 'don't forget to,' and 'next steps are,' followed by specific suggestions for improvement. A variety of words, including 'autonomy' and 'independence' denoted entrustment decisions. Linguistic politeness strategies such as hedging were pervasive, seemingly to minimize harm to the supervisor-trainee relationship. CONCLUSION Evidence of written coaching feedback suggests that EPA assessment comments are being used as intended as a means of formative feedback to promote learning. Yet, the frequent use of polite language suggests that EPAs may be higher-stakes than expected, highlighting a need for changes to the assessment culture and improved feedback literacy.
Collapse
Affiliation(s)
- Leora Branfield Day
- Department of Medicine, Temerty Faculty of Medicine, University of Toronto, Toronto, Canada
| | - James Rassos
- Department of Medicine, Temerty Faculty of Medicine, University of Toronto, Toronto, Canada
| | - Maxime Billick
- Department of Medicine, Temerty Faculty of Medicine, University of Toronto, Toronto, Canada
| | - Shiphra Ginsburg
- Department of Medicine, Temerty Faculty of Medicine, University of Toronto, Toronto, Canada
- Wilson Centre for Research in Education, Toronto, Canada
| |
Collapse
|
73
|
Lu AD, Atkinson AR, Johnstone JC. Understanding the near-peer relationship: resident perspectives around a novel on-call workplace-based assessment. CANADIAN MEDICAL EDUCATION JOURNAL 2022; 13:73-79. [PMID: 36440082 PMCID: PMC9684053 DOI: 10.36834/cmej.73164] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Indexed: 06/16/2023]
Abstract
BACKGROUND Workplace-based assessment (WBA) is a critical component of competency-based medical education (CBME), though literature on WBA for overnight call is limited. We evaluated a WBA tool completed by supervising subspecialty trainees on paediatric residents during subspecialty overnight call, for usefulness facilitating feedback/coaching in this setting. METHODS Web-based surveys were sent to residents pre- and post-WBA tool implementation monthly for four months (August-December 2018), exploring feedback frequency, Likert-scaled opinions of tool feasibility/usefulness facilitating feedback, and qualitative experiences. Assessor comments were categorized as actionable/non-actionable. Quantitative data was summarized using descriptive statistics. Qualitative data was coded to identify themes. RESULTS Total response rates averaged 41% (total 25 responses, average five respondents/12 residents on-call each month). Post-implementation (n = 16 responses), a non-sustained trend of increased Medical Expert feedback was observed. Residents were generally divided or disagreed on tool usefulness facilitating feedback and feasibility. Comments contained actionable feedback in < 10% of completed WBAs. Qualitative analysis revealed barriers to tool-facilitated coaching including: feedback quality and setting/environment, role of senior near-peer as assessor, interpersonal burden in encounters, and tool-specific issues. CONCLUSIONS Increasing frequency of WBA tool completion is not sufficient to achieve CBME goals. Factors impacting feedback/coaching within the resident/near-peer dyad must be addressed.
Collapse
Affiliation(s)
- Amy D Lu
- Postgraduate Medical Education, Paediatrics, University of Toronto, Ontario, Canada
| | - Adelle R Atkinson
- Postgraduate Medical Education, Paediatrics, University of Toronto, Ontario, Canada
- Department of Paediatrics, The Hospital for Sick Children, Ontario, Canada
| | - Julie C Johnstone
- Department of Paediatrics, The Hospital for Sick Children, Ontario, Canada
| |
Collapse
|
74
|
Woods R, Singh S, Thoma B, Patocka C, Cheung W, Monteiro S, Chan TM. Validity evidence for the Quality of Assessment for Learning score: a quality metric for supervisor comments in Competency Based Medical Education. CANADIAN MEDICAL EDUCATION JOURNAL 2022; 13:19-35. [PMID: 36440075 PMCID: PMC9684040 DOI: 10.36834/cmej.74860] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Indexed: 06/16/2023]
Abstract
BACKGROUND Competency based medical education (CBME) relies on supervisor narrative comments contained within entrustable professional activities (EPA) for programmatic assessment, but the quality of these supervisor comments is unassessed. There is validity evidence supporting the QuAL (Quality of Assessment for Learning) score for rating the usefulness of short narrative comments in direct observation. OBJECTIVE We sought to establish validity evidence for the QuAL score to rate the quality of supervisor narrative comments contained within an EPA by surveying the key end-users of EPA narrative comments: residents, academic advisors, and competence committee members. METHODS In 2020, the authors randomly selected 52 de-identified narrative comments from two emergency medicine EPA databases using purposeful sampling. Six collaborators (two residents, two academic advisors, and two competence committee members) were recruited from each of four EM Residency Programs (Saskatchewan, McMaster, Ottawa, and Calgary) to rate these comments with a utility score and the QuAL score. Correlation between utility and QuAL score were calculated using Pearson's correlation coefficient. Sources of variance and reliability were calculated using a generalizability study. RESULTS All collaborators (n = 24) completed the full study. The QuAL score had a high positive correlation with the utility score amongst the residents (r = 0.80) and academic advisors (r = 0.75) and a moderately high correlation amongst competence committee members (r = 0.68). The generalizability study found that the major source of variance was the comment indicating the tool performs well across raters. CONCLUSION The QuAL score may serve as an outcome measure for program evaluation of supervisors, and as a resource for faculty development.
Collapse
Affiliation(s)
- Rob Woods
- Department of Emergency Medicine, University of Saskatchewan, Saskatchewan, Canada
| | - Sim Singh
- College of Medicine, University of Saskatchewan, Saskatchewan, Canada
| | - Brent Thoma
- Department of Emergency Medicine, University of Saskatchewan, Saskatchewan, Canada
| | - Catherine Patocka
- Department of Emergency Medicine, University of Calgary, Alberta, Canada
| | - Warren Cheung
- Department of Emergency Medicine, University of Ottawa, Ontario, Canada
| | - Sandra Monteiro
- Department of Health Research Methods Evidence and Impact, McMaster University, Ontario, Canada
| | - Teresa M Chan
- Division of Emergency Medicine and Education & Innovation, Department of Medicine, McMaster University, Ontario, Canada
| | | |
Collapse
|
75
|
Sebok-Syer SS, Shaw JM, Sedran R, Shepherd L, McConnell A, Dukelow AM, Syer MD, Lingard L. Facilitating Residents' Understanding of Electronic Health Record Report Card Data Using Faculty Feedback and Coaching. ACADEMIC MEDICINE : JOURNAL OF THE ASSOCIATION OF AMERICAN MEDICAL COLLEGES 2022; 97:S22-S28. [PMID: 35947480 DOI: 10.1097/acm.0000000000004900] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/15/2023]
Abstract
PURPOSE Feedback continues to present a challenge for competency-based medical education. Clear, consistent, and credible feedback is vital to supporting one's ongoing development, yet it can be difficult to gather clinical performance data about residents. This study sought to determine whether providing residents with electronic health record (EHR)-based report cards, as well as an opportunity to discuss these data with faculty trained using the R2C2 model, can help residents understand and interpret their clinical performance metrics. METHOD Using action research methodology, the author team collected EHR data from July 2017 to February 2020, for all residents (n = 21) in one 5-year Emergency Medicine program and created personalized report cards for each resident. During October 6-17, 2020, 8 out of 17 eligible residents agreed to have their feedback conversations recorded and participate in a subsequent interview with a nonphysician member of the research team. Data were analyzed using thematic analysis, and the authors used inductive analysis to identify themes in the data. RESULTS In analyzing both the feedback conversations as well as the individual interviews with faculty and residents, the authors identified 2 main themes: (1) Reactions and responses to receiving personalized EHR data and (2) The value of EHR data for assessment and feedback purposes. All participants believed that EHR data metrics are useful for prompting self-reflection, and many pointed to their utility in providing suggestions for actionable changes in their clinical practice. For faculty, having a tool through which underperforming residents can be shown "objective" data about their clinical performance helps underscore the need for improvement, particularly when residents are resistant. CONCLUSIONS The EHR is a valuable source of educational data, and this study demonstrates one of the many thoughtful ways it can be used for assessment and feedback purposes.
Collapse
Affiliation(s)
- Stefanie S Sebok-Syer
- S.S. Sebok-Syer is assistant professor, Department of Emergency Medicine, Stanford University School of Medicine, Palo Alto, California; ORCID: http://orcid.org/0000-0002-3572-5971
| | - Jennifer M Shaw
- J.M. Shaw is research associate, Centre for Education, Research, and Innovation, Schulich School of Medicine and Dentistry, Western University, London, Ontario, Canada
| | - Robert Sedran
- R. Sedran is associate professor, Division of Emergency Medicine, Schulich School of Medicine and Dentistry, London, Ontario, Canada
| | - Lisa Shepherd
- L. Shepherd is professor, Division of Emergency Medicine, Schulich School of Medicine and Dentistry, London, Ontario, Canada
| | - Allison McConnell
- A. McConnell is assistant professor, Division of Emergency Medicine, Schulich School of Medicine and Dentistry, London, Ontario, Canada
| | - Adam M Dukelow
- A.M. Dukelow is associate professor, Division of Emergency Medicine, Schulich School of Medicine and Dentistry, London, Ontario, Canada
| | - Mark D Syer
- M.D. Syer is an affiliate, School of Computing, Queen's University, Kingston, Ontario, Canada
| | - Lorelei Lingard
- L. Lingard is professor, Department of Medicine and Faculty of Education, and senior scientist, Centre for Education, Research, and Innovation, Schulich School of Medicine and Dentistry, Western University, London, Ontario, Canada; ORCID: http://orcid.org/0000-0002-1524-0723
| |
Collapse
|
76
|
Pippitt KA, Moore KB, Lindsley JE, Cariello PF, Smith AG, Formosa T, Moser K, Morton DA, Colbert-Getz JM, Chow CJ. Assessment for Learning with Ungraded and Graded Assessments. MEDICAL SCIENCE EDUCATOR 2022; 32:1045-1054. [PMID: 36276764 PMCID: PMC9584017 DOI: 10.1007/s40670-022-01612-y] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Accepted: 08/25/2022] [Indexed: 06/16/2023]
Abstract
Introduction Assessment for learning has many benefits, but learners will still encounter high-stakes decisions about their performance throughout training. It is unknown if assessment for learning can be promoted with a combination model where scores from some assessments are factored into course grades and scores from other assessments are not used for course grading. Methods At the University of Utah School of Medicine, year 1-2 medical students (MS) completed multiple-choice question quiz assessments and final examinations in six systems-based science courses. Quiz and final examination performance counted toward course grades for MS2017-MS2018. Starting with the MS2020 cohort, quizzes no longer counted toward course grades. Quiz, final examination, and Step 1 scores were compared between ungraded quiz and graded quiz cohorts with independent samples t-tests. Student and faculty feedback was collected. Results Quiz performance was not different for the ungraded and graded cohorts (p = 0.173). Ungraded cohorts scored 4% higher on final examinations than graded cohorts (p ≤ 0.001, d = 0.88). Ungraded cohorts scored above the national average and 11 points higher on Step 1 compared to graded cohorts, who had scored below the national average (p ≤ 0.001, d = 0.64). During the study period, Step 1 scores increased by 2 points nationally. Student feedback was positive, and faculty felt it improved their relationship with students. Discussion The change to ungraded quizzes did not negatively affect final examination or Step 1 performance, suggesting a combination of ungraded and graded assessments can effectively promote assessment for learning.
Collapse
Affiliation(s)
- Karly A. Pippitt
- Department of Family and Preventive Medicine, University of Utah School of Medicine, 317 Chipeta Way, Suite A, Salt Lake City, UT 84108 USA
- Community Faculty, University of Utah School of Medicine, Salt Lake City, UT USA
| | - Kathryn B. Moore
- Department of Neurobiology, University of Utah School of Medicine, Salt Lake City, UT USA
| | - Janet E. Lindsley
- Department of Biochemistry, University of Utah School of Medicine, Salt Lake City, UT USA
- Curriculum University of Utah School of Medicine, Salt Lake City, UT USA
| | - Paloma F. Cariello
- Department of Internal Medicine, University of Utah School of Medicine, Salt Lake City, UT USA
- Health Equity, Diversity, and Inclusion, University of Utah School of Medicine, Salt Lake City, UT USA
| | - Andrew G. Smith
- Department of Pediatrics, University of Utah School of Medicine, Salt Lake City, UT USA
| | - Tim Formosa
- Department of Biochemistry, University of Utah School of Medicine, Salt Lake City, UT USA
| | - Karen Moser
- Department of Pathology, University of Utah School of Medicine, Salt Lake City, UT USA
| | - David A. Morton
- Department of Neurobiology, University of Utah School of Medicine, Salt Lake City, UT USA
| | - Jorie M. Colbert-Getz
- Department of Internal Medicine, University of Utah School of Medicine, Salt Lake City, UT USA
- Education Quality Improvement, University of Utah School of Medicine, Salt Lake City, UT USA
| | - Candace J. Chow
- Department of Internal Medicine, University of Utah School of Medicine, Salt Lake City, UT USA
- Education Research, University of Utah School of Medicine, Salt Lake City, UT USA
| |
Collapse
|
77
|
Menchetti I, Eagles D, Ghanem D, Leppard J, Fournier K, Cheung WJ. Gender differences in emergency medicine resident assessment: A scoping review. AEM EDUCATION AND TRAINING 2022; 6:e10808. [PMID: 36189450 PMCID: PMC9513437 DOI: 10.1002/aet2.10808] [Citation(s) in RCA: 5] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 07/26/2022] [Revised: 09/05/2022] [Accepted: 09/06/2022] [Indexed: 05/26/2023]
Abstract
Background Growing literature within postgraduate medical education demonstrates that female resident physicians experience gender bias throughout their training and future careers. This scoping review aims to describe the current body of literature on gender differences in emergency medicine (EM) resident assessment. Methods We conducted a scoping review which adhered to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews guidelines. We included research involving resident physicians or fellows in EM (population and context), which focused on the impact of gender on assessments (concept). We searched seven databases from the databases' inception to April 4, 2022. Two reviewers independently screened citations, completed full-text review, and abstracted data. A third reviewer resolved any discrepancies. Results A total of 667 unique citations were identified; 10 studies were included, and all were conducted within the United States. Four studies reported differences in EM resident assessments attributable to gender within workplace-based assessments (qualitative comments and quantitative scores) by both attending physicians and nonphysicians. Six studies investigating clinical competency committee scores, procedural scores, and simulation-based assessments did not report any significant differences attributable to gender. Conclusions This scoping review found that gender bias exists within EM resident assessment most notably at the level of narrative comments typically received via workplace-based assessments. As female EM residents receive higher rates of negative or critical comments and discordant feedback documented on assessment, these findings raise concern about added barriers female EM residents may face while progressing through residency and the impact on their clinical and professional development.
Collapse
Affiliation(s)
| | - Debra Eagles
- Department of Emergency MedicineUniversity of OttawaOttawaOntarioCanada
- School of Epidemiology and Public HealthUniversity of OttawaOttawaOntarioCanada
| | - Dana Ghanem
- Faculty of MedicineUniversity of OttawaOttawaOntarioCanada
| | - Jennifer Leppard
- Department of Emergency MedicineUniversity of OttawaOttawaOntarioCanada
| | - Karine Fournier
- Health Sciences LibraryUniversity of OttawaOttawaOntarioCanada
| | - Warren J. Cheung
- Department of Emergency MedicineUniversity of OttawaOttawaOntarioCanada
- Royal College of Physicians and Surgeons of CanadaOttawaOntarioCanada
| |
Collapse
|
78
|
Lee ASO, Donoff C, Ross S. Using Learning Analytics to Examine Differences in Assessment Forms From Continuous Versus Episodic Supervisors of Family Medicine Residents. J Grad Med Educ 2022; 14:606-612. [PMID: 36274777 PMCID: PMC9580309 DOI: 10.4300/jgme-d-21-00832.1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 08/24/2021] [Revised: 01/29/2022] [Accepted: 06/28/2022] [Indexed: 11/06/2022] Open
Abstract
BACKGROUND It is assumed that there is a need for continuity of supervision within competency-based medical education, despite most evidence coming from the undergraduate medical education rather than the graduate medical education (GME) context. This evidence gap must be addressed to justify the time and effort needed to redesign GME programs to support continuity of supervision. OBJECTIVE To examine differences in assessment behaviors of continuous supervisors (CS) versus episodic supervisors (ES), using completed formative assessment forms, FieldNotes, as a proxy. METHODS The FieldNotes CS- and ES-entered for family medicine residents (N=186) across 3 outpatient teaching sites over 3 academic years (2015-2016, 2016-2017, 2017-2018) were examined using 2-sample proportion z-tests to determine differences on 3 FieldNote elements: competency (Sentinel Habit [SH]), Clinical Domain (CD), and Progress Level (PL). RESULTS Sixty-nine percent (6104 of 8909) of total FieldNotes were analyzed. Higher proportions of CS-entered FieldNotes indicated SH3 (Managing patients with best practices), z=-3.631, P<.0001; CD2 (Care of adults), z=-8.659, P<.0001; CD3 (Care of the elderly), z=-4.592, P<.0001; and PL3 (Carry on, got it), z=-4.482, P<.0001. Higher proportions of ES-entered FieldNotes indicated SH7 (Communication skills), z=4.268, P<.0001; SH8 (Helping others learn), z=20.136, P<.0001; CD1 (Doctor-patient relationship/ethics), z=14.888, P<.0001; CD9 (Not applicable), z=7.180, P<.0001; and PL2 (In progress), z=5.117, P<.0001. CONCLUSIONS The type of supervisory relationship impacts assessment: there is variability in which competencies are paid attention to, which contexts or populations are included, and which progress levels are chosen.
Collapse
Affiliation(s)
- Ann S. O. Lee
- Ann S. O. Lee, MD, MEd, is Assistant Professor, Department of Family Medicine, University of Alberta, Edmonton, Alberta, Canada
| | - Christopher Donoff
- Christopher Donoff, MSc, is Junior Data Scientist, Blackline Safety, Calgary, Alberta, Canada
| | - Shelley Ross
- Shelley Ross, PhD, is Professor, Department of Family Medicine, University of Alberta Edmonton, Alberta, Canada
| |
Collapse
|
79
|
Wong SN, Luo CJ, MacDonald G, Hatala R. A qualitative study of medical students' perceptions of resident feedback. MEDICAL EDUCATION 2022; 56:994-1001. [PMID: 35639522 DOI: 10.1111/medu.14847] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 12/20/2021] [Revised: 05/25/2022] [Accepted: 05/27/2022] [Indexed: 06/15/2023]
Abstract
CONTEXT Residents play a pivotal role in medical students' clinical education. From a feedback lens, the near-peer relationship between student and resident holds the potential to foster an educational alliance that could influence learning. We undertook the current qualitative study to explore medical students' perceptions of feedback experiences with residents, addressing when, how and why (and conversely when not and why not) resident feedback plays a role in their clinical education. METHODS Our methodology was qualitative interpretive description, informed by phenomenology. We conducted 24 semi-structured interviews with third and fourth year medical students at one institution. The interviews aimed to foster rich discussion about students' feedback experiences with residents during clinical rotations. Data collection and analysis proceeded iteratively. Initial interviews were independently open-coded by three investigators and then collaboratively refined. Codes were applied to subsequent interviews, and new codes were developed. During the final stages of analysis, we organised our themes by drawing on a sociocultural perspective to examine students' perceptions of relationship-building with residents and when and how this influenced feedback and learning. RESULTS From the students' perspectives, when residents contributed to building interpersonal relationships with students, this in turn influenced students' receptivity to both encouraging and constructive feedback conversations. In the context of resident-student relationships that were perceived as supportive, resident feedback influenced how students approached learning and working in the clinical environment, as well as students' visions of their future selves. In unsupportive relationships, students were less inclined to engage in feedback with residents and students noted resident behaviours that they wanted to avoid in themselves. CONCLUSION Residents are uniquely positioned to create a strong educational alliance with students in which feedback conversations can flourish. Focusing educational efforts on resident feedback conversations has the potential to significantly impact the feedback culture of our clinical environments.
Collapse
Affiliation(s)
- Shannon N Wong
- Department of Medicine, Faculty of Medicine, University of British Columbia, Vancouver, British Columbia, Canada
| | - Cong John Luo
- Department of Medicine, Faculty of Medicine, University of British Columbia, Vancouver, British Columbia, Canada
| | - Graham MacDonald
- Department of Occupational Science and Occupational Therapy, Faculty of Medicine, University of British Columbia, Vancouver, British Columbia, Canada
| | - Rose Hatala
- Department of Medicine and Centre for Health Education Scholarship, Faculty of Medicine, University of British Columbia, Vancouver, British Columbia, Canada
| |
Collapse
|
80
|
Phinney LB, Fluet A, O'Brien BC, Seligman L, Hauer KE. Beyond Checking Boxes: Exploring Tensions With Use of a Workplace-Based Assessment Tool for Formative Assessment in Clerkships. ACADEMIC MEDICINE : JOURNAL OF THE ASSOCIATION OF AMERICAN MEDICAL COLLEGES 2022; 97:1511-1520. [PMID: 35703235 DOI: 10.1097/acm.0000000000004774] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/15/2023]
Abstract
PURPOSE To understand the role of a workplace-based assessment (WBA) tool in facilitating feedback for medical students, this study explored changes and tensions in a clerkship feedback activity system through the lens of cultural historical activity theory (CHAT) over 2 years of tool implementation. METHOD This qualitative study uses CHAT to explore WBA use in core clerkships by identifying feedback activity system elements (e.g., community, tools, rules, objects) and tensions among these elements. University of California, San Francisco core clerkship students were invited to participate in semistructured interviews eliciting experience with a WBA tool intended to enhance direct observation and feedback in year 1 (2019) and year 2 (2020) of implementation. In year 1, the WBA tool required supervisor completion in the school's evaluation system on a computer. In year 2, both students and supervisors had WBA completion abilities and could access the form via a smartphone separate from the school's evaluation system. RESULTS Thirty-five students participated in interviews. The authors identified tensions that shifted with time and tool iterations. Year 1 students described tensions related to cumbersome tool design, fear of burdening supervisors, confusion over WBA purpose, WBA as checking boxes, and WBA usefulness depending on clerkship context and culture. Students perceived dissatisfaction with the year 1 tool version among peers and supervisors. The year 2 mobile-based tool and student completion capabilities helped to reduce many of the tensions noted in year 1. Students expressed wider WBA acceptance among peers and supervisors in year 2 and reported understanding WBA to be for low-stakes feedback, thereby supporting formative assessment for learning. CONCLUSIONS Using CHAT to explore changes in a feedback activity system with WBA tool iterations revealed elements important to WBA implementation, including designing technology for tool efficiency and affording students autonomy to document feedback with WBAs.
Collapse
Affiliation(s)
- Lauren B Phinney
- L.B. Phinney is a first-year internal medicine resident, Department of Medicine, University of California, San Francisco School of Medicine, San Francisco, California
| | - Angelina Fluet
- A. Fluet is a fourth-year medical student, University of California, San Francisco School of Medicine, San Francisco, California
| | - Bridget C O'Brien
- B.C. O'Brien is professor of medicine and education scientist, Department of Medicine and Center for Faculty Educators, University of California, San Francisco School of Medicine, San Francisco, California
| | - Lee Seligman
- L. Seligman is a second-year internal medicine resident, Department of Medicine, New York-Presbyterian Hospital, Columbia University Irving Medical Center, New York, New York
| | - Karen E Hauer
- K.E. Hauer is associate dean for competency assessment and professional standards and professor, Department of Medicine, University of California, San Francisco School of Medicine, San Francisco, California
| |
Collapse
|
81
|
Roberts C, Khanna P, Bleasel J, Lane S, Burgess A, Charles K, Howard R, O'Mara D, Haq I, Rutzou T. Student perspectives on programmatic assessment in a large medical programme: A critical realist analysis. MEDICAL EDUCATION 2022; 56:901-914. [PMID: 35393668 PMCID: PMC9542097 DOI: 10.1111/medu.14807] [Citation(s) in RCA: 5] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 11/15/2021] [Revised: 03/28/2022] [Accepted: 04/05/2022] [Indexed: 06/14/2023]
Abstract
BACKGROUND Fundamental challenges exist in researching complex changes of assessment practice from traditional objective-focused 'assessments of learning' towards programmatic 'assessment for learning'. The latter emphasise both the subjective and social in collective judgements of student progress. Our context was a purposively designed programmatic assessment system implemented in the first year of a new graduate entry curriculum. We applied critical realist perspectives to unpack the underlying causes (mechanisms) that explained student experiences of programmatic assessment, to optimise assessment practice for future iterations. METHODS Data came from 14 in-depth focus groups (N = 112/261 students). We applied a critical realist lens drawn from Bhasker's three domains of reality (the actual, empirical and real) and Archer's concept of structure and agency to understand the student experience of programmatic assessment. Analysis involved induction (pattern identification), abduction (theoretical interpretation) and retroduction (causal explanation). RESULTS As a complex educational and social change, the assessment structures and culture systems within programmatic assessment provided conditions (constraints and enablements) and conditioning (acceptance or rejection of new 'non-traditional' assessment processes) for the actions of agents (students) to exercise their learning choices. The emergent underlying mechanism that most influenced students' experience of programmatic assessment was one of balancing the complex relationships between learner agency, assessment structures and the cultural system. CONCLUSIONS Our study adds to debates on programmatic assessment by emphasising how the achievement of balance between learner agency, structure and culture suggests strategies to underpin sustained changes (elaboration) in assessment practice. These include; faculty and student learning development to promote collective reflexivity and agency, optimising assessment structures by enhancing integration of theory with practice, and changing learning culture by both enhancing existing and developing new social structures between faculty and the student body to gain acceptance and trust related to the new norms, beliefs and behaviours in assessing for and of learning.
Collapse
Affiliation(s)
- Chris Roberts
- Faculty of Medicine and Health, Sydney Medical School, Education OfficeThe University of SydneySydneyNew South Wales
| | - Priya Khanna
- Faculty of Medicine and Health, Sydney Medical School, Education OfficeThe University of SydneySydneyNew South Wales
| | - Jane Bleasel
- Faculty of Medicine and Health, Sydney Medical School, Education OfficeThe University of SydneySydneyNew South Wales
| | - Stuart Lane
- Faculty of Medicine and Health, Sydney Medical School, Education OfficeThe University of SydneySydneyNew South Wales
| | - Annette Burgess
- Faculty of Medicine and Health, Sydney Medical School, Education OfficeThe University of SydneySydneyNew South Wales
| | - Kellie Charles
- Faculty of Medicine and Health, Sydney Medical School, Education OfficeThe University of SydneySydneyNew South Wales
- Faculty of Medicine and Health, Sydney Pharmacy School, Discipline of PharmacologyThe University of SydneySydneyNew South WalesAustralia
| | - Rosa Howard
- Faculty of Medicine and Health, Sydney Medical School, Education OfficeThe University of SydneySydneyNew South Wales
| | - Deborah O'Mara
- Faculty of Medicine and Health, Sydney Medical School, Education OfficeThe University of SydneySydneyNew South Wales
| | - Inam Haq
- Faculty of Medicine and HealthThe University of SydneySydneyNew South WalesAustralia
| | - Timothy Rutzou
- School of MedicineThe University of Notre DameChippendaleNew South WalesAustralia
| |
Collapse
|
82
|
Huang K, Mak D, Hafferty FW, Eva KW. The Advice Given During Near-Peer Interactions Before and After Curriculum Change. TEACHING AND LEARNING IN MEDICINE 2022; 34:351-359. [PMID: 34524067 DOI: 10.1080/10401334.2021.1957685] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 01/06/2021] [Accepted: 07/02/2021] [Indexed: 06/13/2023]
Abstract
PhenomenonNear-peer interactions (NPIs) provide formal and informal mentorship that can allow medical students to share strategies for successful training. Such capacity to convey valuable advice, however, may depend on the similitude of experiences. Given that many factors can disrupt homogeneity, including curriculum renewal, we must better understand whether junior trainees feel disadvantaged when they do not have more senior peers with similar experiences. This study was, therefore, conducted to examine the focus of, and engagement with, advice available through NPIs during curriculum renewal. Approach: We used a generic exploratory qualitative research approach. Twenty MD undergraduate students, seven from the Class of 2019 (the first cohort post-curriculum change), and thirteen from the Class of 2020 (the first cohort with access to more senior students in the new curriculum), participated in semi-structured interviews. Anonymized transcriptions were analyzed with open, axial, and selective coding to generate themes until saturation was attained. Findings: Participants from the Class of 2019 reported having particularly few reasons to seek advice; because curriculum renewal disrupted their near peers' capacity to provide critical insights, students exerted little effort to learn from them. That said, this vacuum was not generally cause for concern. Deeper probing illustrated why: advice given during NPIs in both classes more commonly focused on nonacademic (e.g., work-life balance issues) than academic advice; academic advice, when sought or offered, tended not to be aimed at improving understanding of curriculum dependent content; and, while students in both classes welcomed advice, both were wary of accepting it at face value, precluding a sense of dependence on senior peers. Insights: Students' skepticism about the overall utility of academic advice raises a number of important issues for medical education and training. Positively, it shielded students from feeling loss when advice from similarly trained students was not available, reducing concerns about disadvantage that could arise during periods of curriculum revision. On the other hand, knowing that what students perceive and what educators claim to be important aspects of training can be at odds and knowing that self-assessment is flawed makes it surprising and unsettling, respectively, that participants so readily treated the lessons learned by those who came before them as irrelevant.
Collapse
Affiliation(s)
- Kelly Huang
- Department of Emergency Medicine, University of British Columbia, Vancouver, British Columbia, Canada
| | - David Mak
- Radiation Oncology, University of Toronto, Toronto, Ontario, Canada
| | - Frederic W Hafferty
- Division of General Internal Medicine and Program in Professionalism and Values, Mayo Clinic, Rochester, Minnesota, USA
| | - Kevin W Eva
- Education Research and Scholarship, University of British Columbia, Vancouver, British Columbia, Canada
| |
Collapse
|
83
|
Torre D, Schuwirth L, Van der Vleuten C, Heeneman S. An international study on the implementation of programmatic assessment: Understanding challenges and exploring solutions. MEDICAL TEACHER 2022; 44:928-937. [PMID: 35701165 DOI: 10.1080/0142159x.2022.2083487] [Citation(s) in RCA: 8] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/15/2023]
Abstract
INTRODUCTION Programmatic assessment is an approach to assessment aimed at optimizing the learning and decision function of assessment. It involves a set of key principles and ground rules that are important for its design and implementation. However, despite its intuitive appeal, its implementation remains a challenge. The purpose of this paper is to gain a better understanding of the factors that affect the implementation process of programmatic assessment and how specific implementation challenges are managed across different programs. METHODS An explanatory multiple case (collective) approach was used for this study. We identified 6 medical programs that had implemented programmatic assessment with variation regarding health profession disciplines, level of education and geographic location. We conducted interviews with a key faculty member from each of the programs and analyzed the data using inductive thematic analysis. RESULTS We identified two major factors in managing the challenges and complexity of the implementation process: knowledge brokers and a strategic opportunistic approach. Knowledge brokers were the people who drove and designed the implementation process acting by translating evidence into practice allowing for real-time management of the complex processes of implementation. These knowledge brokers used a 'strategic opportunistic' or agile approach to recognize new opportunities, secure leadership support, adapt to the context and take advantage of the unexpected. Engaging in an overall curriculum reform process was a critical factor for a successful implementation of programmatic assessment. DISCUSSION The study contributes to the understanding of the intricacies of implementation processes of programmatic assessment across different institutions. Managing opportunities, adaptive planning, awareness of context, were all critical aspects of thinking strategically and opportunistically in the implementation of programmatic assessment. Future research is needed to provide a more in-depth understanding of values and beliefs that underpin the assessment culture of an organization, and how such values may affect implementation.
Collapse
Affiliation(s)
- Dario Torre
- Director of Assessment, and Professor of Medicine, University of Central Florida College of Medicine, Orlando, FL, USA
| | - Lambert Schuwirth
- College of Medicine and Public Health, Flinders University, Adelaide, Australia
| | - Cees Van der Vleuten
- Department of Educational Development and Research, School of Health Profession Education, Maastricht University, Maastricht, The Netherlands
| | - Sylvia Heeneman
- Department of Pathology, School Health Profession Education, Cardiovascular Research Institute Maastricht, Maastricht University, Maastricht, The Netherlands
| |
Collapse
|
84
|
Landreville JM, Wood TJ, Frank JR, Cheung WJ. Does direct observation influence the quality of workplace-based assessment documentation? AEM EDUCATION AND TRAINING 2022; 6:e10781. [PMID: 35903424 PMCID: PMC9305723 DOI: 10.1002/aet2.10781] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 04/14/2022] [Revised: 06/07/2022] [Accepted: 06/08/2022] [Indexed: 05/30/2023]
Abstract
BACKGROUND A key component of competency-based medical education (CBME) is direct observation of trainees. Direct observation has been emphasized as integral to workplace-based assessment (WBA) yet previously identified challenges may limit its successful implementation. Given these challenges, it is imperative to fully understand the value of direct observation within a CBME program of assessment. Specifically, it is not known whether the quality of WBA documentation is influenced by observation type (direct or indirect). METHODS The objective of this study was to determine the influence of observation type (direct or indirect) on quality of entrustable professional activity (EPA) assessment documentation within a CBME program. EPA assessments were scored by four raters using the Quality of Assessment for Learning (QuAL) instrument, a previously published three-item quantitative measure of the quality of written comments associated with a single clinical performance score. An analysis of variance was performed to compare mean QuAL scores among the direct and indirect observation groups. The reliability of the QuAL instrument for EPA assessments was calculated using a generalizability analysis. RESULTS A total of 244 EPA assessments (122 direct observation, 122 indirect observation) were rated for quality using the QuAL instrument. No difference in mean QuAL score was identified between the direct and indirect observation groups (p = 0.17). The reliability of the QuAL instrument for EPA assessments was 0.84. CONCLUSIONS Observation type (direct or indirect) did not influence the quality of EPA assessment documentation. This finding raises the question of how direct and indirect observation truly differ and the implications for meta-raters such as competence committees responsible for making judgments related to trainee promotion.
Collapse
Affiliation(s)
| | - Timothy J. Wood
- Department of Innovation in Medical EducationUniversity of OttawaOttawaOntarioCanada
| | - Jason R. Frank
- Department of Emergency MedicineUniversity of OttawaOttawaOntarioCanada
| | - Warren J. Cheung
- Department of Emergency MedicineUniversity of OttawaOttawaOntarioCanada
| |
Collapse
|
85
|
Kealey A, Naik VN. Competency-Based Medical Training in Anesthesiology: Has It Delivered on the Promise of Better Education? Anesth Analg 2022; 135:223-229. [PMID: 35839492 DOI: 10.1213/ane.0000000000006091] [Citation(s) in RCA: 11] [Impact Index Per Article: 5.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/05/2022]
Affiliation(s)
- Alayne Kealey
- From the Department of Anesthesia, Sunnybrook Health Sciences Centre, Toronto, Ontario, Canada.,Department of Anesthesiology and Pain Medicine, University of Toronto, Toronto, Ontario, Canada
| | - Viren N Naik
- Department of Anesthesiology and Pain Medicine, University of Ottawa, Ottawa, Ontario, Canada
| |
Collapse
|
86
|
McConnell MM. The Importance of Sociocultural Factors in Moderating the Applicability of Test-Enhanced Learning to Continuing Professional Development. THE JOURNAL OF CONTINUING EDUCATION IN THE HEALTH PROFESSIONS 2022; 42:190-196. [PMID: 36007517 DOI: 10.1097/ceh.0000000000000434] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/15/2023]
Abstract
Many studies have shown that repeated testing promotes learning, an effect known as test-enhanced learning. Although test-enhanced learning has been well-established within health care trainees, there are mixed findings of whether testing facilitates learning within continuing professional development (CPD) contexts. This piece argues that for testing to support learning in CPD, there is need to understand the various social, cultural, and structural factors that influence clinicians' willingness to participate in regular testing activities. The solution to this problem is conceptual in nature, whereby CPD programs must consider how learning cultures shape practicing clinicians' perspectives toward testing.
Collapse
Affiliation(s)
- Meghan M McConnell
- Dr. McConnell: Associate Professor, Department of Innovation in Medical Education and Department of Anesthesiology and Pain Medicine, University of Ottawa, Ottawa, Ontario, Canada
| |
Collapse
|
87
|
Trier J, Turnnidge J, McGuire CS, Côté J, Dagnone JD. It's a 'two-way street': resident perspectives of effective coaching relationships in the clinical learning environment. CANADIAN MEDICAL EDUCATION JOURNAL 2022; 13:5-12. [PMID: 35875451 PMCID: PMC9297249 DOI: 10.36834/cmej.72940] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Indexed: 06/15/2023]
Abstract
BACKGROUND Coaching has gained traction in postgraduate medical education to enhance interactions between residents and clinical teachers, but these relationships present unique challenges and tensions. In order to realize the promises of coaching in medical education, we must understand how coaching relationships can be enacted to optimize resident development. The purpose of this study was to explore residents' perceptions of key characteristics for effective clinical teacher-resident (CT-R) coaching relationships. METHODS We conducted four focus groups and eight interviews with residents at a Canadian academic center. Using a social constructionist approach, focus groups and interviews were audio recorded, transcribed verbatim, and thematically analyzed. RESULTS Residents described three main characteristics that contributed to effective CT-R coaching relationships: safe, meaningful, and collaborative. Residents emphasized that these characteristics needed to be bidirectional in nature to be most effective, in that both the resident and clinical teacher embodied these characteristics. CONCLUSIONS Residents identified that effective coaching relationships were shaped not only by clinical teacher behaviours, but importantly, the quality of the interpersonal relationship that was fostered. Thus, it is imperative to consider the bidirectional nature of the CT-R coaching relationship when striving to enhance resident development.
Collapse
Affiliation(s)
- Jessica Trier
- Department of Physical Medicine and Rehabilitation, Queen’s University, Ontario, Canada
- Providence Care Hospital, Ontario, Canada
| | - Jennifer Turnnidge
- Office of Professional Development and Educational Scholarship, Queen’s University, Ontario, Canada
- School of Kinesiology and Health Studies, Queen’s University, Ontario, Canada
| | - Cailie S McGuire
- School of Kinesiology and Health Studies, Queen’s University, Ontario, Canada
| | - Jean Côté
- School of Kinesiology and Health Studies, Queen’s University, Ontario, Canada
| | - J Damon Dagnone
- Department of Emergency Medicine, Queen’s University, Ontario, Canada
- Kingston Health Sciences Centre, Ontario, Canada
| |
Collapse
|
88
|
Hadley Strout EK, Wahlberg EA, Kennedy AG, Tompkins BJ, Sobel HG. A Mixed-Methods Program Evaluation of a Self-directed Learning Panel Management Curriculum in an Internal Medicine Residency Clinic. J Gen Intern Med 2022; 37:2246-2250. [PMID: 35710657 PMCID: PMC9202988 DOI: 10.1007/s11606-022-07507-3] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 08/22/2021] [Accepted: 03/23/2022] [Indexed: 11/30/2022]
Abstract
BACKGROUND Panel management (PM) curricula in internal medicine (IM) residency programs often assign performance measures which may not address the varied interests or needs of resident-learners. AIM To evaluate a self-directed learning (SDL)-based PM curriculum. SETTING University-based primary care practice in Burlington, Vermont. PARTICIPANTS Thirty-five internal medicine residents participated. PROGRAM DESCRIPTION Residents completed a PM curriculum that integrated SDL, electronic health record (EHR)-driven performance feedback, mentorship, and autonomy to set learning and patient care goals. PROGRAM EVALUATION Pre/post-curricular surveys assessed EHR tool acceptability, weekly curricular surveys and post-curricular focus groups assessed resident perceptions and goals, and an interrupted time series analysis of care gap closure rates was used to compare the pre-intervention and intervention periods. Majority of residents (28-32 or 80-91%) completed the surveys and focus groups. Residents found the EHR tools acceptable and valued protected time, mentorship, and autonomy to set goals. A total of 13,313 patient visits were analyzed. There were no significant differences between rates between the pre-intervention period and the first intervention period (p=0.44). DISCUSSION A longitudinal PM curriculum that incorporated SDL and goal setting with EHR-driven performance feedback was well-received by residents, however did not significantly impact the rate of care gap closure.
Collapse
Affiliation(s)
- Emily K Hadley Strout
- Department of Medicine, University of Vermont Medical Center, Burlington, VT, USA. .,The Robert Larner, M.D. College of Medicine at the University of Vermont, Burlington, VT, USA. .,Burlington Adult Primary Care, Burlington, VT, USA.
| | - Elizabeth A Wahlberg
- Department of Medicine, University of Vermont Medical Center, Burlington, VT, USA.,The Robert Larner, M.D. College of Medicine at the University of Vermont, Burlington, VT, USA
| | - Amanda G Kennedy
- Department of Medicine, University of Vermont Medical Center, Burlington, VT, USA.,The Robert Larner, M.D. College of Medicine at the University of Vermont, Burlington, VT, USA
| | - Bradley J Tompkins
- Department of Medicine, University of Vermont Medical Center, Burlington, VT, USA.,The Robert Larner, M.D. College of Medicine at the University of Vermont, Burlington, VT, USA
| | - Halle G Sobel
- Department of Medicine, University of Vermont Medical Center, Burlington, VT, USA.,The Robert Larner, M.D. College of Medicine at the University of Vermont, Burlington, VT, USA.,Burlington Adult Primary Care, Burlington, VT, USA
| |
Collapse
|
89
|
Defining Foundational Competence for Prelicensure and Graduate Nursing Students: A Concept Analysis and Conceptual Model. Nurse Educ Pract 2022; 64:103415. [DOI: 10.1016/j.nepr.2022.103415] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/25/2022] [Revised: 07/07/2022] [Accepted: 07/14/2022] [Indexed: 11/15/2022]
|
90
|
Henderson B, Chipchase L, Aitken R, Lewis LK. Consensus marking as a grading method for the development of evaluative judgement: Comparing assessor and students. Nurse Educ Pract 2022; 63:103386. [PMID: 35772306 DOI: 10.1016/j.nepr.2022.103386] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/03/2022] [Revised: 05/09/2022] [Accepted: 06/20/2022] [Indexed: 11/24/2022]
Abstract
AIM This study explored postgraduate nursing students' perceptions, anxiety and satisfaction of an innovative and novel grading method for online vivas, consensus marking, compared with traditional assessor judgement. BACKGROUND Reflection, self-evaluation and feedback conversations have the potential to develop nursing students' evaluative judgement. Consensus marking is a novel method of grading students' performance that supports students to reflect, self-evaluate and grade their own work. Active engagement in a feedback dialogue supports students to calibrate their self-evaluation to the required standard in a grade negotiation. Through this approach, students are supported to develop evaluative judgement and lifelong learning skills. DESIGN A convergent mixed-methods parallel research design was used. METHODS Students enrolled in a postgraduate emergency nursing unit of study completed two online viva assessments. One viva was graded using traditional assessor judgement and the other used consensus marking, involving a two-way feedback dialogue, where students had an opportunity to actively engage in grading their own work with the assessor. Student perceptions of each grading method were explored through semi-structured interviews. Interview data were analysed thematically using a six-stage approach. Student anxiety and satisfaction were measured pre- and post each viva using valid and reliable questionnaires. Non-parametric analyses explored differences in anxiety and satisfaction between the two grading methods. Alpha was set at 0.05. RESULTS Forty-six participants had complete data for anxiety and satisfaction across both test occasions (82%) and were included in the analysis. Of these, 13 students participated in follow up interviews. Students perceived that the ability to self-evaluate performance and discuss their grade with the assessor using consensus marking was less hierarchical and similar to a collegial debrief. Student anxiety was significantly lower prior to consensus marking compared with the assessor judged viva (p < 0.001). Students were significantly more satisfied with consensus marking compared with assessor judgement (p < 0.01). CONCLUSIONS Consensus marking created an opportunity for students to identify knowledge deficits through reflection and self-evaluation of their own performance prior to external judgement. Students were more satisfied and less anxious with the consensus marking grading method compared with traditional assessor judgement. These findings have implications for the development and application of new grading methods in nursing education to facilitate the development of evaluative judgement.
Collapse
Affiliation(s)
- Bridget Henderson
- Caring Futures Institute, College of Nursing and Health Sciences, Flinders University, Australia.
| | - Lucy Chipchase
- Caring Futures Institute, College of Nursing and Health Sciences, Flinders University, Australia.
| | - Robyn Aitken
- College of Medicine and Public Health, Flinders University, Australia.
| | - Lucy K Lewis
- Caring Futures Institute, College of Nursing and Health Sciences, Flinders University, Australia.
| |
Collapse
|
91
|
Khalife R, Gupta M, Gonsalves C, Park YS, Riddle J, Tekian A, Horsley T. Patient involvement in assessment of postgraduate medical learners: A scoping review. MEDICAL EDUCATION 2022; 56:602-613. [PMID: 34981565 DOI: 10.1111/medu.14726] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 06/03/2021] [Revised: 11/18/2021] [Accepted: 12/27/2021] [Indexed: 06/14/2023]
Abstract
CONTEXT Competency-based assessment of learners may benefit from a more holistic, inclusive, approach for determining readiness for unsupervised practice. However, despite movements towards greater patient partnership in health care generally, inclusion of patients in postgraduate medical learners' assessment is largely absent. METHODS We conducted a scoping review to map the nature, extent and range of literature examining the inclusion (or exclusion) of patients within the assessment of postgraduate medical learners. Guided by Arskey and O'Malley's framework and informed by Levac et al. and Thomas et al., we searched two databases (MEDLINE® and Embase®) from inception until February 2021 using subheadings related to assessment, patients and postgraduate learners. Data analysis examined characteristics regarding the nature and factor influencing patient involvement in assessment. RESULTS We identified 41 papers spanning four decades. Some literature suggests patients are willing to be engaged in assessment, however choose not to engage when, for example, language barriers may exist. When stratified by specialty or clinical setting, the influence of factors such as gender, race, ethnicity or medical condition seems to remain consistent. Patients may participate in assessment as a stand-alone group or part of a multi-source feedback process. Patients generally provided high ratings but commented on the observed professional behaviours and communication skills in comparison with physicians who focused on medical expertise. CONCLUSION Factors that influence patient involvement in assessment are multifactorial including patients' willingness themselves, language and reading-comprehension challenges and available resources for training programmes to facilitate the integration of patient assessments. These barriers however are not insurmountable. While understudied, research examining patient involvement in assessment is increasing; however, our review suggests that the extent which the unique insights will be taken up in postgraduate medical education may be dependent on assessment systems readiness and, in particular, physician readiness to partner with patients in this way.
Collapse
Affiliation(s)
- Roy Khalife
- Department of Medicine (Hematology), The Ottawa Hospital, University of Ottawa, Ottawa, Ontario, Canada
| | - Manika Gupta
- Department of Medicine (Hematology), The Ottawa Hospital, University of Ottawa, Ottawa, Ontario, Canada
| | - Carol Gonsalves
- Department of Medicine (Hematology), The Ottawa Hospital, University of Ottawa, Ottawa, Ontario, Canada
| | - Yoon Soo Park
- Massachusetts General Hospital, Harvard Medical School, Boston, Massachusetts, USA
| | - Janet Riddle
- Department of Medical Education, University of Illinois College of Medicine at Chicago, Chicago, Illinois, USA
| | - Ara Tekian
- Department of Medical Education, University of Illinois College of Medicine at Chicago, Chicago, Illinois, USA
| | - Tanya Horsley
- Research Unit, Royal College of Physicians and Surgeons of Canada, Ottawa, Ontario, Canada
- School of Epidemiology and Public Health, University of Ottawa, Ottawa, Ontario, Canada
| |
Collapse
|
92
|
Rassos J, Ginsburg S, Stalmeijer RE, Melvin LJ. The Senior Medical Resident's New Role in Assessment in Internal Medicine. ACADEMIC MEDICINE : JOURNAL OF THE ASSOCIATION OF AMERICAN MEDICAL COLLEGES 2022; 97:711-717. [PMID: 34879012 DOI: 10.1097/acm.0000000000004552] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/13/2023]
Abstract
PURPOSE With the introduction of competency-based medical education, senior residents have taken on a new, formalized role of completing assessments of their junior colleagues. However, no prior studies have explored the role of near-peer assessment within the context of entrustable professional activities (EPAs) and competency-based medical education. This study explored internal medicine residents' perceptions of near-peer feedback and assessment in the context of EPAs. METHOD Semistructured interviews were conducted from September 2019 to March 2020 with 16 internal medicine residents (8 first-year residents and 8 second- and third-year residents) at the University of Toronto, Toronto, Ontario, Canada. Interviews were conducted and coded iteratively within a constructivist grounded theory approach until sufficiency was reached. RESULTS Senior residents noted a tension in their dual roles of coach and assessor when completing EPAs. Senior residents managed the relationship with junior residents to not upset the learner and potentially harm the team dynamic, leading to the documentation of often inflated EPA ratings. Junior residents found senior residents to be credible providers of feedback; however, they were reticent to find senior residents credible as assessors. CONCLUSIONS Although EPAs have formalized moments of feedback, senior residents struggled to include constructive feedback comments, all while knowing the assessment decisions may inform the overall summative decision of their peers. As a result, EPA ratings were often inflated. The utility of having senior residents serve as assessors needs to be reexamined because there is concern that this new role has taken away the benefits of having a senior resident act solely as a coach.
Collapse
Affiliation(s)
- James Rassos
- J. Rassos is assistant professor, Department of Medicine, University of Toronto, Toronto, Ontario, Canada
| | - Shiphra Ginsburg
- S. Ginsburg is professor, Department of Medicine, and scientist, Wilson Centre for Education, University of Toronto, Toronto, Ontario, Canada
| | - Renée E Stalmeijer
- R.E. Stalmeijer is assistant professor, Department of Educational Development and Research, Faculty of Health, Medicine and Life Sciences, Maastricht University, Maastricht, the Netherlands
| | - Lindsay J Melvin
- L.J. Melvin is assistant professor, Department of Medicine, University of Toronto, Toronto, Ontario, Canada
| |
Collapse
|
93
|
Burns J, Chetlen A, Morgan DE, Catanzano TM, McLoud TC, Slanetz PJ, Jay AK. Affecting Change: Enhancing Feedback Interactions with Radiology Trainees. Acad Radiol 2022; 29 Suppl 5:S111-S117. [PMID: 34217615 DOI: 10.1016/j.acra.2021.05.018] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/04/2021] [Revised: 05/15/2021] [Accepted: 05/17/2021] [Indexed: 11/01/2022]
Abstract
Feedback is a critical part of the learning process and is a valuable tool to empower adult learners. Modern feedback theory places the learner at the center of the feedback encounter. Individual and institutional barriers to effective giving and receiving of feedback can be overcome through education and attention to the form and content of feedback. We review the elements of effective feedback and address issues of framing, environmental, and social factors which aid in providing psychological safety and trust, as necessary elements to create a culture of feedback in radiology training programs. We provide practical strategies to empower learners with the necessary skills to solicit, receive, and reflect on feedback.
Collapse
|
94
|
Castanelli DJ, Weller JM, Molloy E, Bearman M. How Trainees Come to Trust Supervisors in Workplace-Based Assessment: A Grounded Theory Study. ACADEMIC MEDICINE : JOURNAL OF THE ASSOCIATION OF AMERICAN MEDICAL COLLEGES 2022; 97:704-710. [PMID: 34732657 PMCID: PMC9028297 DOI: 10.1097/acm.0000000000004501] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/30/2023]
Abstract
PURPOSE In competency-based medical education, workplace-based assessment provides trainees with an opportunity for guidance and supervisors the opportunity to judge the trainees' clinical practice. Learning from assessment is enhanced when trainees reveal their thinking and are open to critique, which requires trust in the assessor. If supervisors knew more about how trainees come to trust them in workplace-based assessment, they could better engender trainee trust and improve trainees' learning experience. METHOD From August 2018 to September 2019, semistructured interviews were conducted with 17 postgraduate anesthesia trainees across Australia and New Zealand. The transcripts were analyzed using constructivist grounded theory methods sensitized by a sociocultural view of learning informed by Wenger's communities of practice theory. RESULTS Participants described a continuum from a necessary initial trust to an experience-informed dynamic trust. Trainees assumed initial trust in supervisors based on accreditation, reputation, and a perceived obligation of trustworthiness inherent in the supervisor's role. With experience and time, trainees' trust evolved based on supervisor actions. Deeper levels of trainee trust arose in response to perceived supervisor investment and allowed trainees to devote more emotional and cognitive resources to patient care and learning rather than impression management. Across the continuum from initial trust to experience-informed trust, trainees made rapid trust judgments that were not preceded by conscious deliberation; instead, they represented a learned "feel for the game." CONCLUSIONS While other factors are involved, our results indicate that the trainee behavior observed in workplace-based assessment is a product of supervisor invitation. Supervisor trustworthiness and investment in trainee development invite trainees to work and present in authentic ways in workplace-based assessment. This authentic engagement, where learners "show themselves" to supervisors and take risks, creates assessment for learning.
Collapse
Affiliation(s)
- Damian J. Castanelli
- D.J. Castanelli is senior lecturer, School of Clinical Sciences at Monash Health, Monash University, consultant anesthetist, Department of Anaesthesia and Perioperative Medicine, Monash Health, Clayton, Victoria, Australia, and PhD candidate, Centre for Research in Assessment and Digital Learning, Deakin University, Geelong, Victoria, Australia; ORCID: https://orcid.org/0000-0002-5377-809X
| | - Jennifer M. Weller
- J.M. Weller is professor and head, Centre for Medical and Health Sciences Education, University of Auckland, and specialist anesthetist, Department of Anaesthesia, Auckland City Hospital, Auckland, New Zealand; ORCID: https://orcid.org/0000-0003-3029-1390
| | - Elizabeth Molloy
- E. Molloy is professor of work-integrated learning, Department of Medical Education, Melbourne Medical School, University of Melbourne, Melbourne, Victoria, Australia; ORCID: https://orcid.org/0000-0001-9457-9348
| | - Margaret Bearman
- M. Bearman is research professor, Centre for Research in Assessment and Digital Learning, Deakin University, Geelong, Victoria, Australia; ORCID: https://orcid.org/0000-0002-6862-9871
| |
Collapse
|
95
|
Lowe B, Ng J, Jordan S, Waynforth D, Angstetra D, Brazil V. Test-enhanced learning improves learner attendance during a laparoscopic box trainer simulation program. Aust N Z J Obstet Gynaecol 2022; 62:589-595. [PMID: 35482668 PMCID: PMC9545122 DOI: 10.1111/ajo.13528] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/31/2021] [Revised: 02/15/2022] [Accepted: 03/18/2022] [Indexed: 11/28/2022]
Abstract
Background Laparoscopy is the gold standard approach for many surgical procedures, but it is a complex skill to learn. Laparoscopic simulation training may help, but it is unclear how to best engage trainees in these programs. Test‐enhanced learning (TEL) uses regular, well‐defined assessments of performance throughout the training phase of learning. Aim The aim of this study was to assess the effects of TEL on a laparoscopic simulation program involving a cohort of medical student volunteers. Materials and methods A prospective cohort study was performed with a convenience sample of 40 medical students. Students were recruited to participate in a ten‐week laparoscopic simulation program. Twenty students participated in a laparoscopic surgical program with TEL (‘TEL group’), and 20 students participated in a standard laparoscopic simulation program (‘control group’). Results Attendance in the TEL group was significantly higher than in the standard group (71 vs 51.5%, P = 0.03). There was no difference between groups in mean time scores. Four themes were identified in qualitative data drawn from student surveys – personal traits and motivators, training context, clear goals and feedback enabling understanding of one's own performance. Conclusion Testing laparoscopic skills throughout a learning program, in conjunction with individualised feedback and tracking of learning trajectory, increases trainee attendance. Laparoscopic simulation training programs are encouraged to reflect on the pedagogic framework in which their procedural skills training operates.
Collapse
Affiliation(s)
- Belinda Lowe
- Faculty of Health Sciences and Medicine, Bond University, Gold Coast, Australia.,Department of Obstetrics and Gynaecology, Gold Coast Hospital and Health Service, Southport, Australia
| | - Jessica Ng
- Faculty of Health Sciences and Medicine, Bond University, Gold Coast, Australia.,Department of Surgery, Gold Coast Hospital and Health Service, Southport, Australia
| | - Stephanie Jordan
- Department of Surgery, Gold Coast Hospital and Health Service, Southport, Australia
| | - David Waynforth
- Faculty of Health Sciences, Bond University, Gold Coast, Australia
| | - Donald Angstetra
- Faculty of Health Sciences and Medicine, Bond University, Gold Coast, Australia.,Department of Obstetrics and Gynaecology, Gold Coast Hospital and Health Service, Southport, Australia
| | - Victoria Brazil
- Faculty of Health Sciences and Medicine, Bond University, Gold Coast, Australia
| |
Collapse
|
96
|
Fabry G. Wie lassen sich professionelle Kompetenzen im Medizinstudium vermitteln? Ethik Med 2022. [DOI: 10.1007/s00481-022-00695-w] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/18/2022]
|
97
|
Boscardin CK, Bullock J, O'Sullivan P, Hauer K. Profiles for Success: Examining the Relationship between Student Profiles and Clerkship Performance Using Latent Profile Analysis. TEACHING AND LEARNING IN MEDICINE 2022; 34:145-154. [PMID: 35349389 DOI: 10.1080/10401334.2021.1937180] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 10/23/2020] [Revised: 04/28/2021] [Accepted: 05/12/2021] [Indexed: 06/14/2023]
Abstract
PHENOMENON Learning is a complex phenomenon in which learners can vary in their learning orientation, learning approaches, and perception of the learning environment. Learners may be characterized as mastery oriented or performance oriented, and this learning orientation can influence their learning approaches, such as whether to seek feedback or ask for help. The learning environment includes institutional assessment policies and informal interactions that emphasize outcomes and differentiation among students. When learners perceive the learning environment as performance oriented, they feel competition from other learners and fear negative feedback. This complex interplay of learning orientation, its influence on feedback-seeking behavior, and the environmental influence and reinforcement of both can be captured through learner profiles. APPROACH In this multi-institutional cross-sectional survey study of students from six U.S. medical schools, we investigated learner profiles that characterize how these domains manifest together across individual learners. We then determined how these profiles are associated with students' clerkship grades. Measures included the Patterns of Adaptive Learning for learning orientation, self-reports for learning approaches, and perceptions of the learning environment. We used latent profile analysis to cluster students who share common characteristics around orientation, behavior, and environment. The relationship between these profiles and the percentage of honors earned was examined using multiple regression analysis. FINDINGS We found four distinct learner profiles within 666 of 974 (68%) analyzable responses: (1) performance oriented with negative perceptions of environment, (2) mastery oriented with desirable learning approaches, (3) "average" group, and (4) mastery orientation only. Profile 1 (M = 39%) and Profile 4 (M = 38%) students received fewer clerkship honors compared with other profiles. Profile 2 students earned a significantly higher percentage of honors grades (M = 54%) compared with other students. Profile 3 students (average group) earned the second highest percentage of honors (M = 46%). Effect sizes comparing the percentage of honors varied from 0.32 to 0.65, indicating medium to large differences. INSIGHTS The highest performers, as measured by clerkship honors grades, were mastery oriented, perceived a more positive learning environment, and were comfortable asking questions and seeking feedback for their learning. The lowest performing students were performance oriented and had negative perceptions of their learning environments. Contrary to previous findings, we found that the relationship between mastery orientation and learning approaches is not unidirectional but rather seems to be mediated by perceptions of the learning environment.
Collapse
Affiliation(s)
- Christy K Boscardin
- Department of Medicine, University of California San Francisco, San Francisco, California, USA
- Department of Anesthesia and Perioperative Care, University of California San Francisco, San Francisco, California, USA
| | - Justin Bullock
- Department of Medicine, University of California San Francisco, San Francisco, California, USA
| | - Patricia O'Sullivan
- Department of Medicine, University of California San Francisco, San Francisco, California, USA
- Department of Surgery, University of California San Francisco, San Francisco, California, USA
| | - Karen Hauer
- Department of Medicine, University of California San Francisco, San Francisco, California, USA
| |
Collapse
|
98
|
A mobile application to facilitate implementation of programmatic assessment in anaesthesia training. Br J Anaesth 2022; 128:990-996. [DOI: 10.1016/j.bja.2022.02.038] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/30/2021] [Revised: 02/03/2022] [Accepted: 02/28/2022] [Indexed: 11/22/2022] Open
|
99
|
Lee SS, Lee H, Hwang H. New approach to learning medical procedures using a smartphone and the Moodle platform to facilitate assessments and written feedback. KOSIN MEDICAL JOURNAL 2022. [DOI: 10.7180/kmj.22.010] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/11/2022] Open
Abstract
Background: To overcome communication obstacles between medical students and trainers, we designed serial learning activities utilizing a smartphone and web-based instruction (WBI) on the Moodle platform to provide clear and retrievable trainer feedback to students on an objective structured clinical examination (OSCE) item. Methods: We evaluated students’ learning achievement and satisfaction with the new learning tool. A total of 80 fourth-year medical students participated. They installed the Moodle app (the WBI platform) on their smartphones and practiced an endotracheal suction procedure on a medical simulation mannequin while being evaluated by a trainer regarding competence in clinical skills on the smartphone app. Students’ competency was evaluated by comparing the scores between the formative assessment and the summative assessment. The degree of satisfaction and usefulness for the smartphone and WBI system were analyzed. Results: The means (standard deviations, SDs) of the formative and summative assessments were 8.80 (2.53) and 14.24 (1.97) out of a total of 17 points, respectively, reflecting a statistically significant difference (P<0.05). The degree of satisfaction and perceived usefulness of the smartphone app and WBI system were excellent, with means (SDs) of 4.60 (0.58), and 4.60 (0.65), respectively. Conclusion: We believe that the learning process using a smartphone and the Moodle platform offers good guidance for OSCE skill development because trainers’ written feedback is recorded online and is retrievable at all times, enabling students to build and maintain competency through frequent feedback review.
Collapse
|
100
|
de Groot JM, Kassam A, Swystun D, Topps M. Residents' transformational changes through self-regulated, experiential learning for professionalism. CANADIAN MEDICAL EDUCATION JOURNAL 2022; 13:5-16. [PMID: 35291457 PMCID: PMC8909828 DOI: 10.36834/cmej.70234] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Subscribe] [Scholar Register] [Indexed: 06/14/2023]
Abstract
PURPOSE Postgraduate trainees ('residents") are required to convey professional behaviours as they navigate complex clinical environments. However, little is known about experiential learning for professionalism. Thus, we asked residents about professionalism challenges within the clinical learning environment: 1) how challenges were identified, 2) what supported successfully addressing challenges and 3) the impact of addressing challenges to further inform resident education. METHOD From 2015-2016, twenty-five residents across specialties and multiple university affiliated teaching hospitals participated in appreciative inquiry informed audio-taped semi-structured interviews. Transcripts were categorized deductively for the 2015 CanMEDS Professional Role element addressed (commitment to patients, society, the profession, and physician health). A pragmatic research paradigm focussed descriptive data analysis on actions and outcomes. RESULTS Residents actively identify opportunities for experiential learning of professionalism within the clinical workplace- addressing conflicting priorities with interprofessional clinicians to ensure excellent patient care, providing informal feedback regarding peers' and other healthcare clinicians' professionalism lapses and by gaining self-awareness and maintaining wellness. There were no descriptions of commitment to society. Values, relationships, and reflection supported professional behaviours. Many described transformative personal and professional growth as an outcome of addressing professionalism challenges. CONCLUSIONS Residents self-regulated experiential learning for professionalism often results in transformational changes personally and professionally. Elucidation of how residents successfully navigate power dynamics and conflict to provide excellent patient care and feedback for professional regulatory behaviour will support professionalism education. An interprofessional research lens will be valuable to explore how best to incorporate commitment to society within clinical environments.
Collapse
Affiliation(s)
- Janet M de Groot
- Departments of Psychiatry, Oncology, and Community Health Sciences, Cumming School of Medicine, University of Calgary, Alberta, Canada
| | - Aliya Kassam
- Department of Community Health Sciences, Cumming School of Medicine, University of Calgary, Alberta, Canada
| | - Dana Swystun
- Radius Child and Youth Services, Ontario, Canada
| | - Maureen Topps
- Department of Family Medicine, Cumming School of Medicine, University of Calgary, Alberta, Canada
- Medical Council of Canada, Ontario, Canada
| |
Collapse
|