1
|
J K, Nayak VKR, N S, S H. Using word cloud as a formative assessment classroom technique in physiology. ADVANCES IN PHYSIOLOGY EDUCATION 2024; 48:708-711. [PMID: 39291936 DOI: 10.1152/advan.00142.2023] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 07/28/2023] [Revised: 05/08/2024] [Accepted: 05/30/2024] [Indexed: 09/19/2024]
Abstract
Formative assessment is a valuable tool in medical education as it allows students to reflect on their own learning and make necessary corrections. The use of innovative techniques in formative assessment can make it more engaging and effective for students, particularly utilizing electronic tools. A word cloud is a visual representation of data that is created using text inputs from a group of individuals. The resulting image shows the most frequently used words in a larger font, allowing quick understanding of the common themes and concepts in the group's responses. Mentimeter is an online platform for interactive presentations where students can provide their responses to a question, which can then be visualized as a word cloud. The study aims to explore the benefits and drawbacks of using word cloud formation as a formative assessment technique in the classroom, utilizing the online platform Mentimeter.NEW & NOTEWORTHY The current study was conducted to present a novel formative assessment classroom technique using Mentimeter. The benefit of this novel technique is that it is easy to implement, quick, anonymous, and strengthens student's conceptual learning. It also gives the teacher an opportunity to give feedback and stress commonly committed mistakes.
Collapse
Affiliation(s)
- Keba J
- Department of Physiology, Sree Gokulam Medical College and Research Foundation, Trivandrum, India
| | - Vineetha K Ramdas Nayak
- Department of Physiology, Sree Gokulam Medical College and Research Foundation, Trivandrum, India
| | - Sajitha N
- Department of Physiology, Sree Gokulam Medical College and Research Foundation, Trivandrum, India
| | - Himiki S
- Department of Community Medicine, Sree Gokulam Medical College and Research Foundation, Trivandrum, India
| |
Collapse
|
2
|
Mc Donald M, Muir F. A phenomenological study of resident and faculty experiences with learner engagement in the normalization of workplace-based assessment. CANADIAN MEDICAL EDUCATION JOURNAL 2024; 15:5-14. [PMID: 39310311 PMCID: PMC11415731 DOI: 10.36834/cmej.76192] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Indexed: 09/25/2024]
Abstract
Background Workplace-based assessments (WPBA) have become integral to learner-centred medical education. As previous research has linked learner engagement to WPBA implementation, this study explores residents' and faculty members' experiences with learner engagement in the normalisation of WPBA practice. Methods Transcendental phenomenology was used as the qualitative approach, focusing on the participants' lived experiences. A semi-structured interview guide was used to interview five faculty members and five residents who had conducted WPBA. The interviews were transcribed and analysed using phenomenological data analysis. Results Three themes were identified between learner engagement and WPBA conduct: (a) work environment, (b) roles and relationships, and (c) mutually beneficial teaching and learning. WPBA learner engagement occurred when participants interacted with each other and with the clinical setting to facilitate teaching and learning. Both participant groups reported a desire to participate in WPBA, but time constraints at times hindered participation. The residents indicated that WPBA improved their knowledge and admitted to experiencing negative emotions during the assessment. Overall, participants recognised the reciprocal benefits of WPBA participation for their professional development. Conclusion The findings of the study suggest that learner engagement influences the use of WPBA. Consequently, it may be beneficial to consider the role of learner engagement to normalise WPBA application for teaching and learning in the clinical context.
Collapse
Affiliation(s)
- Melissa Mc Donald
- Institute of Health Science Education, Georgetown Public Hospital Corporation, Georgetown, Guyana
| | - Fiona Muir
- School of Medicine, University of Dundee, Nethergate, UK
| |
Collapse
|
3
|
Oswald A, Dubois D, Snell L, Anderson R, Karpinski J, Hall AK, Frank JR, Cheung WJ. Implementing Competence Committees on a National Scale: Design and Lessons Learned. PERSPECTIVES ON MEDICAL EDUCATION 2024; 13:56-67. [PMID: 38343555 PMCID: PMC10854462 DOI: 10.5334/pme.961] [Citation(s) in RCA: 4] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 03/08/2023] [Accepted: 07/03/2023] [Indexed: 02/15/2024]
Abstract
Competence committees (CCs) are a recent innovation to improve assessment decision-making in health professions education. CCs enable a group of trained, dedicated educators to review a portfolio of observations about a learner's progress toward competence and make systematic assessment decisions. CCs are aligned with competency based medical education (CBME) and programmatic assessment. While there is an emerging literature on CCs, little has been published on their system-wide implementation. National-scale implementation of CCs is complex, owing to the culture change that underlies this shift in assessment paradigm and the logistics and skills needed to enable it. We present the Royal College of Physicians and Surgeons of Canada's experience implementing a national CC model, the challenges the Royal College faced, and some strategies to address them. With large scale CC implementation, managing the tension between standardization and flexibility is a fundamental issue that needs to be anticipated and addressed, with careful consideration of individual program needs, resources, and engagement of invested groups. If implementation is to take place in a wide variety of contexts, an approach that uses multiple engagement and communication strategies to allow for local adaptations is needed. Large-scale implementation of CCs, like any transformative initiative, does not occur at a single point but is an evolutionary process requiring both upfront resources and ongoing support. As such, it is important to consider embedding a plan for program evaluation at the outset. We hope these shared lessons will be of value to other educators who are considering a large-scale CBME CC implementation.
Collapse
Affiliation(s)
- Anna Oswald
- Division of Rheumatology, Department of Medicine, Faculty of Medicine and Dentistry, University of Alberta, Edmonton, AB, Canada
- Competency Based Medical Education, University of Alberta, Edmonton, AB, Canada
- Royal College of Physicians and Surgeons of Canada, Ottawa, ON, Canada
- 8-130 Clinical Sciences building, 11350-83 Avenue, Edmonton, AB, Canada
| | - Daniel Dubois
- Royal College of Physicians and Surgeons of Canada, Ottawa, ON, Canada
- Department of Anesthesiology and Pain Medicine, University of Ottawa, Ottawa, ON, Canada
| | - Linda Snell
- Royal College of Physicians and Surgeons of Canada, Ottawa, ON, Canada
- Institute of Health Sciences Education and Department of Medicine, McGill University, Montreal, QC, Canada
| | - Robert Anderson
- Royal College of Physicians and Surgeons of Canada, Ottawa, ON, Canada
- Northern Ontario School of Medicine University, Sudbury, ON, Canada
| | - Jolanta Karpinski
- Royal College of Physicians and Surgeons of Canada, Ottawa, ON, Canada
- Department of Medicine, University of Ottawa, Ottawa, ON, Canada
| | - Andrew K. Hall
- Royal College of Physicians and Surgeons of Canada, Ottawa, ON, Canada
- Dept. of Emergency Medicine, University of Ottawa, Canada
| | - Jason R. Frank
- Centre for Innovation in Medical Education, Faculty of Medicine, University of Ottawa, Canada
| | - Warren J. Cheung
- Dept. of Emergency Medicine, University of Ottawa, Canada
- Royal College of Physicians and Surgeons of Canada, 1053 Carling Avenue, Rm F660, Ottawa, Canada
| |
Collapse
|
4
|
Erumeda NJ, Jenkins LS, George AZ. Workplace-based learning opportunities in a South African family medicine training programme. Afr J Prim Health Care Fam Med 2023; 15:e1-e13. [PMID: 37916718 PMCID: PMC10623584 DOI: 10.4102/phcfm.v15i1.4073] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/08/2023] [Revised: 06/09/2023] [Accepted: 06/16/2023] [Indexed: 11/03/2023] Open
Abstract
BACKGROUND Workplace-based learning (WBL) provides authentic learning opportunities to develop fit-for-practice healthcare workers. Different types of WBL opportunities have been described in high-income countries, but the opportunities in the district health systems of sub-Saharan Africa have not been characterised. AIM This study explored family physicians' (FPs) and registrars' perceptions of WBL opportunities in a decentralised postgraduate family medicine registrar training programme. SETTING The study was conducted at five decentralised training sites across two provinces affiliated with the University of the Witwatersrand in South Africa. METHODS This instrumental case study involved semi-structured qualitative interviews with 11 FPs and 11 registrars purposively sampled across the training sites. The verbatim transcripts were thematically analysed using Braun and Clark's six-step approach. RESULTS Workplace-based learning opportunities were grouped into four themes: Learning from interpersonal interactions, learning from district activities, self-directed learning and contextual influences on learning opportunities. Registrars learnt from patients, peers, FPs and other professionals. Feedback, self-reflection, portfolio use, involvement in various district events, such as student and staff teaching, and continuous medical education augmented learning. Contextual influences originated from health facilities, resource availability, district management and university support, excessive workload and a need for standardised district learning opportunities. CONCLUSION Registrars are exposed to several types of WBL opportunities in district health systems. Effective engagement with available opportunities and addressing contextual challenges could enhance registrar learning.Contribution: Maximising learning opportunities to promote registrars' acquisition of required skills and competencies to efficiently address community needs in a middle-income country such as South Africa.
Collapse
Affiliation(s)
- Neetha J Erumeda
- Department of Family Medicine and Primary Care, Faculty of Health Sciences, University of the Witwatersrand, Johannesburg, South Africa; and Gauteng Department of Health, Ekurhuleni District Health Services, Germiston.
| | | | | |
Collapse
|
5
|
Martin L, Blissett S, Johnston B, Tsang M, Gauthier S, Ahmed Z, Sibbald M. How workplace-based assessments guide learning in postgraduate education: A scoping review. MEDICAL EDUCATION 2023; 57:394-405. [PMID: 36286100 DOI: 10.1111/medu.14960] [Citation(s) in RCA: 6] [Impact Index Per Article: 6.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 05/30/2022] [Revised: 09/16/2022] [Accepted: 10/21/2022] [Indexed: 06/16/2023]
Abstract
INTRODUCTION Competency-based medical education (CBME) led to the widespread adoption of workplace-based assessment (WBA) with the promise of achieving assessment for learning. Despite this, studies have illustrated tensions between the summative and formative role of WBA which undermine learning goals. Models of workplace-based learning (WBL) provide insight, however, these models excluded WBA. This scoping review synthesizes the primary literature addressing the role of WBA to guide learning in postgraduate medical education, with the goal of identifying gaps to address in future studies. METHODS The search was applied to OVID Medline, Web of Science, ERIC and CINAHL databases, articles up to September 2020 were included. Titles and abstracts were screened by two reviewers, followed by a full text review. Two members independently extracted and analysed quantitative and qualitative data using a descriptive-analytic technique rooted in Billett's four premises of WBL. Themes were synthesized and discussed until consensus. RESULTS All 33 papers focused on the perception of learning through WBA. The majority applied qualitative methodology (70%), and 12 studies (36%) made explicit reference to theory. Aligning with Billett's first premise, results reinforce that learning always occurs in the workplace. WBA helped guide learning goals and enhanced feedback frequency and specificity. Billett's remaining premises provided an important lens to understand how tensions that existed in WBL have been exacerbated with frequent WBA. As individuals engage in both work and WBA, they are slowly transforming the workplace. Culture and context frame individual experiences and the perceived authenticity of WBA. Finally, individuals will have different goals, and learn different things, from the same experience. CONCLUSION Analysing WBA literature through the lens of WBL theory allows us to reframe previously described tensions. We propose that future studies attend to learning theory, and demonstrate alignment with philosophical position, to advance our understanding of assessment-for-learning in the workplace.
Collapse
Affiliation(s)
- Leslie Martin
- Department of Medicine, McMaster University, Hamilton, Ontario, Canada
| | - Sarah Blissett
- Department of Medicine, Western University, London, Ontario, Canada
| | - Bronte Johnston
- McMaster Education Research, Innovation, and Theory Program, McMaster University, Hamilton, Ontario, Canada
| | - Michael Tsang
- Department of Medicine, McMaster University, Hamilton, Ontario, Canada
| | - Stephen Gauthier
- Department of Medicine, Queens University, Kingston, Ontario, Canada
| | - Zeeshan Ahmed
- Department of Medicine, Ottawa University, Ottawa, Ontario, Canada
| | - Matthew Sibbald
- Department of Medicine, McMaster University, Hamilton, Ontario, Canada
| |
Collapse
|
6
|
Almahal EA, Osman AAA, Tahir ME, Hamdan HZ, Gaddal AY, Alkhidir OTA, Gasmalla HEE. Fostering formative assessment: teachers' perception, practice and challenges of implementation in four Sudanese medical schools, a mixed-method study. BMC MEDICAL EDUCATION 2023; 23:247. [PMID: 37060025 PMCID: PMC10105490 DOI: 10.1186/s12909-023-04214-3] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 07/12/2022] [Accepted: 03/30/2023] [Indexed: 06/19/2023]
Abstract
Formative assessment (assessment for learning) enhances learning (especially deep learning) by using feedback as a central tool. However, implementing it properly faces many challenges. We aimed to describe the perception of medical teachers towards FA, their practice, challenges of implementing FA and present applicable solutions. A mixed-method, explanatory approach study was applied by administering a validated questionnaire to 190 medical teachers in four medical schools in Sudan. The obtained results were further studied using the Delphi method. Quantitative analysis revealed that medical teachers perceived their grasping of the concept of FAs and their ability to differentiate formative from summative assessments as very well (83.7%) and (77.4%), respectively. However, in contradiction to the former results, it was noteworthy that (41%) of them mistakenly perceived FA as an approach conducted for purposes of grading and certification. The qualitative study defined the challenges into two main themes: lack of understanding of formative assessment and lack of resources. Medical teachers' development and resource allocation were the main recommendations. We conclude that there is misunderstanding and malpractice in implementing formative assessment attributed to the lack of understanding of FA as well as the lack of resources. We as well present suggested solutions derived from the perception of the medical teachers in the study and evolved around three approaches: faculty development, managing the curriculum by allocating time and resources for FA, and advocacy among stakeholders.
Collapse
Affiliation(s)
| | | | | | - Hamdan Zaki Hamdan
- Biochemistry Department, Faculty of Medicine, Al‑Neelain University, Khartoum, Sudan
- Department of Basic Medical Sciences, Unaizah College of Medicine and Medical Sciences, Qassim University, Unaizah, Saudi Arabia
| | | | | | - Hosam Eldeen Elsadig Gasmalla
- Anatomy Department, Faculty of Medicine, Al-Neelain University, Khartoum, Sudan.
- Clinical Anatomy and Imaging, Warwick Medical School, University of Warwick, Coventry, United Kingdom.
| |
Collapse
|
7
|
Stephan A, Cheung G, van der Vleuten C. Entrustable Professional Activities and Learning: The Postgraduate Trainee Perspective. ACADEMIC PSYCHIATRY : THE JOURNAL OF THE AMERICAN ASSOCIATION OF DIRECTORS OF PSYCHIATRIC RESIDENCY TRAINING AND THE ASSOCIATION FOR ACADEMIC PSYCHIATRY 2023; 47:134-142. [PMID: 36224504 PMCID: PMC10060374 DOI: 10.1007/s40596-022-01712-2] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 05/05/2022] [Accepted: 09/16/2022] [Indexed: 05/25/2023]
Abstract
OBJECTIVE Entrustable professional activities (EPAs) are used as clinical activities in postgraduate psychiatry training in Australasia. This study aimed to explore psychiatry trainees' perceptions of the impact of EPAs on their motivation and learning. METHODS A constructivist grounded theory approach was used to conceptualize the impact of EPAs on trainees' motivation and learning. A purposive sample of trainees was recruited from across New Zealand. Semi-structured individual interviews were used for data collection and continued until theoretical saturation was reached. RESULTS The impact of EPAs on learning was mediated by the trainee's appraisals of subjective control, value, and the costs of engaging with EPAs. When appraisals were positive, EPAs encouraged a focus on particular learning needs and structured learning with the supervisor. However, when appraisals were negative, EPAs encouraged a superficial approach to learning. Trainee appraisals and their subsequent impact on motivation and learning were most affected by EPA granularity, alignment of EPAs with clinical practice, and the supervisor's conscientiousness in their approach to EPAs. CONCLUSIONS To stimulate learning, EPAs must be valued by both trainees and supervisors as constituting a coherent work-based curriculum that encompasses the key fellowship competencies. If EPAs are to be effective as clinical tasks for learning, ongoing faculty development must be the leading priority.
Collapse
Affiliation(s)
- Alice Stephan
- Mental Health and Addictions Service, Waikato District Health Board, Hamilton, New Zealand
| | - Gary Cheung
- School of Medicine, Faculty of Medical and Health Sciences, The University of Auckland, Auckland, New Zealand.
| | - Cees van der Vleuten
- School of Health Professions Education, Faculty of Health, Medicine and Life Sciences, University of Maastricht, Maastricht, Netherlands
| |
Collapse
|
8
|
Mowchun JJ, Davila CH. How Am I Doing in Small Group? Student Perceptions of Feedback in Case-Based Learning Sessions. MEDICAL SCIENCE EDUCATOR 2022; 32:1487-1493. [PMID: 36532402 PMCID: PMC9755430 DOI: 10.1007/s40670-022-01677-9] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Accepted: 10/28/2022] [Indexed: 06/17/2023]
Abstract
Introduction Small group case-based learning (CBL) facilitators are content experts that may provide feedback to students on cognitive reasoning skills and knowledge acquisition. However, student feedback-seeking behavior and response toward faculty feedback in CBL sessions are not known, and it is essential to maximize feedback in this setting where it can be a challenge to observe student performance while groups may have varied emphasis on individual versus team performance. We explored student perceptions of the effectiveness of faculty feedback processes during CBL sessions. Methods This qualitative study used semi-structured interviews with ten second year medical students enrolled in the Geisel School of Medicine preclinical neurology course. Investigator triangulation was used with interpretation comparisons that included independent content analysis. The constructed themes were discussed and final theme consensus was reached. Results Three major themes arose: (1) students value frequent feedback on their understanding of key clinical case concepts; (2) the CBL learning environment is not conducive to individual feedback; and (3) student feedback-seeking behavior and response are influenced by self-perceived level of preparedness for the sessions and overall comfort with the CBL facilitator and learning environment. Conclusions Students value content-based feedback from CBL sessions and need more individualized feedback. The style of the facilitator and overall learning environment can vary widely in the small group setting and has direct impact on feedback opportunities and student feedback-seeking behavior.
Collapse
Affiliation(s)
- Justin J. Mowchun
- Departments of Neurology and Medical Education, Geisel School of Medicine at Dartmouth, Dartmouth-Hitchcock Medical Center, Lebanon, NH USA
| | - Claire Hogue Davila
- Departments of Neurology and Medical Education, Geisel School of Medicine at Dartmouth, Dartmouth-Hitchcock Medical Center, Lebanon, NH USA
| |
Collapse
|
9
|
Marcotte KM, Holmstrom AL, Thelen AE, Gates R, George BC, Krumm AE. Opportunities to Increase Workplace Feedback: A Multi-institutional Examination. JOURNAL OF SURGICAL EDUCATION 2022; 79:e124-e129. [PMID: 36207256 DOI: 10.1016/j.jsurg.2022.08.012] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 03/25/2022] [Revised: 06/20/2022] [Accepted: 08/15/2022] [Indexed: 06/16/2023]
Abstract
OBJECTIVE While feedback is an essential component of resident education, there are few large-scale studies examining when and under what conditions formative feedback is provided. Workplace-based assessment systems offer an opportunity to identify factors influencing when faculty provides feedback to trainees. Influential factors affecting feedback may provide targets for increasing and improving feedback in resident education. DESIGN Data on whether dictated feedback was provided were obtained from the Society for Improving Medical Professional Learning (SIMPL) mobile application. We used generalized linear mixed effects models to identify the degree to which faculty members, procedures, surgical case characteristics, and trainee performance were associated with whether narrative feedback was provided using SIMPL. SETTING This study was conducted using data from members of the SIMPL collaborative. PARTICIPANTS 67,434 evaluations from 70 general surgery programs were included from 2015 to 2021. Of these, 25,355 evaluations included dictated feedback. RESULTS Approximately 61% of the variation in whether dictated feedback was provided was attributable to the individual faculty member. Compared to residents who achieved autonomy ratings of "Active Help," residents who achieved ratings of "Supervision Only" (odds ratio (OR) = 0.80, 95% confidence interval (CI) = 0.72, 0.88) had a lower likelihood of receiving dictated feedback. Residents who achieved ratings of "Intermediate" (OR = 0.81, CI = 0.74, 0.89), "Practice-Ready" (OR = 0.50, CI = 0.45, 0.57), or "Exceptional (OR = 0.64, CI = 0.54, 0.76) showed a lower likelihood of receiving dictated feedback compared to those rated as "Inexperienced." Cases rated as "High" in terms of complexity were associated with an increased likelihood of having dictation (OR = 1.35, CI = 1.26, 1.44). CONCLUSIONS The largest contributing factor for whether dictated feedback is included in a SIMPL evaluation are factors specific to the attending surgeon. Resident performance, resident autonomy, and case complexity had only modest associations with feedback decisions. Efforts to improve the amount of formative feedback for trainees should be directed towards reducing the variation in which attending surgeons elect to provide feedback.
Collapse
Affiliation(s)
- Kayla M Marcotte
- Center for Surgical Training and Research, Department of Surgery, Michigan Medicine, Ann Arbor, Michigan; Department of Learning Health Sciences, University of Michigan, Ann Arbor, Michigan.
| | - Amy L Holmstrom
- Northwestern University Feinberg School of Medicine, Department of Surgery, Chicago, Illinois
| | - Angela E Thelen
- Center for Surgical Training and Research, Department of Surgery, Michigan Medicine, Ann Arbor, Michigan
| | - Rebecca Gates
- Center for Surgical Training and Research, Department of Surgery, Michigan Medicine, Ann Arbor, Michigan
| | - Brian C George
- Center for Surgical Training and Research, Department of Surgery, Michigan Medicine, Ann Arbor, Michigan; Department of Surgery, Michigan Medicine, Ann Arbor, Michigan
| | - Andrew E Krumm
- Center for Surgical Training and Research, Department of Surgery, Michigan Medicine, Ann Arbor, Michigan; Department of Learning Health Sciences, University of Michigan, Ann Arbor, Michigan
| |
Collapse
|
10
|
Lee ASO, Donoff C, Ross S. Using Learning Analytics to Examine Differences in Assessment Forms From Continuous Versus Episodic Supervisors of Family Medicine Residents. J Grad Med Educ 2022; 14:606-612. [PMID: 36274777 PMCID: PMC9580309 DOI: 10.4300/jgme-d-21-00832.1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 08/24/2021] [Revised: 01/29/2022] [Accepted: 06/28/2022] [Indexed: 11/06/2022] Open
Abstract
BACKGROUND It is assumed that there is a need for continuity of supervision within competency-based medical education, despite most evidence coming from the undergraduate medical education rather than the graduate medical education (GME) context. This evidence gap must be addressed to justify the time and effort needed to redesign GME programs to support continuity of supervision. OBJECTIVE To examine differences in assessment behaviors of continuous supervisors (CS) versus episodic supervisors (ES), using completed formative assessment forms, FieldNotes, as a proxy. METHODS The FieldNotes CS- and ES-entered for family medicine residents (N=186) across 3 outpatient teaching sites over 3 academic years (2015-2016, 2016-2017, 2017-2018) were examined using 2-sample proportion z-tests to determine differences on 3 FieldNote elements: competency (Sentinel Habit [SH]), Clinical Domain (CD), and Progress Level (PL). RESULTS Sixty-nine percent (6104 of 8909) of total FieldNotes were analyzed. Higher proportions of CS-entered FieldNotes indicated SH3 (Managing patients with best practices), z=-3.631, P<.0001; CD2 (Care of adults), z=-8.659, P<.0001; CD3 (Care of the elderly), z=-4.592, P<.0001; and PL3 (Carry on, got it), z=-4.482, P<.0001. Higher proportions of ES-entered FieldNotes indicated SH7 (Communication skills), z=4.268, P<.0001; SH8 (Helping others learn), z=20.136, P<.0001; CD1 (Doctor-patient relationship/ethics), z=14.888, P<.0001; CD9 (Not applicable), z=7.180, P<.0001; and PL2 (In progress), z=5.117, P<.0001. CONCLUSIONS The type of supervisory relationship impacts assessment: there is variability in which competencies are paid attention to, which contexts or populations are included, and which progress levels are chosen.
Collapse
Affiliation(s)
- Ann S. O. Lee
- Ann S. O. Lee, MD, MEd, is Assistant Professor, Department of Family Medicine, University of Alberta, Edmonton, Alberta, Canada
| | - Christopher Donoff
- Christopher Donoff, MSc, is Junior Data Scientist, Blackline Safety, Calgary, Alberta, Canada
| | - Shelley Ross
- Shelley Ross, PhD, is Professor, Department of Family Medicine, University of Alberta Edmonton, Alberta, Canada
| |
Collapse
|
11
|
Seagrave MP, Foster-Johnson L, Waits JB, Margo K, Leong SL. Enhancing Examination Success: the Cumulative Benefits of Self-Assessment Questions and Virtual Patient Cases. MEDICAL SCIENCE EDUCATOR 2022; 32:985-993. [PMID: 36276775 PMCID: PMC9583958 DOI: 10.1007/s40670-022-01568-z] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Accepted: 05/29/2022] [Indexed: 06/16/2023]
Abstract
PURPOSE Research on the learning benefits of the feedback-rich formative assessment environment of virtual patient cases (VPCs) has largely been limited to single institutions and focused on discrete clinical skills or topical knowledge. To augment current understanding, we designed a multi-institutional study to explore the distinct and cumulative effects of VPC formative assessments and optional self-assessment questions (SAQs) on exam performance. METHOD In this correlational study, we examined the records of 1,692 students on their family medicine (FM) clerkship at 20 medical schools during the 2014-2015 academic year. Schools utilized an established online curriculum, which included family medicine VPCs, embedded formative assessments, context-rich SAQs corresponding with each VPC, and an associated comprehensive family medicine exam. We used mixed-effects modeling to relate the student VPC composite formative assessment score, SAQ completion, and SAQ performance to students' scores on the FM final examination. RESULTS Students scored higher on the final exam when they performed better on the VPC formative assessments, completed associated SAQs, and scored higher on those SAQs. Students' SAQ completion enhanced examination performance above that explained by engagement with the VPC formative assessments alone. CONCLUSIONS This large-scale, multi-institutional study furthers the body of research on the effect of formative assessments associated with VPCs on exam performance and demonstrates the added benefit of optional associated SAQs. Findings highlight opportunities for future work on the broader impact of formative assessments for learning, exploring the benefits of integrating VPCs and SAQs, and documenting effects on clinical performance and summative exam scores.
Collapse
Affiliation(s)
- Martha P. Seagrave
- Department of Family Medicine, Robert Larner MD College of Medicine, at the University of Vermont, Burlington, VT USA
| | - Lynn Foster-Johnson
- Department of Medical Education and The Dartmouth Institute, Geisel School of Medicine at Dartmouth, Hanover, NH USA
| | - John B. Waits
- Cahaba-UAB Family Medicine Residency, Cahaba Medical Care Foundation, Centreville, AB USA
| | - Katherine Margo
- Family Medicine and Community Health, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA USA
| | - Shou Ling Leong
- Department of Family and Community Medicine, Pennsylvania State University College of Medicine, Hershey, PA USA
| |
Collapse
|
12
|
Hennel EK, Trachsel A, Subotic U, Lörwald AC, Harendza S, Huwendiek S. How does multisource feedback influence residency training? A qualitative case study. MEDICAL EDUCATION 2022; 56:660-669. [PMID: 35263461 PMCID: PMC9314722 DOI: 10.1111/medu.14798] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 09/26/2021] [Revised: 01/20/2022] [Accepted: 02/22/2022] [Indexed: 06/14/2023]
Abstract
INTRODUCTION Multisource feedback (MSF), also called 360-degree assessment, is one form of assessment used in postgraduate training. However, there is an ongoing discussion on its value, because the factors that influence the impact of MSF and the main impact of MSF are not fully understood. In this study, we investigated both the influencing factors and the impact of MSF on residency training. METHODS We conducted a qualitative case study within the boundaries of the residency training for paediatricians and paediatric surgeons at a University Hospital. We collected data from seven focus group interviews with stakeholders of MSF (residents, raters and supervisors). By performing a reflexive thematic analysis, we extracted the influencing factors and the impact of MSF. RESULTS We found seven influencing factors: MSF is facilitated by the announcement of a clear goal of MSF, the training of raters on the MSF instrument, a longitudinal approach of observation, timing not too early and not too late during the rotation, narrative comments as part of the ratings, the residents' self-assessment and a supervisor from the same department. We found three themes on the impact of MSF: MSF supports the professional development of residents, enhances interprofessional teamwork and increases the raters' commitment to the training of residents. CONCLUSION This study illuminates the influencing factors and impact of MSF on residency training. We offer novel recommendations on the continuity of observation, the timing during rotations and the role of the supervisor. Moreover, by discussing our results through the lens of identity formation theory, this work advances our conceptual understanding of MSF. We propose identity formation theory as a framework for future research on MSF to leverage the potential of MSF in residency training.
Collapse
Affiliation(s)
- Eva K. Hennel
- Department for Assessment and Evaluation (AAE), Institute for Medical EducationUniversity of BernBernSwitzerland
| | - Andrea Trachsel
- Department for Assessment and Evaluation (AAE), Institute for Medical EducationUniversity of BernBernSwitzerland
| | | | - Andrea C. Lörwald
- Department for Assessment and Evaluation (AAE), Institute for Medical EducationUniversity of BernBernSwitzerland
| | - Sigrid Harendza
- Department of Internal MedicineUniversity Medical Center Hamburg‐EppendorfHamburgGermany
| | - Sören Huwendiek
- Department for Assessment and Evaluation (AAE), Institute for Medical EducationUniversity of BernBernSwitzerland
| |
Collapse
|
13
|
Long S, Rodriguez C, St-Onge C, Tellier PP, Torabi N, Young M. Factors affecting perceived credibility of assessment in medical education: A scoping review. ADVANCES IN HEALTH SCIENCES EDUCATION : THEORY AND PRACTICE 2022; 27:229-262. [PMID: 34570298 DOI: 10.1007/s10459-021-10071-w] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 07/29/2020] [Revised: 08/31/2021] [Accepted: 09/11/2021] [Indexed: 06/13/2023]
Abstract
Assessment is more educationally effective when learners engage with assessment processes and perceive the feedback received as credible. With the goal of optimizing the educational value of assessment in medical education, we mapped the primary literature to identify factors that may affect a learner's perceptions of the credibility of assessment and assessment-generated feedback (i.e., scores or narrative comments). For this scoping review, search strategies were developed and executed in five databases. Eligible articles were primary research studies with medical learners (i.e., medical students to post-graduate fellows) as the focal population, discussed assessment of individual learners, and reported on perceived credibility in the context of assessment or assessment-generated feedback. We identified 4705 articles published between 2000 and November 16, 2020. Abstracts were screened by two reviewers; disagreements were adjudicated by a third reviewer. Full-text review resulted in 80 articles included in this synthesis. We identified three sets of intertwined factors that affect learners' perceived credibility of assessment and assessment-generated feedback: (i) elements of an assessment process, (ii) learners' level of training, and (iii) context of medical education. Medical learners make judgments regarding the credibility of assessments and assessment-generated feedback, which are influenced by a variety of individual, process, and contextual factors. Judgments of credibility appear to influence what information will or will not be used to improve later performance. For assessment to be educationally valuable, design and use of assessment-generated feedback should consider how learners interpret, use, or discount assessment-generated feedback.
Collapse
Affiliation(s)
- Stephanie Long
- Department of Family Medicine, McGill University, Montreal, QC, Canada
| | - Charo Rodriguez
- Department of Family Medicine, McGill University, Montreal, QC, Canada
| | - Christina St-Onge
- Faculty of Medicine and Health Sciences, University of Sherbrooke, Sherbrooke, QC, Canada
| | | | - Nazi Torabi
- Science Collections, University of Toronto Libraries, Toronto, ON, Canada
| | - Meredith Young
- Institute of Health Sciences Education, McGill University, 1110 Pine Ave West, Montreal, QC, H3A 1A3, Canada.
- Department of Medicine, McGill University, Montreal, QC, Canada.
| |
Collapse
|
14
|
Interviews with Global Pharmacists and Healthcare Professionals in Great Britain to Establish Personal Experiences around Professional Development Activity. PHARMACY 2022; 10:pharmacy10010007. [PMID: 35076590 PMCID: PMC8788491 DOI: 10.3390/pharmacy10010007] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/01/2021] [Revised: 12/26/2021] [Accepted: 12/28/2021] [Indexed: 11/16/2022] Open
Abstract
Professional development activity is needed to ensure practitioners are up to date and providing optimal patient care. This includes, but is not restricted to, mandatory continuing professional development (CPD) or continuing education (CE) requirements, which differ by professions globally and within countries. This study aimed to investigate perceptions, participation, and individual practice for healthcare professionals in Great Britain (GB) and pharmacists globally to identify similarities and differences after the introduction of revalidation for pharmacists in GB. Qualitative data was received through interviews, which was analysed using content analysis. In total, 24 interviews were completed with pharmacists registered globally, and healthcare professionals registered in GB. A culture of CPD was seen for healthcare professionals in GB and globally for pharmacists; there was no consistent model. Face-to-face activity was common, with an increase in online provision, especially where large geographies were seen. Most learning was completed in the professional’s own time. Multiple providers were seen, with the evaluation of events using questionnaires being commonplace. Different formats of learning were useful for different topics, with skills learning being better when face-to-face. Although varied requirements were in place, regulation should support patient-based practice outcomes. This study showed that commitment to learning was similar in different professions in GB and by pharmacists globally, with similar benefits and challenges.
Collapse
|
15
|
Amir Rad FA, Otaki F, AlGurg R, Khan E, Davis D. A qualitative study of trainer and trainee perceptions and experiences of clinical assessment in post-graduate dental training. EUROPEAN JOURNAL OF DENTAL EDUCATION : OFFICIAL JOURNAL OF THE ASSOCIATION FOR DENTAL EDUCATION IN EUROPE 2021; 25:215-224. [PMID: 32813939 DOI: 10.1111/eje.12593] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 04/25/2020] [Revised: 07/13/2020] [Accepted: 08/07/2020] [Indexed: 05/08/2023]
Abstract
BACKGROUND The implementation of workplace-based assessment (WBA) needs to ensure the achievement of pre-set competences but may look different across varying contexts, such as in post-graduate dental education. The purpose of this study is to explore the perception of residents, faculty members and alumni concerning their experience with clinical assessment, and what configurations they consider as optimal to maximise the entailed learning experience. METHODS This study relied on a qualitative descriptive design using two data collection tools: focus group sessions, and semi-structured, one-to-one interviews. Data were triangulated from three sources: residents, faculty members and alumni. The data were inductively analysed based on constructivist epistemology. This was done using the Thematic Analysis approach, facilitated by NVivo software. RESULTS The analysis revealed two mutually exclusive themes: process and people. Within process, variables related to quality, workflow and feedback surfaced. As for the people theme, the main two group of stakeholders referred to in the related analysis were the trainees and the trainers. DISCUSSION There are many variables that need to be considered when developing an evidence-driven WBA. In addition, factoring into the design of the WBA the perception of the main stakeholders will enable contextualisation which is expected to raise the reliability of the adapted tools. CONCLUSION This study introduced a framework that could support post-graduate universities in their journey towards developing context-specific WBA.
Collapse
Affiliation(s)
- Fatemeh A Amir Rad
- Hamdan Bin Mohammed College of Dental Medicine, Mohammed Bin Rashid University of Medicine and Health Sciences, Dubai, United Arab Emirates
- Masters in Medical Education Programme, Centre for Medical Education, School of Medicine, University of Dundee, Nethergate, Dundee, UK
| | - Farah Otaki
- Strategy and Institutional Excellence, Mohammed Bin Rashid University of Medicine and Health Sciences, Dubai, United Arab Emirates
| | - Reem AlGurg
- Strategy and Institutional Excellence, Mohammed Bin Rashid University of Medicine and Health Sciences, Dubai, United Arab Emirates
- College of Medicine, Mohammed Bin Rashid University of Medicine and Health Sciences, Dubai, United Arab Emirates
| | - Erum Khan
- College of Medicine, Ajman University, Ajman, United Arab Emirates
| | - Dave Davis
- Centre for Outcomes and Research in Education, Mohammed Bin Rashid University of Medicine and Health Sciences, Dubai, United Arab Emirates
- Department of Family and Community Medicine, University of Toronto, Toronto, Canada
| |
Collapse
|
16
|
Dai CM, Bertram K, Chahine S. Feedback Credibility in Healthcare Education: a Systematic Review and Synthesis. MEDICAL SCIENCE EDUCATOR 2021; 31:923-933. [PMID: 34457934 PMCID: PMC8368112 DOI: 10.1007/s40670-020-01167-w] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Accepted: 11/19/2020] [Indexed: 05/21/2023]
Abstract
PURPOSE The purpose of this study was to systematically review and synthesize factors that influence learners' perceptions of credibility when feedback is provided by an authority figure in a healthcare environment. METHODS This study reviewed literature from medicine, psychology, and education using systematic review and qualitative synthesis methods. In a multi-step process, major electronic bibliographic databases were searched for relevant studies until October 2020. RESULTS The search identified 9216 articles. A total of 134 abstracts underwent full-text review. Of these, 22 articles met inclusion criteria. The studies were heterogenous and the majority utilized a qualitative design with interviews and focus groups. A few studies employed mixed methodology (n = 2) and two studies used a quantitative design. Four main themes were identified: feedback characteristics, context of feedback, source credibility, and recipient characteristics. CONCLUSION As programs implement major educational change initiatives to create more formative assessment practices, feedback will become even more crucial. The four main themes identified are important factors that contribute to the perception of feedback credibility. While the factors are described independently, they may be viewed as interrelated and the association between these factors and feedback may be driven more by learning culture than each characteristic. SUPPLEMENTARY INFORMATION The online version contains supplementary material available at 10.1007/s40670-020-01167-w.
Collapse
Affiliation(s)
- Cecilia M. Dai
- Schulich School of Medicine & Dentistry, Western University, London, Ontario Canada
- Health Sciences Addition, Western University, Room H110B, London, Ontario N6A 5C1 Canada
| | - Kaitlyn Bertram
- Royal Victoria Regional Health Centre (RVH), University of Toronto Department of Family and Community Medicine, Toronto, Ontario Canada
| | - Saad Chahine
- Centre for Education Research & Innovation, Schulich School of Medicine & Dentistry, Western University, London, Ontario Canada
| |
Collapse
|
17
|
McDonald JA, Lai CJ, Lin MYC, O'Sullivan PS, Hauer KE. "There Is a Lot of Change Afoot": A Qualitative Study of Faculty Adaptation to Elimination of Tiered Grades With Increased Emphasis on Feedback in Core Clerkships. ACADEMIC MEDICINE : JOURNAL OF THE ASSOCIATION OF AMERICAN MEDICAL COLLEGES 2021; 96:263-270. [PMID: 32909998 DOI: 10.1097/acm.0000000000003730] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/11/2023]
Abstract
PURPOSE Medical schools responding to challenges with fairness, equity, learning environments, and student wellness have reconsidered clerkship grades. This study used the Concerns-Based Adoption Model (CBAM) to explore how faculty responded to a change in the approach to assessment from focus on grades toward focus on feedback. METHOD This qualitative study used an inductive approach to analyze data from semistructured interviews with teaching faculty and education leaders at University of California, San Francisco, School of Medicine 6 months following the elimination of traditional tiered clerkship grades. Participants were recruited in 2019 using purposive sampling. Interview questions addressed participants' perceptions of the new approach to assessment and grading that emphasizes assessment for learning, including advantages and concerns. The authors analyzed data using thematic analysis informed by sensitizing concepts from CBAM. RESULTS Nineteen faculty participated (11 medicine, 8 surgery). Faculty expressed optimism about the impact of the change on their clinical educator roles and sought faculty development to enhance feedback and assessment skills. Perceived benefits to students addressed learning and wellness, with concern for students' motivation and professional development. Faculty shared uncertainty about their roles and larger systemic consequences involving the residency match and institutional reputation. Participants acknowledged that traditional grading is imperfect, yet felt a departure from this system represented significant culture change. CONCLUSIONS Faculty perceptions of the assessment and grade change in 2 large departments demonstrated tensions around grading. Their descriptions of change as an ongoing process aligned with CBAM and required them to consider new approaches and perspectives. While discourse about assessment and grading frequently focuses on learners, these findings support institutional consideration of the impact of changes in assessment on teaching faculty.
Collapse
MESH Headings
- Clinical Clerkship/standards
- Clinical Clerkship/statistics & numerical data
- Clinical Competence/statistics & numerical data
- Education, Medical/standards
- Education, Medical/statistics & numerical data
- Education, Medical, Undergraduate/standards
- Education, Medical, Undergraduate/statistics & numerical data
- Educational Measurement/methods
- Faculty, Medical/statistics & numerical data
- Faculty, Medical/trends
- Feedback
- Female
- Humans
- Leadership
- Learning/physiology
- Male
- Perception
- Qualitative Research
- San Francisco/epidemiology
- Schools, Medical/organization & administration
- Students, Medical/psychology
- Surveys and Questionnaires
- Thematic Apperception Test/statistics & numerical data
- Universities/organization & administration
Collapse
Affiliation(s)
- Jordan A McDonald
- J.A. McDonald is a third-year medical student, University of California, San Francisco, School of Medicine, San Francisco, California
| | - Cindy J Lai
- C.J. Lai is director of internal medicine clerkships and professor, Department of Medicine, University of California, San Francisco, School of Medicine, San Francisco, California
| | - Matthew Y C Lin
- M.Y.C. Lin is director of surgery clerkships and associate professor, Department of Surgery, University of California, San Francisco, School of Medicine, San Francisco, California
| | - Patricia S O'Sullivan
- P.S. O'Sullivan is director of research and development in medical education and professor, Department of Medicine and Department of Surgery, University of California, San Francisco, School of Medicine, San Francisco, California
| | - Karen E Hauer
- K.E. Hauer is associate dean for competency assessment and professional standards and professor, Department of Medicine, University of California, San Francisco, School of Medicine, San Francisco, California
| |
Collapse
|
18
|
Rawekar A, Choudhari SG, Mishra V, Vagha S. Formative assessment in practical for Indian postgraduates in health professions education: A strategic initiative towards competency-based education. J Family Med Prim Care 2020; 9:3399-3404. [PMID: 33102303 PMCID: PMC7567224 DOI: 10.4103/jfmpc.jfmpc_185_20] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/30/2020] [Revised: 03/13/2020] [Accepted: 04/07/2020] [Indexed: 11/17/2022] Open
Abstract
Background: Observational assessment (OA) or Workplace Based Assessment (WPBA) is being increasingly used to assess the trainees by direct observation to shape their learning. The current deficiencies in our conventional assessment system are due to lack of conceptualization of assessment as a process for continuous improvement and learning, leading to non-utilization of many available tools of assessment. Objective: The present study aimed to introduce formative assessment (FA) for postgraduates at all the constituent colleges (Medical, Dental, Ayurveda, Nursing) under the ambit of Datta Meghe Institute of Medical Sciences (Deemed University) Sawangi (M) Wardha, (Maharashtra), India. Methodology: Observational Assessment tools used for FA were Mini CEX (Mini Clinical Evaluation Exercise) and DOPS (Direct Observation of Procedural Skills)in addition to Objectively Structured Clinical/Practical Examination (OSCE/OSPE). Six encounters of OAs were conducted at the end of each semester. The OA tool used was either the MiniCEX or/and DOPS, depending on the type of subject (Preclinical, Para-clinical, Clinical). Results: A significant improvement in scores from one semester to other as the postgraduate learner progresses, was observed. The overall response from the postgraduate students and faculty for FA was satisfactory. Conclusion: Formative assessment in practical's, was feasible, acceptable and effective program to improve the overall learning and competency of postgraduates.
Collapse
Affiliation(s)
- Alka Rawekar
- Department of Physiology, Jawaharlal Nehru Medical College, Maharashtra, India.,Department of Assessment and Evaluation, School for Health Professions Education and Research, Datta Meghe Institute of Medical Sciences, Wardha (MS), Maharashtra, India
| | - Sonali G Choudhari
- Department of Assessment and Evaluation, School for Health Professions Education and Research, Datta Meghe Institute of Medical Sciences, Wardha (MS), Maharashtra, India.,Department of Community Medicine, Jawaharlal Nehru Medical College, Wardha (MS), Maharashtra, India
| | - Vedprakash Mishra
- Pro Chancellor, Datta Meghe Institute of Medical Sciences, Wardha (MS), Maharashtra, India
| | - Sunita Vagha
- Ex-Director, School for Health Professions Education and Research, Wardha (MS), Maharashtra, India.,Department of Pathology, Jawaharlal Nehru Medical College, Datta Meghe Institute of Medical Sciences, Wardha (MS), Maharashtra, India
| |
Collapse
|
19
|
Buckley C, Natesan S, Breslin A, Gottlieb M. Finessing Feedback: Recommendations for Effective Feedback in the Emergency Department. Ann Emerg Med 2020; 75:445-451. [DOI: 10.1016/j.annemergmed.2019.05.016] [Citation(s) in RCA: 13] [Impact Index Per Article: 3.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/30/2018] [Indexed: 01/11/2023]
|
20
|
Brouwers M, van Weel C, Laan R, van Weel-Baumgarten E. Training Undergraduates Skills in Breaking Bad News: How Students Value Educators' Feedback. JOURNAL OF CANCER EDUCATION : THE OFFICIAL JOURNAL OF THE AMERICAN ASSOCIATION FOR CANCER EDUCATION 2019; 34:1103-1106. [PMID: 30128954 PMCID: PMC6882760 DOI: 10.1007/s13187-018-1415-8] [Citation(s) in RCA: 7] [Impact Index Per Article: 1.4] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/09/2023]
Abstract
Feedback is a key factor in acquiring breaking bad news (BBN) communication skills and its' acceptance depends on the perceived credibility of the provider. Our aim was to investigate students' opinions on the provided feedback by different educators (surgeons, psychologists, and simulated patient (SP)) during BBN skills training. We developed a questionnaire investigating provided feedback by the surgeon, psychologist, and SP (yes or no statements), regarding (1) perceived safety of the atmosphere, (2) perceived positive feedback, (3) perceived specific feedback, and (4) perceived usefulness for improvement during BBN skills training. Five hundred twenty students returned the questionnaire after BBN skills training. Most students rated the feedback as positive, specific, and useful. Also, the atmosphere was considered safe. Feedback ratings of the SP were the same as for the surgeon and valued higher than for the psychologist. An unsafe atmosphere, or not receiving positive, specific, or useful feedback was mostly related to the psychologist's feedback. Feedback on BBN skills training by surgeons and SPs is rated equally helpful by students and is regarded specific, useful, and positive. When designing a BBN training, it is worth to consider involving SP's as well as clinicians.
Collapse
Affiliation(s)
- Marianne Brouwers
- Department of Primary and Community Care, Radboud Institute for Health Sciences, Radboud University Medical Center, Nijmegen, The Netherlands.
| | - Chris van Weel
- Department of Health Services Research and Policy, Australian National University, Canberra, Australia
| | - Roland Laan
- Health Academy, Radboud University Medical Center, Nijmegen, The Netherlands
| | - Evelyn van Weel-Baumgarten
- Department of Primary and Community Care, Radboud Institute for Health Sciences, Radboud University Medical Center, Nijmegen, The Netherlands
| |
Collapse
|
21
|
Andreassen P, Malling B. How are formative assessment methods used in the clinical setting? A qualitative study. INTERNATIONAL JOURNAL OF MEDICAL EDUCATION 2019; 10:208-215. [PMID: 31759332 PMCID: PMC7246116 DOI: 10.5116/ijme.5db3.62e3] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 06/07/2019] [Accepted: 10/25/2019] [Indexed: 05/21/2023]
Abstract
OBJECTIVES To explore how formative assessment methods are used and perceived by second-year junior doctors in different clinical settings. METHODS A focused ethnography study was carried out. Ten second-year junior doctors from different specialties were selected using purposive sampling. The junior doctors were observed during a day in their clinical workplace where formative assessment was in focus. They were subsequently phone interviewed using a semi-structured interview guide regarding their experiences and attitudes towards formative assessment. Field notes from observations and interview transcriptions were analyzed using an inductive content analysis approach, and the concept of "everyday resistance" was used as a theoretical lens. RESULTS Three themes were identified: First, there were several barriers to the use of formative assessment methods in the clinical context, including subtle tactics of everyday resistance such as avoidance, deprioritizing, and contesting formative assessment methods. Secondly, junior doctors made careful selections when arranging a formative assessment. Finally, junior doctors had ambiguous attitudes towards the use of mandatory formative assessment methods and mixed experiences with their educational impact. CONCLUSIONS This study emphasizes that the use of formative assessment methods in the clinical setting is not a neutral and context-independent exercise, but rather is affected by a myriad of factors such as collegial relations, educational traditions, emotional issues, and subtle forms of resistance. An important implication for the health care sector will be to address these issues for formative assessment methods to be properly implemented in the clinic.
Collapse
Affiliation(s)
| | - Bente Malling
- Centre for Health Sciences Education, Aarhus University, Denmark
| |
Collapse
|
22
|
Rassos J, Melvin LJ, Panisko D, Kulasegaram K, Kuper A. Unearthing Faculty and Trainee Perspectives of Feedback in Internal Medicine: the Oral Case Presentation as a Model. J Gen Intern Med 2019; 34:2107-2113. [PMID: 31388910 PMCID: PMC6816591 DOI: 10.1007/s11606-019-05134-z] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.4] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 08/27/2018] [Revised: 02/05/2019] [Accepted: 05/21/2019] [Indexed: 11/26/2022]
Abstract
BACKGROUND The case presentation is a fundamental activity used in both patient care and trainee education, partly due to feedback from supervisor to trainee. Although feedback in medical education is well studied, prior studies have not focused on the perceptions of feedback by Internal Medicine supervisors and trainees as it relates to clinical activities like the case presentation. METHODS Semi-structured interviews were conducted with eight Internal Medicine physicians, and 18 Internal Medicine trainees (5 medical students, 13 residents) at the University of Toronto. Purposive sampling was used. Interviews were conducted and coded iteratively within a constructivist grounded theory approach until saturation was reached. RESULTS Supervisors and trainees recognized feedback as an important part of the case presentation that can be (1) explicit, labeled feedback or (2) implicit, unlabeled feedback. Both trainees and supervisors perceived that not enough feedback occurs, likely stemming from a hesitancy by supervisors to label implicit feedback, calling it an interruption instead. Although trainees were keenly aware of non-verbal feedback from their supervisors as implicit feedback, they often interpreted explicit constructive feedback negatively. Interestingly, the same feedback from senior residents was regarded as highly educational, as it was uncoupled from assessment. CONCLUSION Feedback occurs more frequently in case presentations than previously described, particularly in an implicit, unlabeled format. Even though under-recognized, trainees identify and utilize implicit feedback from supervisors, and coaching from senior residents, to develop learned behaviors. This is reassuring in the age of Competency-Based Medical Education, as feedback has an essential role in workplace-based assessment and promotion.
Collapse
Affiliation(s)
- James Rassos
- General Internal Medicine, University of Toronto, Toronto, Ontario, Canada.
| | - Lindsay J Melvin
- Division of General Internal Medicine, University Health Network and Faculty of Medicine, University of Toronto, Toronto Western Hospital, 399 Bathurst Street, Toronto, Ontario, M5T 2S8, Canada
| | - Daniel Panisko
- Division of General Internal Medicine, University Health Network and Faculty of Medicine, University of Toronto, Toronto Western Hospital, 399 Bathurst Street, Toronto, Ontario, M5T 2S8, Canada
| | | | - Ayelet Kuper
- Department of Medicine, Faculty of Medicine, University of Toronto, Toronto, Ontario, Canada
- Wilson Centre for Research in Education, University Health Network/University of Toronto, Toronto, Ontario, Canada
- Division of General Internal Medicine, Sunnybrook Health Sciences Centre, Toronto, Ontario, Canada
| |
Collapse
|
23
|
Kordestani Moghaddam A, Khankeh HR, Shariati M, Norcini J, Jalili M. Educational impact of assessment on medical students' learning at Tehran University of Medical Sciences: a qualitative study. BMJ Open 2019; 9:e031014. [PMID: 31362972 PMCID: PMC6677973 DOI: 10.1136/bmjopen-2019-031014] [Citation(s) in RCA: 8] [Impact Index Per Article: 1.6] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 11/04/2022] Open
Abstract
OBJECTIVES It has been shown that assessment strongly affects students' performance. A deeper insight needs to be gained into the interplay of assessment and learning. The aim of the current study was to develop a model to explain the educational impact of assessments on students' learning, before, during and after the test. DESIGN This study used semistructured interviews, focus group discussions and observation and collection of field notes. A qualitative methodology using the grounded theory data analysis approach was then used to generate an explanation of the process of how assessment impacts students' learning. SETTING School of Medicine, Tehran University of Medical Sciences. PARTICIPANTS Participants were medical students and teachers with first-hand experience or expertise in assessment as well as their willingness to participate in the study. Fifteen people (eight medical students, seven faculty members) were interviewed. One focus group discussion (with five students) was held. RESULTS The extracted concepts from our study were classified into four main categories. These categories include elements of the assessment programme which affect learning, the mechanism through which they exert their effects, contextual factors and the impact they have on learning. These elements and their interplay occur within an environment with its antecedent characteristics. CONCLUSIONS This study suggested a model for understanding the elements of the assessment which, within the context, affect learning, the mechanisms through which they impart their effects and the final outcomes obtained.
Collapse
Affiliation(s)
| | - Hamid Reza Khankeh
- Health in Emergency and Disaster Research Center, University of Social Welfare and Rehabilitation Sciences, Tehran, Iran
- Department of Clinical Science and Education, Karolinska Institute, Stockholm, Sweden
| | - Mohammad Shariati
- Department of Medical Education, Department of Community Medicine, School of Medicine, Tehran University of Medical Sciences, Tehran, Iran
| | - John Norcini
- Foundation for Advancement of International Medical Education and Research, Philadelphia, Pennsylvania, USA
| | - Mohammad Jalili
- Department of Medical Education, Tehran University of Medical Sciences, Tehran, Iran
- Department of Emergency Medicine, School of Medicine, Health Professions Education Research Center, Tehran University of Medical Sciences, Tehran, Iran
| |
Collapse
|
24
|
Abstract
OBJECTIVES The formative aspect of the mini-clinical evaluation exercise (mini-CEX) in postgraduate medical workplace-based assessment is intended to afford opportunities for active learning. Yet, there is little understanding of the perceived relationship between the mini-CEX and how trainees self-regulate their learning. Our objective was to explore trainees' perceptions of their mini-CEX experiences from a learning perspective, using Zimmerman's self-regulated learning theoretical framework as an interpretive lens. DESIGN Qualitative, using semi-structured interviews conducted in 2017. The interviews were analysed thematically. SETTING Geriatric medicine training. PARTICIPANTS Purposive sampling was employed to recruit geriatric medicine trainees in Melbourne, Australia. Twelve advanced trainees participated in the interviews. RESULTS Four themes were found with a cyclical inter-relationship between three of these themes: namely, goal setting, task translation and perceived outcome. These themes reflect the phases of the self-regulated learning framework. Each phase was influenced by the fourth theme, supervisor co-regulation. Goal setting had motivational properties that had significant impact on the later phases of the cycle. A 'tick box' goal aligned with an opportunistic approach and poorer perceived educational outcomes. Participants reported that external feedback following assessment was critical for their self-evaluation, affective responses and perceived outcomes. CONCLUSIONS Trainees perceived the performance of a mini-CEX as a complex, inter-related cyclical process, influenced at all stages by the supervisor. Based on our trainee perspectives of the mini-CEX, we conclude that supervisor engagement is essential to support trainees to individually regulate their learning in the clinical environment.
Collapse
Affiliation(s)
- Eva Kipen
- Department of Medical Education, University of Melbourne, Melbourne, Victoria, Australia
- Central Clinical School, Faculty of Medicine Nursing and Health Sciences, Monash University, Australia
- Alfred Hospital, Melbourne, Victory, Australia
| | - Eleanor Flynn
- Department of Medical Education, University of Melbourne, Melbourne, Victoria, Australia
| | - Robyn Woodward-Kron
- Department of Medical Education, University of Melbourne, Melbourne, Victoria, Australia
| |
Collapse
|
25
|
Scarff CE, Bearman M, Chiavaroli N, Trumble S. Trainees' perspectives of assessment messages: a narrative systematic review. MEDICAL EDUCATION 2019; 53:221-233. [PMID: 30672012 DOI: 10.1111/medu.13775] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 03/02/2018] [Revised: 04/17/2018] [Accepted: 10/16/2018] [Indexed: 06/09/2023]
Abstract
OBJECTIVES This study was designed as a narrative systematic literature review of medical specialist trainees' perspectives of the assessment messages they receive in the context of clinical performance assessments. The aim of the study was to determine if trainees value the information they receive through the formats designed to promote their development and, if not, the reasons for this. METHODS The authors searched the ERIC, EMBASE, Ovid MEDLINE and PsycINFO databases for articles published up to 16 June 2018 that present original data on trainees' perspectives of the assessment messages they receive in the context of work-based assessments (WBAs) and in-training assessments (ITAs) used within their training programmes. All authors screened 938 abstracts and 139 full-text articles were assessed after this. Descriptions of quantitative data and thematic analysis of qualitative data were used to present the opinions of trainees. RESULTS Thirty-three articles met the inclusion criteria. Twenty-six articles (79%) described trainees' perspectives in the context of WBA and the remaining articles referred to ITA formats. Wide-ranging opinions were reported. The analysis categorised these into three themes: trainees value developmental assessment messages; trainees become disengaged when assessment messages are not developmental, and trainees' views depend on the environment, the assessor and themselves. Some trainees reported that the assessment messages were valuable and provided input on their performance to guide their development, but many disagreed. In particular, the trainee's own level of engagement with the assessments influenced his or her perspectives on the messages received. CONCLUSIONS Trainees do not universally perceive that clinical performance assessments provide them with the valuable developmental input on their performance they were designed to do. Factors related to the environment, the assessor and themselves influence their perspectives.
Collapse
Affiliation(s)
- Catherine E Scarff
- Department of Medical Education, Melbourne Medical School, University of Melbourne, Melbourne, Victoria, Australia
| | - Margaret Bearman
- Centre for Research in Assessment and Digital Learning (CRADLE), Deakin University, Geelong, Victoria, Australia
| | - Neville Chiavaroli
- Department of Medical Education, Melbourne Medical School, University of Melbourne, Melbourne, Victoria, Australia
| | - Steve Trumble
- Department of Medical Education, Melbourne Medical School, University of Melbourne, Melbourne, Victoria, Australia
| |
Collapse
|
26
|
Stigt JA, Koele JH, Brand PLP, Jaarsma DAC, Slootweg IA. Workplace mentoring of residents in generic competencies by an independent coach. PERSPECTIVES ON MEDICAL EDUCATION 2018; 7:337-341. [PMID: 30187390 PMCID: PMC6191395 DOI: 10.1007/s40037-018-0452-7] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/08/2023]
Abstract
INTRODUCTION During postgraduate education in pulmonology, supervisors are responsible for training residents in generic competencies such as communication, professionalism and collaboration, but their focus commonly lies more on medical-technical competencies. As an alternative approach to supporting residents to develop generic skills, we developed a personal mentoring program with a non-medical professional as mentor. In this study, the residents' experiences with the mentoring program were evaluated. METHODS After an introductory session in which individual learning goals were established, pulmonology residents received at least six, 60-90-minute, individual, mentoring sessions largely consisting of feedback after being observed during daily clinical activities, over a period of 9 months. The residents' experiences with mentoring were explored through in-depth interviews followed by a qualitative content analysis. RESULTS From March to November 2016, ten residents in pulmonology completed the program. Despite initial scepticism, mentoring encouraged residents to reflect deeply on their professional interactions. This caused an increased awareness of the effects of their communication and behaviour on patients. Experimenting with communication and different behaviours in subsequent interactions felt rewarding and contributed to further development, resulting in increased self-confidence and job satisfaction. DISCUSSION Mentoring residents by non-medical coaching was associated with improved residents' proficiency in generic competencies.
Collapse
Affiliation(s)
- Jos A Stigt
- Department of Pulmonology, Isala Hospital, Zwolle, The Netherlands.
| | - Janine H Koele
- Department of Pulmonology, Isala Hospital, Zwolle, The Netherlands
| | - Paul L P Brand
- Isala Women's and Children's Hospital, Zwolle, The Netherlands
- UMCG Postgraduate School of Medicine, University of Groningen and University Medical Center Groningen, Groningen, The Netherlands
| | - Debbie A C Jaarsma
- Center for Education Development and Research in Health Professions, University of Groningen and University Medical Center Groningen, Groningen, The Netherlands
| | - Irene A Slootweg
- Department of Public Health and Primary Care, Leiden University Medical Center, Leiden, The Netherlands
| |
Collapse
|
27
|
Wiese A, Kilty C, Bennett D. Supervised workplace learning in postgraduate training: a realist synthesis. MEDICAL EDUCATION 2018; 52. [PMCID: PMC6175369 DOI: 10.1111/medu.13655] [Citation(s) in RCA: 36] [Impact Index Per Article: 6.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/14/2023]
Abstract
Context This paper presents a realist synthesis of the literature that began with the objective of developing a theory of workplace learning specific to postgraduate medical education (PME). As the review progressed, we focused on informal learning between trainee and senior doctor or supervisor, asking what mechanisms occur between trainee and senior doctor that lead to the outcomes of PME, and what contexts shape the operation of these mechanisms and the outcomes they produce? Methods We followed the procedures outlined in the RAMESES Publication Standards for Realist Synthesis. We searched the English‐language literature published between 1995 and 2017 for empirical papers related to informal workplace learning between supervisor and trainee, excluding formal interventions such as workplace‐based assessment. We made a pragmatic decision to exclude general practice training to keep the review within manageable limits. Results We reviewed 5197 papers and selected 90. Synthesis revealed three workplace learning processes occurring between supervisors and trainees, each underpinned by a pair of mechanisms: supervised participation in practice (entrustment and support seeking); mutual observation of practice (monitoring and modelling), and dialogue during practice (meaning making and feedback). These mechanisms result in outcomes of PME, including safe participation in practice, learning skills, attitudes and behaviours and professional identity development. Contexts shaping the outcomes of these mechanisms were identified at individual, interpersonal, local and systems levels. Conclusions Our realist theory of workplace learning between supervisors and trainees is informed by theory and empirical research. It highlights the two‐way nature of supervision, the importance of trainees’ agency in their own learning and the deleterious effect of fragmented working patterns on supervisor–trainee learning mechanisms. Further empirical research is required to test and refine this theory. In the meantime, it provides a useful framework for the design of supportive learning environments and for the preparation of supervisors and trainees for their roles in workplace learning. The authors describe a realist theory of learning between supervisor and postgraduate trainee in the workplace, emphasising the role of individual, interpersonal, local and systems contexts in shaping key mechanisms and their outcomes.
Collapse
Affiliation(s)
- Anel Wiese
- Medical Education UnitUniversity College CorkCorkIreland
| | - Caroline Kilty
- Medical Education UnitUniversity College CorkCorkIreland
| | | |
Collapse
|
28
|
Düzgün ŞA, Zeren S, Bayhan Z. Objectively structured verbal examination to assess surgical clerkship education: An evaluation of students' perception. Turk J Surg 2018; 34:9-12. [PMID: 29756098 DOI: 10.5152/turkjsurg.2017.3681] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/06/2016] [Accepted: 07/20/2017] [Indexed: 11/22/2022]
Abstract
Objective The objectivity and reliability of examining methods are controversial. We subjected fourth-year medical students to a specially designed verbal exam which we called objectively structured verbal examination. We aimed to evaluate feedback from students about objectively structured verbal examination as an assessment instrument for gauging their surgical knowledge. Material and Methods Objectively structured verbal examination modules were developed according to the learning goals of the surgical clerkship. Upon finishing surgery rotation, the students were subjected to objectively structured verbal examination as part of their final evaluation. The students' perception of objectively structured verbal examination was assessed by their responses to a questionnaire. Results Forty-two of 58 students returned filled questionnaires. Objectively structured verbal examination was accepted by 72% of the students as an objective tool, and 86% of them found it enabled unbiased evaluation. Overall, most students expressed positive feedback regarding objectively structured verbal examination. Conclusion The feedback received from students showed that objectively structured verbal examination is a reliable and objective method to assess their knowledge. This feedback reflects that objectively structured verbal examination merits further development and enhancement.
Collapse
Affiliation(s)
| | - Sezgin Zeren
- Department of General Surgery Dumlupınar University School of Medicine, Kütahya, Turkey
| | - Zülfü Bayhan
- Department of General Surgery Dumlupınar University School of Medicine, Kütahya, Turkey
| |
Collapse
|
29
|
Berryman EK, Leonard DJ, Gray AR, Pinnock R, Taylor B. Self-Reflected Well-Being via a Smartphone App in Clinical Medical Students: Feasibility Study. JMIR MEDICAL EDUCATION 2018. [PMID: 29514774 PMCID: PMC5863013 DOI: 10.2196/mededu.9128] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/13/2023]
Abstract
BACKGROUND Well-being in medical students has become an area of concern, with a number of studies reporting high rates of clinical depression, anxiety, burnout, and suicidal ideation in this population. OBJECTIVE The aim of this study was to increase awareness of well-being in medical students by using a smartphone app. The primary objective of this study was to determine the validity and feasibility of the Particip8 app for student self-reflected well-being data collection. METHODS Undergraduate medical students of the Dunedin School of Medicine were recruited into the study. They were asked to self-reflect daily on their well-being and to note what experiences they had encountered during that day. Qualitative data were also collected both before and after the study in the form of focus groups and "free-text" email surveys. All participants consented for the data collected to be anonymously reported to the medical faculty. RESULTS A total of 29 participants (69%, 20/29 female; 31%, 9/29 male; aged 21-30 years) were enrolled, with overall median compliance of 71% at the study day level. The self-reflected well-being scores were associated with both positive and negative experiences described by the participants, with most negative experiences associated with around 20% lower well-being scores for that day; the largest effect being "receiving feedback that was not constructive or helpful," and the most positive experiences associated with around 20% higher scores for that day. CONCLUSIONS The study of daily data collection via the Particip8 app was found to be feasible, and the self-reflected well-being scores showed validity against participant's reflections of experiences during that day.
Collapse
Affiliation(s)
| | - Daniel J Leonard
- Dunedin School of Medicine, University of Otago, Dunedin, New Zealand
| | - Andrew R Gray
- Dunedin School of Medicine, University of Otago, Dunedin, New Zealand
| | - Ralph Pinnock
- Dunedin School of Medicine, University of Otago, Dunedin, New Zealand
| | - Barry Taylor
- Dunedin School of Medicine, University of Otago, Dunedin, New Zealand
| |
Collapse
|
30
|
Beck S, Schirlo C, Breckwoldt J. How the Start into the Clinical Elective Year Could be Improved: Qualitative Results and Recommendations from Student Interviews. GMS JOURNAL FOR MEDICAL EDUCATION 2018; 35:Doc14. [PMID: 29497699 PMCID: PMC5827187 DOI: 10.3205/zma001161] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.2] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Subscribe] [Scholar Register] [Received: 03/30/2017] [Revised: 07/25/2017] [Accepted: 08/17/2017] [Indexed: 05/10/2023]
Abstract
Background: Entering the Clinical Elective Year (CEY) is a challenging transition phase for undergraduate medical students. Students become members of a professional team, thereby taking over certain tasks, which are executed more or less independently. Factors which facilitate (or impede) this transition in the perception of students are not well described. We therefore wanted to explore, what students perceived to be helpful during the first phase of the CEY and possibly derive respective recommendations. Methods: We conducted semi-structured interviews with 5th year medical students after they had completed the first two months of their CEY. Students were asked which problems they had faced and how they felt prepared for the CEY. Interviews were audio-recorded, transcribed, and analysed by qualitative content analysis. Results: From 34 interviews, we included 28 into analysis. Overall, 24 students were satisfied or very satisfied with their start into the CEY. Satisfaction was expressed with respect to workplace experiences, learning progress, responsibilities and team integration. Especially, students appreciated if they were integrated as active members of the team, were given responsibility for certain units of work, and received well-structured formal teaching and supervision. Students had divergent opinions about the quality of teaching and supervision, about their own achievements, and the recognition they received. Students recommended improvements in respect to formal teaching and supervision by clinical supervisors, preparation of the CEY by university, and supporting structures in the hosting institution. Conclusion: Students in this study were generally satisfied with the first two months of their CEY. Facilitating factors were active and responsible involvement into routine patient care, and high quality formal teaching and supervision. Findings may inform universities, teaching hospitals, and students how to better shape the first phase of the CEY.
Collapse
Affiliation(s)
- Samuel Beck
- University of Zurich, Faculty of Medicine, Dean's Office, Zurich, Switzerland
| | - Christian Schirlo
- University of Zurich, Faculty of Medicine, Dean's Office, Zurich, Switzerland
| | - Jan Breckwoldt
- University of Zurich, Faculty of Medicine, Dean's Office, Zurich, Switzerland
- *To whom correspondence should be addressed: Jan Breckwoldt, University of Zurich, Faculty of Medicine, Dean`s Office, Pestalozzistr. 3-5, CH-8091 Zurich, Switzerland, Tel.: +41 (0)44/634-1075, Fax: +41 (0)44/634-1088, E-mail:
| |
Collapse
|
31
|
Bing-You R, Hayes V, Varaklis K, Trowbridge R, Kemp H, McKelvy D. Feedback for Learners in Medical Education: What Is Known? A Scoping Review. ACADEMIC MEDICINE : JOURNAL OF THE ASSOCIATION OF AMERICAN MEDICAL COLLEGES 2017; 92:1346-1354. [PMID: 28177958 DOI: 10.1097/acm.0000000000001578] [Citation(s) in RCA: 127] [Impact Index Per Article: 18.1] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/19/2023]
Abstract
PURPOSE To conduct a scoping review of the literature on feedback for learners in medical education. METHOD In 2015-2016, the authors searched the Ovid MEDLINE, ERIC, CINAHL, ProQuest Dissertations and Theses Global, Web of Science, and Scopus databases and seven medical education journals (via OvidSP) for articles published January 1980-December 2015. Two reviewers screened articles for eligibility with inclusion criteria. All authors extracted key data and analyzed data descriptively. RESULTS The authors included 650 articles in the review. More than half (n = 341) were published during 2010-2015. Many centered on medical students (n = 274) or residents (n = 192); some included learners from other disciplines (n = 57). Most (n = 633) described methods used for giving feedback; some (n = 95) described opinions and recommendations regarding feedback. Few studies assessed approaches to feedback with randomized, educational trials (n = 49) or described changes in learner behavior after feedback (n = 49). Even fewer assessed the impact of feedback on patient outcomes (n = 28). CONCLUSIONS Feedback is considered an important means of improving learner performance, as evidenced by the number of articles outlining recommendations for feedback approaches. The literature on feedback for learners in medical education is broad, fairly recent, and generally describes new or altered curricular approaches that involve feedback for learners. High-quality, evidence-based recommendations for feedback are lacking. In addition to highlighting calls to reassess the concepts and complex nature of feedback interactions, the authors identify several areas that require further investigation.
Collapse
Affiliation(s)
- Robert Bing-You
- R. Bing-You is professor, Tufts University School of Medicine, and vice president for medical education, Maine Medical Center, Portland, Maine. V. Hayes is clinical assistant professor, Tufts University School of Medicine, and faculty member, Department of Family Medicine, Maine Medical Center, Portland, Maine. K. Varaklis is clinical associate professor, Tufts University School of Medicine, and residency program director in obstetrics and gynecology, Maine Medical Center, Portland, Maine. R. Trowbridge is associate professor, Tufts University School of Medicine, and director of undergraduate medical education, Department of Medicine, Maine Medical Center, Portland, Maine. H. Kemp is medical librarian, Maine Medical Center, Portland, Maine. D. McKelvy is manager of library and knowledge services, Maine Medical Center, Portland, Maine
| | | | | | | | | | | |
Collapse
|
32
|
Sagasser MH, Fluit CRMG, van Weel C, van der Vleuten CPM, Kramer AWM. How Entrustment Is Informed by Holistic Judgments Across Time in a Family Medicine Residency Program: An Ethnographic Nonparticipant Observational Study. ACADEMIC MEDICINE : JOURNAL OF THE ASSOCIATION OF AMERICAN MEDICAL COLLEGES 2017; 92:792-799. [PMID: 28557945 DOI: 10.1097/acm.0000000000001464] [Citation(s) in RCA: 7] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/14/2023]
Abstract
PURPOSE Entrustment has mainly been conceptualized as delegating discrete professional tasks. Because residents provide most of their patient care independently, not all resident performance is visible to supervisors; the entrustment process involves more than granting discrete tasks. This study explored how supervisors made entrustment decisions based on residents' performance in a long-term family medicine training program. METHOD A qualitative nonparticipant observational study was conducted in 2014-2015 at competency-based family medicine residency programs in the Netherlands. Seven supervisor-resident pairs participated. During two days, one researcher observed first-year residents' patient encounters, debriefing sessions, and supervisor-resident educational meetings and interviewed them separately afterwards. Data were collected and analyzed using iterative, phenomenological inductive research methodology. RESULTS The entrustment process developed over three phases. Supervisors based their initial entrustment on prior knowledge about the resident. In the ensuing two weeks, entrustment decisions regarding independent patient care were derived from residents' observed general competencies necessary for a range of health problems (clinical reasoning, decision making, relating to patients); medical knowledge and skills; and supervisors' intuition. Supervisors provided supervision during and after encounters. Once residents performed independently, supervisors kept reevaluating their decisions, informed by residents' overall growth in competencies rather than by adhering to a predefined set of tasks. CONCLUSIONS Supervisors in family medicine residency training took a holistic approach to trust, based on general competencies, knowledge, skills, and intuition. Entrustment started before training and developed over time. Building trust is a mutual process between supervisor and resident, requiring a good working relationship.
Collapse
Affiliation(s)
- Margaretha H Sagasser
- M.H. Sagasser was educationalist and researcher, Department of Primary and Community Care, Radboud University Medical Center, Nijmegen, the Netherlands, at the time of the study. The author is now educationalist and researcher, Huisartsopleiding Nederland, Utrecht, the Netherlands. C.R.M.G. Fluit is head, Center on Research in Learning and Education, Radboud University Medical Center Health Academy, Nijmegen, the Netherlands. C. van Weel is professor emeritus of general practice, Department of Primary and Community Care, Radboud University Medical Center, Nijmegen, the Netherlands, and honorary professor of primary health care research, Department of Health Services Research and Policy, Australian National University, Canberra, ACT, Australia. C.P.M. van der Vleuten is professor of education and director, School of Health Professions Education, Faculty of Health, Medicine, and Life Sciences, Maastricht University, the Netherlands. A.W.M. Kramer is general practitioner, professor of general practice, and head, Family Medicine Residency Program, Department of Public Health and Primary Care, Leiden University Medical Center, Leiden, the Netherlands
| | | | | | | | | |
Collapse
|
33
|
Lee V, Brain K, Martin J. Factors Influencing Mini-CEX Rater Judgments and Their Practical Implications: A Systematic Literature Review. ACADEMIC MEDICINE : JOURNAL OF THE ASSOCIATION OF AMERICAN MEDICAL COLLEGES 2017; 92:880-887. [PMID: 28030422 DOI: 10.1097/acm.0000000000001537] [Citation(s) in RCA: 10] [Impact Index Per Article: 1.4] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/06/2023]
Abstract
PURPOSE At present, little is known about how mini-clinical evaluation exercise (mini-CEX) raters translate their observations into judgments and ratings. The authors of this systematic literature review aim both to identify the factors influencing mini-CEX rater judgments in the medical education setting and to translate these findings into practical implications for clinician assessors. METHOD The authors searched for internal and external factors influencing mini-CEX rater judgments in the medical education setting from 1980 to 2015 using the Ovid MEDLINE, PsycINFO, ERIC, PubMed, and Scopus databases. They extracted the following information from each study: country of origin, educational level, study design and setting, type of observation, occurrence of rater training, provision of feedback to the trainee, research question, and identified factors influencing rater judgments. The authors also conducted a quality assessment for each study. RESULTS Seventeen articles met the inclusion criteria. The authors identified both internal and external factors that influence mini-CEX rater judgments. They subcategorized the internal factors into intrinsic rater factors, judgment-making factors (conceptualization, interpretation, attention, and impressions), and scoring factors (scoring integration and domain differentiation). CONCLUSIONS The current theories of rater-based judgment have not helped clinicians resolve the issues of rater idiosyncrasy, bias, gestalt, and conflicting contextual factors; therefore, the authors believe the most important solution is to increase the justification of rater judgments through the use of specific narrative and contextual comments, which are more informative for trainees. Finally, more real-world research is required to bridge the gap between the theory and practice of rater cognition.
Collapse
Affiliation(s)
- Victor Lee
- V. Lee is codirector of emergency medicine training, Department of Emergency Medicine, Austin Health, Heidelberg, Victoria, Australia.K. Brain is doctor, South West Healthcare, Warrnambool, Victoria, Australia.J. Martin is associate professor and director, Medical Student Programs, Monash University and Deakin University, Eastern Health Clinical School, Box Hill, Victoria, Australia
| | | | | |
Collapse
|
34
|
Micallef R, Kayyali R. Factors affecting a face-to-face learning event. INTERNATIONAL JOURNAL OF PHARMACY PRACTICE 2017; 26:183-190. [PMID: 28514043 DOI: 10.1111/ijpp.12373] [Citation(s) in RCA: 4] [Impact Index Per Article: 0.6] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/28/2016] [Accepted: 03/13/2017] [Indexed: 10/19/2022]
Abstract
BACKGROUND Pharmacy professionals have multiple opportunities to attend learning events run by a series of providers. However, there has never been a large-scale evaluation of events. Currently, formats of learning sessions differ by provider with no optimum model identified. Pharmacy Education South London was formed in April 2014 for the provision of education and training for pharmacy professionals in South London, running sessions multiple times across various locations. OBJECTIVE The objective of this work was to identify factors that influence the perceived success of a face-to-face supplementary education and training event from the perspective of attendees. METHODS Evaluation forms from 600 participants at training events followed by semi-structured individual interviews with 11 participants. KEY FINDINGS Participants over 55 years were more likely to attend lecture style events versus those aged under 25 years who attended more workshops (P < 0.001); there was no correlation with gender. About 57.3% (n = 344) of participants agreed fully that the event increased their understanding of the topics, although only 38.5% (n = 231) stated that it would change their practice. Themes influencing an event fell into three broad themes: personal reasons affecting attendance, success factors for the session and application of learning, all with related subthemes. Subthemes included commitments, convenience, awareness, topic and personal relevance, content and delivery and reference, review and action. CONCLUSIONS In publicising events, the topic, including the driver for the topic and the skills that will be obtained, the speaker and their experience plus how learning can be applied after the event should be included.
Collapse
Affiliation(s)
- Ricarda Micallef
- Department of Pharmacy, Kingston University, Kingston Upon Thames, Surrey, UK
| | - Reem Kayyali
- Department of Pharmacy, Kingston University, Kingston Upon Thames, Surrey, UK
| |
Collapse
|
35
|
Stephan A, Cheung G. Clinical teaching and supervision in postgraduate psychiatry training: the trainee perspective. Australas Psychiatry 2017; 25:191-197. [PMID: 27879425 DOI: 10.1177/1039856216679539] [Citation(s) in RCA: 9] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 11/17/2022]
Abstract
OBJECTIVE We aimed to explore psychiatry trainees' perspectives on clinical teaching and supervision as well as how this might be improved. METHOD New Zealand trainees ( n=51) completed online the Maastricht Clinical Teaching Questionnaire (MCQT) and three open-ended questions. RESULTS The majority rated 'agree' or 'strongly agree' to all items of the MCQT. Weaknesses in the feedback process including observation, feedback provision, and formulating learning goals were highlighted. College training requirements and workplace environment were identified as factors impacting on clinical teaching and supervision. CONCLUSIONS A model was proposed to enhance awareness of the various factors involved in the feedback process.
Collapse
Affiliation(s)
- Alice Stephan
- Director of Training (Psychiatry), Upper Central North Island, Waikato District Health Board, Hamilton, New Zealand
| | - Gary Cheung
- Senior Lecturer, Department of Psychological Medicine, The University of Auckland, Auckland, New Zealand
| |
Collapse
|
36
|
Ginsburg S, van der Vleuten CP, Eva KW, Lingard L. Cracking the code: residents' interpretations of written assessment comments. MEDICAL EDUCATION 2017; 51:401-410. [PMID: 28093833 DOI: 10.1111/medu.13158] [Citation(s) in RCA: 46] [Impact Index Per Article: 6.6] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 01/13/2016] [Revised: 02/26/2016] [Accepted: 07/18/2016] [Indexed: 05/09/2023]
Abstract
CONTEXT Interest is growing in the use of qualitative data for assessment. Written comments on residents' in-training evaluation reports (ITERs) can be reliably rank-ordered by faculty attendings, who are adept at interpreting these narratives. However, if residents do not interpret assessment comments in the same way, a valuable educational opportunity may be lost. OBJECTIVES Our purpose was to explore residents' interpretations of written assessment comments using mixed methods. METHODS Twelve internal medicine (IM) postgraduate year 2 (PGY2) residents were asked to rank-order a set of anonymised PGY1 residents (n = 48) from a previous year in IM based solely on their ITER comments. Each PGY1 was ranked by four PGY2s; generalisability theory was used to assess inter-rater reliability. The PGY2s were then interviewed separately about their rank-ordering process, how they made sense of the comments and how they viewed ITERs in general. Interviews were analysed using constructivist grounded theory. RESULTS Across four PGY2 residents, the G coefficient was 0.84; for a single resident it was 0.56. Resident rankings correlated extremely well with faculty member rankings (r = 0.90). Residents were equally adept at reading between the lines to construct meaning from the comments and used language cues in ways similarly reported in faculty attendings. Participants discussed the difficulties of interpreting vague language and provided perspectives on why they thought it occurs (time, discomfort, memorability and the permanency of written records). They emphasised the importance of face-to-face discussions, the relative value of comments over scores, staff-dependent variability of assessment and the perceived purpose and value of ITERs. They saw particular value in opportunities to review an aggregated set of comments. CONCLUSIONS Residents understood the 'hidden code' in assessment language and their ability to rank-order residents based on comments matched that of faculty. Residents seemed to accept staff-dependent variability as a reality. These findings add to the growing evidence that supports the use of narrative comments and subjectivity in assessment.
Collapse
Affiliation(s)
- Shiphra Ginsburg
- Department of Medicine, Faculty of Medicine, University of Toronto, Toronto, Ontario, Canada
| | - Cees Pm van der Vleuten
- Department of Educational Development and Research, Faculty of Health, Medicine and Life Sciences, Maastricht University, Maastricht, the Netherlands
| | - Kevin W Eva
- Department of Medicine, Faculty of Medicine, University of British Columbia, Vancouver, British Columbia, Canada
| | - Lorelei Lingard
- Department of Medicine, Schulich School of Medicine and Dentistry, Western University, London, Ontario, Canada
| |
Collapse
|
37
|
Chan CYW, Sum MY, Lim WS, Chew NWM, Samarasekera DD, Sim K. Adoption and correlates of Postgraduate Hospital Educational Environment Measure (PHEEM) in the evaluation of learning environments - A systematic review . MEDICAL TEACHER 2016; 38:1248-1255. [PMID: 27564848 DOI: 10.1080/0142159x.2016.1210108] [Citation(s) in RCA: 12] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/06/2023]
Abstract
BACKGROUND The Postgraduate Hospital Educational Environment Measure (PHEEM) is a highly reliable and valid instrument to measure the educational environment during post graduate medical training. This review extends earlier reports by evaluating the extant adoption of PHEEM in various international clinical training sites, and its significant correlations in order to expand our understanding on the use of PHEEM and facilitate future applications and research. METHOD A systematic literature review was conducted on all articles between 2005 and October 2015 that adopted and reported data using the PHEEM. RESULTS Overall 30 studies were included, encompassing data from 14 countries internationally. Notable differences in the PHEEM scores were found between different levels of training, disciplines, and clinical training sites. Common strengths and weaknesses in learning environments were observed and there were significant correlations between PHEEM scores and In-Training Exam (ITE) performance (positive correlation) and level of burnout (negative correlation), respectively. CONCLUSIONS PHEEM is widely adopted in different learning settings, and is a useful tool to identify the strengths and weaknesses of an educational environment. Future research can examine other correlates of PHEEM and longitudinal changes in interventional studies.
Collapse
Affiliation(s)
| | - Min Yi Sum
- b Research Division , Institute of Mental Health , Singapore , Singapore
| | - Wee Shiong Lim
- c Centre for Geriatric Medicine, Cognition and Memory Disorders Service , Tan Tock Seng Hospital , Singapore , Singapore
| | | | - Dujeepa D Samarasekera
- e Centre for Medical Education , National University of Singapore , Singapore , Singapore
| | - Kang Sim
- a Department of General Psychiatry , Institute of Mental Health , Singapore , Singapore
- b Research Division , Institute of Mental Health , Singapore , Singapore
| |
Collapse
|
38
|
Castanelli DJ, Jowsey T, Chen Y, Weller JM. Perceptions of purpose, value, and process of the mini-Clinical Evaluation Exercise in anesthesia training. Can J Anaesth 2016; 63:1345-1356. [DOI: 10.1007/s12630-016-0740-9] [Citation(s) in RCA: 23] [Impact Index Per Article: 2.9] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/16/2016] [Revised: 08/15/2016] [Accepted: 09/13/2016] [Indexed: 10/21/2022] Open
|
39
|
Ross S, Dudek N, Halman S, Humphrey-Murto S. Context, time, and building relationships: bringing in situ feedback into the conversation. MEDICAL EDUCATION 2016; 50:893-895. [PMID: 27562888 DOI: 10.1111/medu.13138] [Citation(s) in RCA: 4] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/06/2023]
|
40
|
Aggarwal M, Singh S, Sharma A, Singh P, Bansal P. Impact of structured verbal feedback module in medical education: A questionnaire- and test score-based analysis. Int J Appl Basic Med Res 2016; 6:220-5. [PMID: 27563592 PMCID: PMC4979308 DOI: 10.4103/2229-516x.186968] [Citation(s) in RCA: 5] [Impact Index Per Article: 0.6] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/18/2022] Open
Abstract
Introduction: Feedback is a divalent bond between the supplier (teacher) and the recipient (student). The strength of the bond depends on the instructional design of the feedback. Feedback is central to medical education in promoting self-directed learning in students. In the present study, a structured verbal feedback module was prepared, implemented, and evaluated. Methods: The study was done on 280 students from four consecutive batches (2011 to 2014) of the 1st year MBBS students exposed to different types and modes of feedback. Analysis was done using student feedback questionnaire for the perception of students to verbal feedback. Quantitative analysis using post hoc test and ANOVA for the impact of type of feedback (verbal or written) and effect of modes (individual or group) of verbal feedback on test score performance were done. Result: In this study, ≥95% of the students preferred verbal feedback of both positive and negative attributes in student questionnaires. It was observed that verbal feedback sessions made a difference of up to 2–2.4 grade points in the mean score of batch when compared to the written feedback. The initial mean test score (T1) of 2011 + 2012 and 2013 + 2014 was not statistically significant (P = 0.113). But, in all subsequent tests (T2, T3, and T4), there was a statistically significant difference in the mean test scores (P = 0.000). Conclusion: (1) Students prefer verbal one-to-one feedback over written feedback. (2) Verbal feedback changes learning process and causes sustained improvement in learning strategies.
Collapse
Affiliation(s)
- Meenakshi Aggarwal
- Department of Anatomy, Dayanand Medical College and Hospital, Ludhiana, Punjab, India
| | - Sonia Singh
- Department of Anatomy, Dayanand Medical College and Hospital, Ludhiana, Punjab, India
| | - Anu Sharma
- Department of Anatomy, Dayanand Medical College and Hospital, Ludhiana, Punjab, India
| | - Poonam Singh
- Department of Anatomy, Dayanand Medical College and Hospital, Ludhiana, Punjab, India
| | - Priya Bansal
- Department of SPM, Dayanand Medical College and Hospital, Ludhiana, Punjab, India
| |
Collapse
|
41
|
Pugh D, Regehr G. Taking the sting out of assessment: is there a role for progress testing? MEDICAL EDUCATION 2016; 50:721-729. [PMID: 27295476 DOI: 10.1111/medu.12985] [Citation(s) in RCA: 27] [Impact Index Per Article: 3.4] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 09/23/2015] [Revised: 11/13/2015] [Accepted: 11/25/2015] [Indexed: 06/06/2023]
Abstract
CONTEXT It has long been understood that assessment is an important driver for learning. However, recently, there has been growing recognition that this powerful driving force of assessment has the potential to undermine curricular efforts. When the focus of assessment is to categorise learners into competent or not (i.e. assessment of learning), rather than being a tool to promote continuous learning (i.e. assessment for learning), there may be unintended consequences that ultimately hinder learning. In response, there has been a movement toward constructing assessment not only as a measurement problem, but also as an instructional design problem, and exploring more programmatic models of assessment across the curriculum. Progress testing is one form of assessment that has been introduced, in part, to attempt to address these concerns. However, in order for any assessment tool to be successful in promoting learning, careful consideration must be given to its implementation. OBJECTIVE The purpose of this paper is to consider the implications of implementing progress testing within practice, and how this might promote or impede learning in the three phases of assessment (pre-test, pure-test and post-test). METHODS We will examine the literature on how assessment drives learning and how this might apply to progress testing. We will also explore the distinction between assessment of learning and assessment for learning, including ways in which they overlap and differ. We end by discussing how the properties of an assessment tool can be harnessed to optimise learning. CONCLUSIONS Progress tests are one potential solution to the problem of removing (or at least lessening) the sting associated with assessment. If implemented with careful thought and consideration, progress tests can be used to support the type of deep, meaningful and continuous learning that we are trying to instill in our learners.
Collapse
Affiliation(s)
- Debra Pugh
- Department of Medicine, University of Ottawa, Ottawa, Ontario, Canada
| | - Glenn Regehr
- Faculty of Medicine, Centre for Health Education Scholarship, University of British Columbia, Vancouver, British Columbia, Canada
| |
Collapse
|
42
|
Massie J, Ali JM. Workplace-based assessment: a review of user perceptions and strategies to address the identified shortcomings. ADVANCES IN HEALTH SCIENCES EDUCATION : THEORY AND PRACTICE 2016; 21:455-73. [PMID: 26003590 DOI: 10.1007/s10459-015-9614-0] [Citation(s) in RCA: 86] [Impact Index Per Article: 10.8] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 11/24/2014] [Accepted: 05/13/2015] [Indexed: 05/12/2023]
Abstract
Workplace based assessments (WBAs) are now commonplace in postgraduate medical training. User acceptability and engagement is essential to the success of any medical education innovation. To this end, possessing an insight into trainee and trainer perceptions towards WBAs will help identify the major problems, permitting strategies to be introduced to improve WBA implementation. A review of literature was performed to identify studies examining trainee and trainer perceptions towards WBAs. Studies were excluded if non-English or sampling a non-medical/dental population. The identified literature was synthesised for the purpose of this critical narrative review. It is clear that there is widespread negativity towards WBAs in the workplace. This has negatively impacted on the effectiveness of WBA tools as learning aids. This negativity exists in trainees but also to an extent in their trainers. Insight gained from the literature reveals three dominant problems with WBA implementation: poor understanding as to the purpose of WBAs; insufficient time available for undertaking these assessments; and inadequate training of trainers. Approaches to addressing these three problems with WBA implementation are discussed. It is likely that a variety of solutions will be required. The prevalence of negativity towards WBAs is substantial in both trainees and trainers, eroding the effectiveness of learning that is consequent upon them. The educational community must now listen to the concerns being raised by the users and consider the range of strategies being proposed to improve the experiences of trainees, and their trainers.
Collapse
Affiliation(s)
- Jonathan Massie
- School of Clinical Medicine, University of Cambridge, Cambridge, UK
| | - Jason M Ali
- Department of Surgery, University of Cambridge, BOX 202, Addenbrookes Hospital, Cambridge, CB2 0QQ, UK.
| |
Collapse
|
43
|
McGill DA, van der Vleuten CPM, Clarke MJ. Construct validation of judgement-based assessments of medical trainees' competency in the workplace using a "Kanesian" approach to validation. BMC MEDICAL EDUCATION 2015; 15:237. [PMID: 26715145 PMCID: PMC4696206 DOI: 10.1186/s12909-015-0520-1] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.2] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 10/01/2015] [Accepted: 12/16/2015] [Indexed: 05/30/2023]
Abstract
BACKGROUND Evaluations of clinical assessments that use judgement-based methods have frequently shown them to have sub-optimal reliability and internal validity evidence for their interpretation and intended use. The aim of this study was to enhance that validity evidence by an evaluation of the internal validity and reliability of competency constructs from supervisors' end-of-term summative assessments for prevocational medical trainees. METHODS The populations were medical trainees preparing for full registration as a medical practitioner (74) and supervisors who undertook ≥2 end-of-term summative assessments (n = 349) from a single institution. Confirmatory Factor Analysis was used to evaluate assessment internal construct validity. The hypothesised competency construct model to be tested, identified by exploratory factor analysis, had a theoretical basis established in workplace-psychology literature. Comparisons were made with competing models of potential competency constructs including the competency construct model of the original assessment. The optimal model for the competency constructs was identified using model fit and measurement invariance analysis. Construct homogeneity was assessed by Cronbach's α. Reliability measures were variance components of individual competency items and the identified competency constructs, and the number of assessments needed to achieve adequate reliability of R > 0.80. RESULTS The hypothesised competency constructs of "general professional job performance", "clinical skills" and "professional abilities" provides a good model-fit to the data, and a better fit than all alternative models. Model fit indices were χ2/df = 2.8; RMSEA = 0.073 (CI 0.057-0.088); CFI = 0.93; TLI = 0.95; SRMR = 0.039; WRMR = 0.93; AIC = 3879; and BIC = 4018). The optimal model had adequate measurement invariance with nested analysis of important population subgroups supporting the presence of full metric invariance. Reliability estimates for the competency construct "general professional job performance" indicated a resource efficient and reliable assessment for such a construct (6 assessments for an R > 0.80). Item homogeneity was good (Cronbach's alpha = 0.899). Other competency constructs are resource intensive requiring ≥11 assessments for a reliable assessment score. CONCLUSION Internal validity and reliability of clinical competence assessments using judgement-based methods are acceptable when actual competency constructs used by assessors are adequately identified. Validation for interpretation and use of supervisors' assessment in local training schemes is feasible using standard methods for gathering validity evidence.
Collapse
Affiliation(s)
- D A McGill
- Department of Cardiology, The Canberra Hospital, Garran, ACT 2605, Australia.
| | - C P M van der Vleuten
- Department of Educational Research and Development, Maastricht University, Maastricht, The Netherlands
| | - M J Clarke
- Clinical Trial Service Unit, University of Oxford, Oxford, UK
| |
Collapse
|
44
|
Lefroy J, Watling C, Teunissen PW, Brand P. Guidelines: the do's, don'ts and don't knows of feedback for clinical education. PERSPECTIVES ON MEDICAL EDUCATION 2015; 4:284-299. [PMID: 26621488 PMCID: PMC4673072 DOI: 10.1007/s40037-015-0231-7] [Citation(s) in RCA: 185] [Impact Index Per Article: 20.6] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/09/2023]
Abstract
INTRODUCTION The guidelines offered in this paper aim to amalgamate the literature on formative feedback into practical Do's, Don'ts and Don't Knows for individual clinical supervisors and for the institutions that support clinical learning. METHODS The authors built consensus by an iterative process. Do's and Don'ts were proposed based on authors' individual teaching experience and awareness of the literature, and the amalgamated set of guidelines were then refined by all authors and the evidence was summarized for each guideline. Don't Knows were identified as being important questions to this international group of educators which if answered would change practice. The criteria for inclusion of evidence for these guidelines were not those of a systematic review, so indicators of strength of these recommendations were developed which combine the evidence with the authors' consensus. RESULTS A set of 32 Do and Don't guidelines with the important Don't Knows was compiled along with a summary of the evidence for each. These are divided into guidelines for the individual clinical supervisor giving feedback to their trainee (recommendations about both the process and the content of feedback) and guidelines for the learning culture (what elements of learning culture support the exchange of meaningful feedback, and what elements constrain it?) CONCLUSION Feedback is not easy to get right, but it is essential to learning in medicine, and there is a wealth of evidence supporting the Do's and warning against the Don'ts. Further research into the critical Don't Knows of feedback is required. A new definition is offered: Helpful feedback is a supportive conversation that clarifies the trainee's awareness of their developing competencies, enhances their self-efficacy for making progress, challenges them to set objectives for improvement, and facilitates their development of strategies to enable that improvement to occur.
Collapse
Affiliation(s)
- Janet Lefroy
- Keele University School of Medicine, Clinical Education Centre RSUH, ST4 6QG, Staffordshire, UK.
| | - Chris Watling
- Schulich School of Medicine and Dentistry, Western University, Ontario, Canada
| | - Pim W Teunissen
- Maastricht University and VU University Medical Center, Amsterdam, The Netherlands
| | - Paul Brand
- Isala Klinieken, Zwolle, The Netherlands
| |
Collapse
|
45
|
Ross S. Are influential teachers born or can they be taught? MEDICAL EDUCATION 2015; 49:1058-1060. [PMID: 26494058 DOI: 10.1111/medu.12830] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/05/2023]
|
46
|
Teunissen PW. Experience, trajectories, and reifications: an emerging framework of practice-based learning in healthcare workplaces. ADVANCES IN HEALTH SCIENCES EDUCATION : THEORY AND PRACTICE 2015; 20:843-56. [PMID: 25269765 DOI: 10.1007/s10459-014-9556-y] [Citation(s) in RCA: 56] [Impact Index Per Article: 6.2] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 07/31/2014] [Accepted: 09/22/2014] [Indexed: 05/15/2023]
Abstract
Learning by working is omnipresent in healthcare education. It enables people to learn how to perform, think, and interact in ways that work for their specific context. In this paper, I review my approach to studying this process. It centers on the question why healthcare professionals do what they do and how their actions and learning are intertwined. The aim of this paper is to illustrate what I have learned from the research I have been involved in, in such a way that it enables other researchers, educators, and clinicians to understand and study practice-based learning in healthcare workplaces. Therefore, I build on a programmatic line of research to present a framework of practice-based learning consisting of three inextricably linked levels of analysis. The first level focuses on how situations lead to personal experiences, the second level looks at strings of experiences that lead to multiple trajectories, and the third level deals with reifications arising from recurrent activities. This framework, and its interrelations and inherent tensions, helps to understand why healthcare workplaces can be both a powerful learning environment and a frustratingly hard place to change.
Collapse
Affiliation(s)
- Pim W Teunissen
- Faculty of Health Medicine and Life Sciences, School of Health Professions Education (SHE), Maastricht University, P.O. Box 616, 6200 MD, Maastricht, The Netherlands.
| |
Collapse
|
47
|
Reddy ST, Zegarek MH, Fromme HB, Ryan MS, Schumann SA, Harris IB. Barriers and Facilitators to Effective Feedback: A Qualitative Analysis of Data From Multispecialty Resident Focus Groups. J Grad Med Educ 2015; 7. [PMID: 26221437 PMCID: PMC4512792 DOI: 10.4300/jgme-d-14-00461.1] [Citation(s) in RCA: 66] [Impact Index Per Article: 7.3] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 11/06/2022] Open
Abstract
BACKGROUND Despite the importance of feedback, the literature suggests that there is inadequate feedback in graduate medical education. OBJECTIVE We explored barriers and facilitators that residents in anesthesiology, emergency medicine, obstetrics and gynecology, and surgery experience with giving and receiving feedback during their clinical training. METHODS Residents from 3 geographically diverse teaching institutions were recruited to participate in focus groups in 2012. Open-ended questions prompted residents to describe their experiences with giving and receiving feedback, and discuss facilitators and barriers. Data were transcribed and analyzed using the constant comparative method associated with a grounded theory approach. RESULTS A total of 19 residents participated in 1 of 3 focus groups. Five major themes related to feedback were identified: teacher factors, learner factors, feedback process, feedback content, and educational context. Unapproachable attendings, time pressures due to clinical work, and discomfort with giving negative feedback were cited as major barriers in the feedback process. Learner engagement in the process was a major facilitator in the feedback process. CONCLUSIONS Residents provided insights for improving the feedback process based on their dual roles as teachers and learners. Time pressures in the learning environment may be mitigated by efforts to improve the quality of teacher-learner relationships. Forms for collecting written feedback should be augmented by faculty development to ensure meaningful use. Efforts to improve residents' comfort with giving feedback and encouraging learners to engage in the feedback process may foster an environment conducive to increasing feedback.
Collapse
|
48
|
Heeneman S, Oudkerk Pool A, Schuwirth LWT, van der Vleuten CPM, Driessen EW. The impact of programmatic assessment on student learning: theory versus practice. MEDICAL EDUCATION 2015; 49:487-98. [PMID: 25924124 DOI: 10.1111/medu.12645] [Citation(s) in RCA: 117] [Impact Index Per Article: 13.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 06/15/2014] [Revised: 09/15/2014] [Accepted: 10/21/2014] [Indexed: 05/07/2023]
Abstract
CONTEXT It is widely acknowledged that assessment can affect student learning. In recent years, attention has been called to 'programmatic assessment', which is intended to optimise both learning functions and decision functions at the programme level of assessment, rather than according to individual methods of assessment. Although the concept is attractive, little research into its intended effects on students and their learning has been conducted. OBJECTIVES This study investigated the elements of programmatic assessment that students perceived as supporting or inhibiting learning, and the factors that influenced the active construction of their learning. METHODS The study was conducted in a graduate-entry medical school that implemented programmatic assessment. Thus, all assessment information, feedback and reflective activities were combined into a comprehensive, holistic programme of assessment. We used a qualitative approach and interviewed students (n = 17) in the pre-clinical phase of the programme about their perceptions of programmatic assessment and learning approaches. Data were scrutinised using theory-based thematic analysis. RESULTS Elements from the comprehensive programme of assessment, such as feedback, portfolios, assessments and assignments, were found to have both supporting and inhibiting effects on learning. These supporting and inhibiting elements influenced students' construction of learning. Findings showed that: (i) students perceived formative assessment as summative; (ii) programmatic assessment was an important trigger for learning, and (iii) the portfolio's reflective activities were appreciated for their generation of knowledge, the lessons drawn from feedback, and the opportunities for follow-up. Some students, however, were less appreciative of reflective activities. For these students, the elements perceived as inhibiting seemed to dominate the learning response. CONCLUSIONS The active participation of learners in their own learning is possible when learning is supported by programmatic assessment. Certain features of the comprehensive programme of assessment were found to influence student learning, and this influence can either support or inhibit students' learning responses.
Collapse
Affiliation(s)
- Sylvia Heeneman
- Department of Pathology, Faculty of Health, Medicine and Life Sciences, Maastricht University, Maastricht, The Netherlands; Faculty of Health, Medicine and Life Sciences, School of Health Professions Education, Maastricht University, Maastricht, The Netherlands
| | | | | | | | | |
Collapse
|
49
|
O'Connell J, Gardner G, Coyer F. Beyond competencies: using a capability framework in developing practice standards for advanced practice nursing. J Adv Nurs 2014; 70:2728-35. [PMID: 25109608 DOI: 10.1111/jan.12475] [Citation(s) in RCA: 76] [Impact Index Per Article: 7.6] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 06/07/2014] [Indexed: 11/25/2022]
Abstract
AIM This paper presents a discussion on the application of a capability framework for advanced practice nursing standards/competencies. BACKGROUND There is acceptance that competencies are useful and necessary for definition and education of practice-based professions. Competencies have been described as appropriate for practice in stable environments with familiar problems. Increasingly competencies are being designed for use in the health sector for advanced practice such as the nurse practitioner role. Nurse practitioners work in environments and roles that are dynamic and unpredictable necessitating attributes and skills to practice at advanced and extended levels in both familiar and unfamiliar clinical situations. Capability has been described as the combination of skills, knowledge, values and self-esteem which enables individuals to manage change, be flexible and move beyond competency. DESIGN A discussion paper exploring 'capability' as a framework for advanced nursing practice standards. DATA SOURCES Data were sourced from electronic databases as described in the background section. IMPLICATIONS FOR NURSING As advanced practice nursing becomes more established and formalized, novel ways of teaching and assessing the practice of experienced clinicians beyond competency are imperative for the changing context of health services. CONCLUSION Leading researchers into capability in health care state that traditional education and training in health disciplines concentrates mainly on developing competence. To ensure that healthcare delivery keeps pace with increasing demand and a continuously changing context there is a need to embrace capability as a framework for advanced practice and education.
Collapse
Affiliation(s)
- Jane O'Connell
- School of Nursing, Queensland University of Technology, Kelvin Grove, Queensland, Australia
| | | | | |
Collapse
|
50
|
Seltz LB, Montgomery A, Lane JL, Soep J, Hanson JL. Medical students' experiences working with frequently rotating pediatric inpatient attending physicians. Hosp Pediatr 2014; 4:239-246. [PMID: 24986994 DOI: 10.1542/hpeds.2014-0016] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 06/03/2023]
Abstract
OBJECTIVE To explore medical students' experiences working with frequently rotating pediatric inpatient attending physicians. METHODS We performed a qualitative study using focus groups and individual interviews of medical students who rotated on the general pediatric inpatient service at Children's Hospital Colorado. The majority of inpatient pediatric attending physicians worked 1-week blocks. We used a semistructured interview guide and analyzed data using the constant comparative method. In accordance with the grounded theory method, codes were developed using an iterative approach, and major themes were identified. Analysis indicated theoretical saturation was achieved. We created a theory that arose from analysis of the data. RESULTS Twenty-seven medical students participated. Data analysis yielded 6 themes: learning climate, continuity, student resilience, opportunity to progress, growth into a physician, and evaluation. In the learning climate, the emotional environment was often stressful, although students valued exposure to different patient care and teaching styles. Senior resident continuity promoted student function; lack of continuity with attending physicians inhibited relationship development. Students were resilient in adjusting to changing faculty with different expectations. In the context of frequently rotating faculty, students had difficulty showing improvement to a single attending physician after feedback, which limited students' opportunities to progress. Students perceived summative evaluation as less meaningful in the absence of having a relationship with their attending physicians. CONCLUSIONS Medical students valued exposure to different patient care and teaching styles. However, frequently changing attending physicians caused students stress and limited students' perceived ability to achieve and show professional growth.
Collapse
Affiliation(s)
- L Barry Seltz
- Children's Hospital Colorado and University of Colorado School of Medicine
| | - Alison Montgomery
- Children's Hospital Colorado and University of Colorado School of Medicine
| | - J Lindsey Lane
- Children's Hospital Colorado and University of Colorado School of Medicine
| | - Jennifer Soep
- Children's Hospital Colorado and University of Colorado School of Medicine
| | - Janice L Hanson
- Children's Hospital Colorado and University of Colorado School of Medicine
| |
Collapse
|