1
|
Wollney EN, Vasquez TS, Stalvey C, Close J, Markham MJ, Meyer LE, Cooper LA, Bylund CL. Are evaluations in simulated medical encounters reliable among rater types? A comparison between standardized patient and outside observer ratings of OSCEs. PEC INNOVATION 2023; 2:100125. [PMID: 37214504 PMCID: PMC10194306 DOI: 10.1016/j.pecinn.2023.100125] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Received: 04/11/2022] [Revised: 12/20/2022] [Accepted: 01/20/2023] [Indexed: 05/24/2023]
Abstract
Objective By analyzing Objective Structured Clinical Examination (OSCE) evaluations of first-year interns' communication with standardized patients (SP), our study aimed to examine the differences between ratings of SPs and a set of outside observers with training in healthcare communication. Methods Immediately following completion of OSCEs, SPs evaluated interns' communication skills using 30 items. Later, two observers independently coded video recordings using the same items. We conducted two-tailed t-tests to examine differences between SP and observers' ratings. Results Rater scores differed significantly on 21 items (p < .05), with 20 of the 21 differences due to higher SP in-person evaluation scores. Items most divergent between SPs and observers included items related to empathic communication and nonverbal communication. Conclusion Differences between SP and observer ratings should be further investigated to determine if additional rater training is needed or if a revised evaluation measure is needed. Educators may benefit from adjusting evaluation criteria to decrease the number of items raters must complete and may do so by encompassing more global questions regarding various criteria. Furthermore, evaluation measures may be strengthened by undergoing reliability and validity testing. Innovation This study highlights the strengths and limitations to rater types (observers or SPs), as well as evaluation methods (recorded or in-person).
Collapse
Affiliation(s)
- Easton N. Wollney
- Dept. of Health Outcomes and Biomedical Informatics, College of Medicine, University of Florida, Gainesville, FL, USA
| | - Taylor S. Vasquez
- College of Journalism and Communications, University of Florida, Gainesville, FL, USA
| | - Carolyn Stalvey
- Dept. of Internal Medicine, College of Medicine, University of Florida, Gainesville, FL USA
| | - Julia Close
- Dept. of Hematology and Oncology, College of Medicine, University of Florida, Gainesville, FL, USA
| | - Merry Jennifer Markham
- Dept. of Hematology and Oncology, College of Medicine, University of Florida, Gainesville, FL, USA
| | - Lynne E. Meyer
- Graduate Medical Education, College of Medicine, University of Florida, Gainesville, FL, USA
| | - Lou Ann Cooper
- Dept. of Medical Education, College of Medicine, University of Florida, Gainesville, FL, USA
| | - Carma L. Bylund
- Dept. of Health Outcomes and Biomedical Informatics, College of Medicine, University of Florida, Gainesville, FL, USA
| |
Collapse
|
2
|
Huang K, Mak D, Hafferty FW, Eva KW. The Advice Given During Near-Peer Interactions Before and After Curriculum Change. TEACHING AND LEARNING IN MEDICINE 2022; 34:351-359. [PMID: 34524067 DOI: 10.1080/10401334.2021.1957685] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 01/06/2021] [Accepted: 07/02/2021] [Indexed: 06/13/2023]
Abstract
PhenomenonNear-peer interactions (NPIs) provide formal and informal mentorship that can allow medical students to share strategies for successful training. Such capacity to convey valuable advice, however, may depend on the similitude of experiences. Given that many factors can disrupt homogeneity, including curriculum renewal, we must better understand whether junior trainees feel disadvantaged when they do not have more senior peers with similar experiences. This study was, therefore, conducted to examine the focus of, and engagement with, advice available through NPIs during curriculum renewal. Approach: We used a generic exploratory qualitative research approach. Twenty MD undergraduate students, seven from the Class of 2019 (the first cohort post-curriculum change), and thirteen from the Class of 2020 (the first cohort with access to more senior students in the new curriculum), participated in semi-structured interviews. Anonymized transcriptions were analyzed with open, axial, and selective coding to generate themes until saturation was attained. Findings: Participants from the Class of 2019 reported having particularly few reasons to seek advice; because curriculum renewal disrupted their near peers' capacity to provide critical insights, students exerted little effort to learn from them. That said, this vacuum was not generally cause for concern. Deeper probing illustrated why: advice given during NPIs in both classes more commonly focused on nonacademic (e.g., work-life balance issues) than academic advice; academic advice, when sought or offered, tended not to be aimed at improving understanding of curriculum dependent content; and, while students in both classes welcomed advice, both were wary of accepting it at face value, precluding a sense of dependence on senior peers. Insights: Students' skepticism about the overall utility of academic advice raises a number of important issues for medical education and training. Positively, it shielded students from feeling loss when advice from similarly trained students was not available, reducing concerns about disadvantage that could arise during periods of curriculum revision. On the other hand, knowing that what students perceive and what educators claim to be important aspects of training can be at odds and knowing that self-assessment is flawed makes it surprising and unsettling, respectively, that participants so readily treated the lessons learned by those who came before them as irrelevant.
Collapse
Affiliation(s)
- Kelly Huang
- Department of Emergency Medicine, University of British Columbia, Vancouver, British Columbia, Canada
| | - David Mak
- Radiation Oncology, University of Toronto, Toronto, Ontario, Canada
| | - Frederic W Hafferty
- Division of General Internal Medicine and Program in Professionalism and Values, Mayo Clinic, Rochester, Minnesota, USA
| | - Kevin W Eva
- Education Research and Scholarship, University of British Columbia, Vancouver, British Columbia, Canada
| |
Collapse
|
3
|
Conrad H, Simpson R, Blackwell TH, Wright WS. Emergency Medical Technician Training in Medical School on Preparation for Required National Board Exams and Clerkship Rotations: Results from a Student Survey. ADVANCES IN MEDICAL EDUCATION AND PRACTICE 2022; 13:709-716. [PMID: 35859777 PMCID: PMC9291422 DOI: 10.2147/amep.s366809] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 03/24/2022] [Accepted: 06/29/2022] [Indexed: 06/15/2023]
Abstract
PURPOSE The University of South Carolina School of Medicine Greenville has incorporated Emergency Medical Technician (EMT) training into the first semester curriculum with students becoming state-certified EMTs and completing one ambulance shift per month throughout their pre-clerkship years. Although there have been programs that have reported EMT experiences in the pre-clinical years of medical education, student perceptions of how the EMT experiences help prepare them for board exams and clerkships is limited. Therefore, the aim of this study was to measure student perceptions regarding the impact of an EMT course and training in the pre-clerkship curriculum in medical school on helping prepare them for national board exams (ie USMLE® Step 1, 2 Clinical Knowledge (CK), 2 Clinical Skills (CS)) and clerkship rotations. METHODS Second-, third-, and fourth-year medical students at the University of South Carolina School of Medicine Greenville completed an anonymous voluntary survey with response rates of 66.3%, 55.2%, and 56.9%, respectively. The study was reviewed and exempted by the University of South Carolina Institutional Review Board. RESULTS Seventeen percent, 14%, and 41% of students agreed/strongly agreed an EMT course helped prepare them for the USMLE Step 1, Step 2 CK, and Step 2 CS exam, respectively. Sixty-four percent of students agreed/strongly agreed that an EMT course and experience helped prepare them for clerkship rotations. CONCLUSION The findings in this study support EMT training and experience as an EMT as one method to help prepare students for clerkship rotations.
Collapse
Affiliation(s)
- Hope Conrad
- School of Medicine, University of South Carolina School of Medicine Greenville, Greenville, SC, USA
| | - Raychel Simpson
- School of Medicine, University of South Carolina School of Medicine Greenville, Greenville, SC, USA
| | | | - William S Wright
- Department of Biomedical Sciences; University of South Carolina School of Medicine Greenville, Greenville, SC, USA
| |
Collapse
|
4
|
Tapasak B, McCall M, Cheung E, Peppler R. Developing Medical Student Competencies, Clinical Skills, and Self-Efficacy With an Emergency Medical Responder Certification Course. Cureus 2022; 14:e26678. [PMID: 35949736 PMCID: PMC9358984 DOI: 10.7759/cureus.26678] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/21/2022] [Accepted: 07/09/2022] [Indexed: 12/03/2022] Open
Abstract
Introduction Previous studies have claimed gap year clinical experiences before medical school matriculation increase student self-efficacy and clinical confidence. At the University of Central Florida College of Medicine, 41 first-year medical students participated in a new certification course to become emergency medical responders before beginning their coursework. This study describes the results of a follow-up study that aims to investigate the impact the course had on student competency, self-efficacy, and clinical efficacy and if the course would prepare students similarly to previous clinical experience. Methods First-year medical students completed a 30-question survey consisting of a Likert scale and free-response questions. Questions were based on the Accreditation Council for Graduate Medical Education core competencies: medical knowledge, practice-based learning and improvement, systems-based practice, patient care, professionalism, and interpersonal and communication skills. Questions on the perceived benefit of the emergency medical responder course and previous clinical experiences were also included. Responses were separated based on participation in the emergency medical responder course and prior clinical experience. Two-tail Welch’s t-tests were performed on the data to determine significance. Results Of 98 responses: 20.4% (20/98) of participants of the emergency medical responder course had previous clinical experience, 21.5% (21/98) of participants of the course had no clinical experience, 26.5% (26/98) did not participate in the course but had previous clinical experience, and 31.6% (31/98) did not participate in the course nor had previous clinical experience. Students with previous clinical experience reported the emergency medical responder course improved both their patient care skills and performance in courses that emphasized patient interviewing and physical exams. Students with clinical experience had significantly higher medical knowledge (p < 0.1) and professionalism (p < 0.1) Likert scores. Eighty-seven percent of students agreed the course had a positive impact on their patient care skills. Conclusion Larger sample size is needed to make stronger conclusions; however, the responses show the emergency medical responder course had a positive subjective impact on students with previous medical experience. Previous clinical experience leads to the most positive subjective reporting of competencies such as medical knowledge and professionalism. Early clinical exposure, such as an emergency medical responder certification course, may improve self-efficacy and patient care skills for medical students with no previous clinical experience.
Collapse
|
5
|
Franco RS, Franco CAGDS, Severo M, Ferreira MA, Karnieli-Miller O. Reflective writing in the teaching of communication skills for medical students-A systematic review. PATIENT EDUCATION AND COUNSELING 2022; 105:1842-1851. [PMID: 35063310 DOI: 10.1016/j.pec.2022.01.003] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 09/16/2021] [Revised: 12/18/2021] [Accepted: 01/08/2022] [Indexed: 06/14/2023]
Abstract
OBJECTIVE To analyse the use, assessment, and measured outcomes of reflective writing (RW) in teaching communication to medical students. METHODS Systematic search of seven electronic databases, focused on using RW in teaching communication skills. Three reviewers selected and prepared the synthesis of the studies. The synthesis was based on thematic analysis using Braun and Clarke's approach. RESULTS We identified 1325 studies, reviewed 101 full-text articles, and included 12 articles in the analysis. The four themes identified showed that RW is not a stand-alone practice. RW is blended with other teaching strategies. Through RW, students identified structural, emotional, and relational aspects and challenges of communication. Only a few studies found a positive correlation between reflective ability and communication skills CONCLUSION: RW can be integrated with various teaching methods, at all stages of learning, to stimulate discussion of interpersonal and intrapersonal topics. Through RW, students explore theirs and their patient's emotions, values, behaviours, and needs identifying challenges and practices relevant to communication. PRACTICE IMPLICATIONS RW can address different structural, relational, and emotional issues that are relevant to communication learning. Further educational development and high-quality empirical research on the use of RW and unique outcomes are needed to support communication skills learning.
Collapse
Affiliation(s)
- Renato Soleiman Franco
- Medicine School and Life Science School - Post Graduate Program in Bioethics, Pontifical Catholic University of Paraná, Curitiba, Brazil.
| | | | - Milton Severo
- Institute of Biomedical Sciences Abel Salazar, University of Porto - School of Health and Life Sciences, Porto, Portugal.
| | - Maria Amélia Ferreira
- Public Health and Forensic Sciences, and Medical Education Department, Faculty of Medicine, University of Porto, Portugal.
| | - Orit Karnieli-Miller
- Sackler Faculty of Medicine, Department of Medical Education, Tel Aviv University, Tel Aviv, Israel.
| |
Collapse
|
6
|
Liu CI, Tang KP, Wang YC, Chiu CH. Impacts of early clinical exposure on undergraduate student professionalism-a qualitative study. BMC MEDICAL EDUCATION 2022; 22:435. [PMID: 35668444 PMCID: PMC9172165 DOI: 10.1186/s12909-022-03505-5] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 01/24/2022] [Accepted: 05/27/2022] [Indexed: 06/15/2023]
Abstract
INTRODUCTION Early clinical exposure (ECE), or authentic human contact in a social or clinical context during preclinical training, has been adopted by many medical schools. This study aims to investigate how medical students' sense of professionalism changed after ECE intervention, with the aim of informing curriculum design to enhance student awareness of the importance of medical professionalism. METHOD Focus groups of ECE students were held to collect data for the study. All participants read interview guidelines before starting. During the focus groups, the students discussed their medical obligations as perceived throughout the course, which offered a choice between four different ECE tracks. They were then asked to report their understanding of the situations they encountered during the course and reflect on their implications. RESULTS Six focus groups of 22 students in total from a medical school in northern Taiwan were held shortly after the students completed an ECE course in September 2019. From their responses, 10 categories relating to medical professionalism were deduced categorized under 5 major dimensions. An additional 8 sub-dimensions on attitudes and 2 sub-dimensions on personal well-being were also identified as new categories separate from but related to medical professionalism. After the ECE intervention, about 59% of participants redefined their understanding of medical professionalism. CONCLUSION ECE and intensive interaction with key stakeholders, including patients and their families, help students in the early stages of medical education form and cultivate a sense of medical professionalism. However, the relationship between participants' personalities, motivations, and clinical activities requires further investigation.
Collapse
Affiliation(s)
- Chun-i Liu
- Wan Fang Hospital, Taipei Medical University, Taipei, Taiwan
| | - Kung-pei Tang
- Department of Early Childhood and Family Education, College of Education, National Taipei University of Education, Taipei, Taiwan
| | - Yun-chu Wang
- Shuang Ho Hospital, Taipei Medical University, New Taipei City, Taiwan
| | - Chiung-hsuan Chiu
- School of Health Care Administration, Taipei Medical University, Taipei, Taiwan
| |
Collapse
|
7
|
Skelly KS, Weerasinghe S, Daly JM, Rosenbaum ME. Impact of Medical Scribe Experiences on Subsequent Medical Student Learning. MEDICAL SCIENCE EDUCATOR 2021; 31:1149-1156. [PMID: 34457958 PMCID: PMC8368573 DOI: 10.1007/s40670-021-01291-1] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Accepted: 04/13/2021] [Indexed: 06/13/2023]
Abstract
INTRODUCTION Many US pre-health professional students prepare for applying to medical training by transcribing clinician-patient encounters. Scant literature of the effects of scribing experiences before starting medical training exists. We conducted a study to investigate student perspectives about medical scribing's educational impact on clinical skill development. METHODS Using a mixed-methods approach, medical and physician assistant students were surveyed and interviewed about clinical experiences and confidence before entering medical training, and the impact of scribing on clinical skills learning. Thematic analysis revealed salient themes in participants' perspectives on the scribing experience. RESULTS A total of 214 (33%) of the 658 students completed the survey; 66 (31%) had scribing experience. Scribes were more confident (p ≤ 0.001) in clinical note writing and history taking than non-scribes. Thematic analysis revealed perceptions that scribing impacted clinical note writing, medical knowledge, communication, and healthcare environment functioning. DISCUSSION Findings suggest that the scribing experience affects confidence and perceived skills, particularly clinical note writing. Future research could explore the impact on clinical skill development through comparative performance-based assessment. CONCLUSION Measuring the impact of scribing before entry to medical/physician assistant school on clinical skill development could provide directives for best approaches to clinical skills education in health professionals training.
Collapse
Affiliation(s)
- Kelly S. Skelly
- Department of Family Medicine, University of Iowa, 01290D PFP, 200 Hawkins Drive, Iowa, IA 52242 USA
| | - Sanjeeva Weerasinghe
- Department of Family Medicine, University of Iowa, 01290D PFP, 200 Hawkins Drive, Iowa, IA 52242 USA
| | - Jeanette M. Daly
- Department of Family Medicine, University of Iowa, 01290D PFP, 200 Hawkins Drive, Iowa, IA 52242 USA
| | - Marcy E. Rosenbaum
- Department of Family Medicine, University of Iowa, 01290D PFP, 200 Hawkins Drive, Iowa, IA 52242 USA
| |
Collapse
|