1
|
Simons MC, Hunt JA, Anderson SL. Trained to cut? A literature review of veterinary surgical resident training. Vet Surg 2024; 53:791-799. [PMID: 38816998 DOI: 10.1111/vsu.14110] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/07/2023] [Revised: 04/04/2024] [Accepted: 04/30/2024] [Indexed: 06/01/2024]
Abstract
A surgical residency trains veterinary graduates to a higher level of expertise in surgical procedures than is possible during veterinary school and prepares a resident to pursue board certification in surgery. The education of veterinary surgical residents has changed minimally since its inception in the twentieth century, and there are insufficient studies to determine if residency programs are producing surgeons with competence in each of the necessary procedural categories. The aims of this review were to report the current theory and methods used to provide surgical education to residents, to discuss the training most likely to create a competent, board-certified surgeon and to review assessment methods used during training. Several literature searches using broad terms such as "veterinary surgery residency," "veterinary surgery resident," and "veterinary surgical training" were performed using PubMed, CAB abstracts, and Google Scholar. Literature pertinent to theory, methods, training, and assessment of veterinary surgical residents was included. The reviewed literature demonstrated the need for research-based learning curves for specific procedures. Simulation training is known to facilitate deliberate practice and should be leveraged where possible to reach competency. The creation of validated assessment methods should be pursued as it enables assessment of competency instead of inferring its development from case logs. Understanding and supporting learner cognition and providing sufficient feedback remain important issues in the field. Surgical educators are urged to continue to search for innovative and evidence-based ways to train competent surgical residents.
Collapse
Affiliation(s)
- Micha C Simons
- Department of Small Animal Clinical Sciences, Virginia Maryland Regional College of Veterinary Medicine, Blacksburg, Virginia, USA
| | - Julie A Hunt
- Department of Clinical Sciences, Lincoln Memorial University Richard A. Gillespie College of Veterinary Medicine, Harrogate, Tennessee, USA
| | - Stacy L Anderson
- Department of Clinical Sciences, Lincoln Memorial University Richard A. Gillespie College of Veterinary Medicine, Harrogate, Tennessee, USA
| |
Collapse
|
2
|
Dejescu CA, Bel LV, Melega I, Muresan SMC, Oana LI. Approaches to Laparoscopic Training in Veterinary Medicine: A Review of Personalized Simulators. Animals (Basel) 2023; 13:3781. [PMID: 38136818 PMCID: PMC10740942 DOI: 10.3390/ani13243781] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/31/2023] [Revised: 11/26/2023] [Accepted: 12/05/2023] [Indexed: 12/24/2023] Open
Abstract
Veterinary minimally invasive surgery (MIS) has experienced notable growth in recent years, yet the availability of specialized training tools remains limited and not readily accessible to practitioners worldwide. While borrowing simulators from human medicine practices suffices for acquiring fundamental laparoscopic skills, it proves inadequate when addressing procedure-specific nuances. Veterinary professionals are now taking steps to create simulators tailored to their patients, although the validation process can be time-consuming. Consequently, the availability of advanced laparoscopic simulators for veterinary training remains scarce. The present study aims to highlight custom-made simulators. A comprehensive search across five databases was conducted to uncover the simulators documented from 2010 to 2022. A total of five simulators emerged from this search, with four grounded in a canine model and only one in an equine model. These models underwent validation and were found to be effective in training surgeons for their designated tasks. The findings underscore a limited array of simulators, predominantly catering to two species (horses and dogs). Considering these findings, it is evident that further research is imperative to create laparoscopic simulators capable of facilitating advanced veterinary training. This would enable the continued evolution of surgical techniques across diverse species, including ruminants, small mammals, and non-mammalian animals.
Collapse
Affiliation(s)
| | - Lucia V. Bel
- Department of Surgery, Anesthesiology and Intensive Care, Faculty of Veterinary Medicine Cluj-Napoca, University of Agricultural Sciences and Veterinary Medicine Cluj-Napoca, 400372 Cluj-Napoca, Romania; (C.A.D.); (I.M.); (S.M.C.M.); (L.I.O.)
| | | | | | | |
Collapse
|
3
|
Elabd K, Abdul-Kadir H, Alkhenizan A, Alkhalifa MK. A Comparison of the Checklist Scoring Systems, Global Rating Systems, and Borderline Regression Method for an Objective Structured Clinical Examination for a Small Cohort in a Saudi Medical School. Cureus 2023; 15:e39968. [PMID: 37415995 PMCID: PMC10320738 DOI: 10.7759/cureus.39968] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 06/05/2023] [Indexed: 07/08/2023] Open
Abstract
BACKGROUND This study aims to compare the effectiveness of using the checklist and global rating scores to evaluate the clinical competency of medical students in Objective Structured Clinical Examinations (OSCEs). Additionally, the study assesses the appropriateness of using the borderline regression method to set standards for small-scale OSCE exams and determines if the estimated passing marks differ significantly from the university's prefixed passing score of 70%. The study also examines whether the university should utilize the borderline regression method to determine passing scores for each OSCE exam instead of a set passing score. METHODS The study analyzed medical students' grades in 11 OSCE exams in the 2022-2023 academic year at Alfaisal University, Riyadh, Saudi Arabia. Students received family medicine clerkship rotations, and after each rotation, they took an OSCE exam consisting of three stations that family medicine consultants graded. The exam included a checklist of 30 tasks and a five-level global rank scale. The study collected all the checklist marks and global rank grades and analyzed them using IBM® Statistical Package for Social Sciences (SPSS® Statistics) software. The statistical tests used were descriptive statistics, the T-test, chi-square tests, Fisher's exact test, and Pearson correlation. RESULTS The study showed that students were more likely to pass when using the global rating system than the checklist scoring system. Additionally, students had a significantly lower passing rate when using the higher cut-off passing score estimated using the borderline regression method compared to the pre-set passing score of 70% established by the university (with a p-value of 0.00). CONCLUSION Each scoring system has advantages and disadvantages, but they complement each other. Combining scoring systems can produce a more comprehensive and precise evaluation of a candidate's performance. The study also emphasizes the importance of carefully selecting and validating cut-off points in OSCE exams to ensure fairness and consistency in assessment.
Collapse
Affiliation(s)
- Kossay Elabd
- Family Medicine, King Faisal Specialist Hospital and Research Centre, Riyadh, SAU
| | - Husam Abdul-Kadir
- Family Medicine and Polyclinics, King Faisal Specialist Hospital and Research Centre, Riyadh, SAU
| | - Abdullah Alkhenizan
- Family Medicine and Polyclinics, King Faisal Specialist Hospital and Research Centre, Riyadh, SAU
| | - Mohammed K Alkhalifa
- Family Medicine and Polyclinics, King Faisal Specialist Hospital and Research Centre, Riyadh, SAU
| |
Collapse
|
4
|
Baillie S, Dilly M, Ciappesoni JL, Read E. The Rapid and International Expansion of Veterinary Clinical Skills Laboratories: A Survey to Establish Recent Developments. JOURNAL OF VETERINARY MEDICAL EDUCATION 2023:e20220113. [PMID: 36795498 DOI: 10.3138/jvme-2022-0113] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/18/2023]
Abstract
Veterinary clinical skills laboratories are used for teaching a wide range of practical, clinical, and surgical skills on models and simulators. A survey conducted in 2015 identified the role of such facilities in veterinary education in North America and Europe. The current study aimed to capture recent changes using a similar survey with three sections to collect data about the structure of the facility, its uses in teaching and assessment, and the staffing. The survey consisted of multiple choice and free text questions, was administered online using Qualtrics and was disseminated in 2021 via clinical skills networks and Associate Deans. Responses were received from 91 veterinary colleges in 34 countries; 68 had an existing clinical skills laboratory and 23 were planning to open one within 1-2 years. Collated information from the quantitative data described the facility, teaching, assessment and staffing. Major themes emerged from the qualitative data relating to aspects of the layout, location, integration in the curriculum, contributions to student learning, and the team managing and supporting the facility. Challenges were associated with budgeting, the ongoing need for expansion and leadership of the program. In summary, veterinary clinical skills laboratories are increasingly common around the world and the contributions to student learning and animal welfare were well recognized. The information about existing and planned laboratories and the tips from those managing the facilities provides valuable guidance for anyone intending to open or expand an existing clinical skills laboratory.
Collapse
Affiliation(s)
- Sarah Baillie
- Bristol Veterinary School, University of Bristol, Langford House, Langford, Bristol BS40 5DU UK
| | - Marc Dilly
- Dina-Weißmann-Allee 6, 68519 Viernheim, Germany
| | - José Luis Ciappesoni
- University of Buenos Aires (UBA), Chorroarín 280 C1427CWO, Autonomous City of Buenos Aires, Argentina
| | - Emma Read
- The Ohio State University College of Veterinary Medicine, Columbus, OH 43210 USA
| |
Collapse
|
5
|
Hall EJ, Baillie S, Hunt JA, Catterall AJ, Wolfe L, Decloedt A, Taylor AJ, Wissing S. Practical Tips for Setting Up and Running OSCEs. JOURNAL OF VETERINARY MEDICAL EDUCATION 2022; 50:e20220003. [PMID: 35617627 DOI: 10.3138/jvme-2022-0003] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/15/2023]
Abstract
Objective structured clinical examinations (OSCEs) are used to assess students' skills on a variety of tasks using live animals, models, cadaver tissue, and simulated clients. OSCEs can be used to provide formative feedback, or they can be summative, impacting progression decisions. OSCEs can also drive student motivation to engage with clinical skill development and mastery in preparation for clinical placements and rotations. This teaching tip discusses top tips for running an OSCE for veterinary and veterinary nursing/technician students as written by an international group of authors experienced with running OSCEs at a diverse set of institutions. These tips include tasks to perform prior to the OSCE, on the day of the examination, and after the examination and provide a comprehensive review of the requirements that OSCEs place on faculty, staff, students, facilities, and animals. These tips are meant to assist those who are already running OSCEs and wish to reassess their existing OSCE processes or intend to increase the number of OSCEs used across the curriculum, and for those who are planning to start using OSCEs at their institution. Incorporating OSCEs into a curriculum involves a significant commitment of resources, and this teaching tip aims to assist those responsible for delivering these assessments with improving their implementation and delivery.
Collapse
|
6
|
Simons MC, Hunt JA, Anderson SL. What's the evidence? A review of current instruction and assessment in veterinary surgical education. Vet Surg 2022; 51:731-743. [PMID: 35543678 DOI: 10.1111/vsu.13819] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/17/2021] [Revised: 01/14/2022] [Accepted: 04/11/2022] [Indexed: 11/30/2022]
Abstract
Veterinary surgical education is improved by studying the relevant literature. The aim of this literature review was to report the theory and methods used to provide surgical education to veterinary students; to discuss the training most likely to create a competent general practitioner; and to review assessment methods for simulated and live surgeries. The literature reviewed demonstrates that new graduates are expected to perform small animal sterilization, dental extraction, onychectomy, abscess treatment, aural hematoma repair, and mass removal with little or no supervision. Students require repetitive practice to reach competence; some practice can take place in a clinical skills laboratory on models or cadavers. When training novices, distributing practice over a longer time improves retention, but months without practice causes skills to decay. Suturing skills may be taught on models at a ratio of 1 instructor per 10 students. Veterinary students require 6-10 repetitions of each small animal sterilization surgery to reach competence; however, learning curves for other surgeries have not been established. Assessment of surgical skills has been undertaken using objective structured clinical examinations (OSCEs) and observed model and live surgeries. Additional validated rubrics are needed in the field. Surgical educators are urged to continue to search for innovative ways to foster deliberate practice, assess skill, and provide feedback for students.
Collapse
Affiliation(s)
- Micha C Simons
- Lincoln Memorial University College of Veterinary Medicine, Harrogate, Tennessee, USA
| | - Julie A Hunt
- Lincoln Memorial University College of Veterinary Medicine, Harrogate, Tennessee, USA
| | - Stacy L Anderson
- Lincoln Memorial University College of Veterinary Medicine, Harrogate, Tennessee, USA
| |
Collapse
|
7
|
Farrell RM, Gilbert GE, Betance L, Huck J, Hunt JA, Dundas J, Pope E. Evaluating validity evidence for 2 instruments developed to assess students' surgical skills in a simulated environment. Vet Surg 2022; 51:788-800. [PMID: 35261056 PMCID: PMC9314123 DOI: 10.1111/vsu.13791] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/14/2021] [Revised: 12/20/2021] [Accepted: 01/31/2022] [Indexed: 11/26/2022]
Abstract
Objective To gather and evaluate validity evidence in the form of content and reliability of scores produced by 2 surgical skills assessment instruments, 1) a checklist, and 2) a modified form of the Objective Structured Assessment of Technical Skills (OSATS) global rating scale (GRS). Study design Prospective randomized blinded study. Sample population Veterinary surgical skills educators (n =10) evaluated content validity. Scores from students in their third preclinical year of veterinary school (n = 16) were used to assess reliability. Methods Content validity was assessed using Lawshe's method to calculate the Content Validity Index (CVI) for the checklist and modified OSATS GRS. The importance and relevance of each item was determined in relation to skills needed to successfully perform supervised surgical procedures. The reliability of scores produced by both instruments was determined using generalizability (G) theory. Results Based on the results of the content validation, 39 of 40 checklist items were included. The 39‐item checklist CVI was 0.81. One of the 6 OSATS GRS items was included. The 1‐item GRS CVI was 0.80. The G‐coefficients for the 40‐item checklist and 6‐item GRS were 0.85 and 0.79, respectively. Conclusion Content validity was very good for the 39‐item checklist and good for the 1‐item OSATS GRS. The reliability of scores from both instruments was acceptable for a moderate stakes examination. Impact These results provide evidence to support the use of the checklist described and a modified 1‐item OSAT GRS in moderate stakes examinations when evaluating preclinical third‐year veterinary students' technical surgical skills on low‐fidelity models.
Collapse
Affiliation(s)
- Robin M Farrell
- School of Veterinary Medicine, University College Dublin, Dublin, Ireland
| | - Gregory E Gilbert
- ΣigmaΣtats Consulting, LLC, Charleston, South Carolina, USA.,Biostatistics and Medical Writing, Real World Evidence Strategy & Analytics, ICON Commercialization & Outcomes Services, North Wales, Pennsylvania, USA
| | - Larry Betance
- School of Veterinary Medicine, Ross University, Basseterre, Saint Kitts and Nevis
| | - Jennifer Huck
- School of Veterinary Medicine, University of Pennsylvania, Philadelphia, Pennsylvania, USA
| | - Julie A Hunt
- College of Veterinary Medicine, Lincoln Memorial University, Harrogate, Tennessee, USA
| | - James Dundas
- Atlantic Veterinary College, Charlottetown, Prince Edward Island, Canada
| | - Eric Pope
- School of Veterinary Medicine, Ross University, Basseterre, Saint Kitts and Nevis
| |
Collapse
|
8
|
Exploring the quality of smoking cessation in community pharmacies: A simulated patient study. Res Social Adm Pharm 2021; 18:2997-3003. [PMID: 34284972 DOI: 10.1016/j.sapharm.2021.07.013] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/13/2021] [Revised: 07/15/2021] [Accepted: 07/15/2021] [Indexed: 11/20/2022]
Abstract
BACKGROUND The prevalence of cigarette smoking continues to be a major public health problem In the United Arab Emirates (UAE); the government has recently implemented policies to reduce smoking prevalence. Innovative strategies to support cessation are needed. Community pharmacies are vital venues to extend the reach and effectiveness of smoking cessation support. OBJECTIVE To evaluate the quality of community pharmacist smoking cessation counseling in the UAE. METHODS A cross-sectional, simulated patient (SP) study was conducted among N = 111 urban community pharmacies selected at random in Sharjah city. Two scenarios were developed to cover different types of cessation needs of treatment-seeking smokers and where pharmacists could have a major role in assisting with smoking cessation pharmacotherapy. The quality of pharmacist counseling was defined in terms of comprehensiveness and communication skills. Two formal assessment tools were used; an analytical checklist to assess the comprehensiveness of pharmacists smoking cessation counseling, and a global assessment form to evaluate communication skills. A descriptive analysis of the data was undertaken. RESULTS A total of 101 pharmacists participated in the study. Pharmacist assessment of smoking cessation-specific information and provision of counseling were minimal. Pharmacists most frequently assessed nicotine dependence and provided generic guidance on the use of nicotine replacement products (NRTs) to manage withdrawal, but they largely did not obtain relevant histories (e.g., medical/medication histories, previous quit attempts, smoking triggers), explain individualized management strategies (e.g., setting quit date, changing environment, reassurance and encouragement), or provide advice about ongoing support. Pharmacists attained low scores in verbal and nonverbal communication and were frequently unempathetic and judgmental towards female SPs. CONCLUSIONS Pharmacist-led smoking cessation programs can expand primary care-based cessation opportunities in the UAE and address the demand for cessation services. Pharmacists will benefit from additional training on the provision of smoking cessation interventions, with an emphasis on patient-centered communication skills.
Collapse
|
9
|
French ED, Griffon DJ, Kass PH, Fahie MA, Gordon-Ross P, Levi O. Evaluation of a laparoscopic abdominal simulator assessment to test readiness for laparoscopic ovariectomy in live dogs. Vet Surg 2021; 50 Suppl 1:O49-O66. [PMID: 33615505 DOI: 10.1111/vsu.13604] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/11/2020] [Revised: 12/09/2020] [Accepted: 01/05/2021] [Indexed: 11/28/2022]
Abstract
OBJECTIVE To predict readiness for laparoscopic ovariectomy of live dogs on the basis of performance on a high-fidelity laparoscopic abdominal simulator and to determine interrater reliability of the assessment. STUDY DESIGN Experimental study. SAMPLE POPULATION Seventeen fourth-year veterinary students. METHODS After a standardized laparoscopic training course, each participant performed a laparoscopic ovariectomy with a simulator. This performance was scored in real time by two evaluators using a rubric. Participants achieving a score of 112 of 160 performed a laparoscopic ovariectomy in a live dog, supervised by an instructor in the room. Two evaluators scored video recordings of each procedure using the rubric. Participants' opinions about the simulator were collected with a survey. RESULTS All participants scored above the threshold (range, 126-151) and successfully completed laparoscopic ovariectomy in a live dog, with an average of 10 of 17 participants requiring verbal guidance and 5 of 17 participants requiring intervention from the instructor. Interrater concordance was excellent for the rubrics used to score performance on the simulator (R = 0.91) and in vivo (R = 0.81). All participants agreed that the simulator should be used to assess trainee readiness prior to surgery in a live dog. CONCLUSION Participants achieving a score of at least 126 of 160 on the simulator were able to perform a laparoscopic ovariectomy in a live dog under supervision. The scoring system for the simulator had excellent interrater concordance. CLINICAL SIGNIFICANCE This simulator and scoring system can be used in laparoscopic training programs to assess readiness for progression to the operative setting.
Collapse
Affiliation(s)
| | - Dominique J Griffon
- College of Veterinary Medicine, Western University of Health Sciences, Pomona, California
| | - Philip H Kass
- School of Veterinary Medicine, University of California, Davis, California
| | - Maria A Fahie
- College of Veterinary Medicine, Western University of Health Sciences, Pomona, California
| | - Paul Gordon-Ross
- College of Veterinary Medicine, Western University of Health Sciences, Pomona, California
| | - Ohad Levi
- College of Veterinary Medicine, Western University of Health Sciences, Pomona, California
| |
Collapse
|
10
|
Royster E, Morin DE, Molgaard L, Wingert D, Fetrow J. Methods Used to Assess Student Performance and Course Outcomes at the National Center of Excellence in Dairy Production Medicine Education for Veterinarians. JOURNAL OF VETERINARY MEDICAL EDUCATION 2020; 47:263-274. [PMID: 32486945 DOI: 10.3138/jvme.1117-162r1] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/11/2023]
Abstract
Between 2012 and 2014, three cohorts of senior veterinary students participated in an 8-week dairy production medicine course created by the National Center of Excellence in Dairy Production Medicine Education for Veterinarians. One goal of this course is to better prepare veterinary students to serve the increasingly complex needs of the dairy industry. In this article, we describe the assessment methods and student performance outcomes of those first three cohorts. A combination of assessment methods was used, including pre- and post-testing; instructor observations and scores on individual and group projects, including a final integrative project; and peer evaluation. Student feedback, collected via anonymous survey, provided insight into students' perceptions about the course and their learning. Performance and feedback suggest that the course was successful in preparing students for careers using skills in dairy production medicine. Pre- and post-testing was conducted for most topic modules in the course. The mean (median) pre- and post-test scores were 47% (50% ) and 83% (88%), respectively. The mean improvement in score was significant (p < .002) for all modules and cohorts. Students indicated a moderate or high degree of confidence in performing dairy production medicine skills after each module. Of students in cohorts 1, 2, and 3, respectively, 55%, 75%, and 82% felt they could provide dairy production medicine services (e.g., records analysis, problem investigation, protocol and standard operating procedure design) either alone or with some mentoring, immediately after graduation. In addition, assessment results and student feedback enabled timely course modifications during these first three cohorts.
Collapse
|
11
|
Cosford K, Briere J, Ambros B, Beazley S, Cartwright C. Effect of Instructional Format on Veterinary Students' Task Performance and Emotional State during a Simulation-Based Canine Endotracheal Intubation Laboratory: Handout versus Video. JOURNAL OF VETERINARY MEDICAL EDUCATION 2020; 47:239-247. [PMID: 31194627 DOI: 10.3138/jvme.0618-077r1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/09/2023]
Abstract
Video- versus handout-based instructions may influence student outcomes during simulation training and competency-based assessments. Forty-five third-year veterinary students voluntarily participated in a simulation module on canine endotracheal intubation. A prospective, randomized, double-blinded study investigated the impact of video (n = 23) versus handout (n = 22) instructions on student confidence, anxiety, and task performance. Students self-scored their confidence and anxiety before and after the simulation. During the simulation laboratory, three raters independently evaluated student performance using a 20-item formal assessment tool with a 5-point global rating scale. No significant between- or within-group differences (p > .05) were found for both confidence and anxiety scores. Video-based instructions were associated with significantly higher (p < .05) total formal assessment scores compared with handout-based instructions. The video group had significantly higher scores than the handout group on 3 of the 20 individual skills (items) assessed: placement of tie to the adaptor-endotracheal tube complex (p < .05), using the anesthetic machine (p < .01), and pop-off valve management (p < .001). Inter-rater reliability as assessed by Cronbach's α (.92), and Kendall's W (.89) was excellent and almost perfect, respectively. A two-faceted crossed-design generalizability analysis yielded G coefficients for both the handout (Ep2 = .68) and the video (Ep2 = .72) groups. Video instructions may be associated with higher performance scores than handout instructions during endotracheal intubation simulation training. Further research into skill retention and learning styles is warranted.
Collapse
Affiliation(s)
- Kevin Cosford
- Department of Small Animal Clinical Sciences, Western College of Veterinary Medicine, University of Saskatchewan
| | - Jennifer Briere
- Department of Psychology at St. Thomas More College, University of Saskatchewan
| | - Barbara Ambros
- Department of Small Animal Clinical Sciences, Western College of Veterinary Medicine, University of Saskatchewan
| | - Shannon Beazley
- Department of Small Animal Clinical Sciences, Western College of Veterinary Medicine, University of Saskatchewan
| | | |
Collapse
|
12
|
Holzhausen Y, Maaz A, März M, Sehy V, Peters H. Exploring the introduction of entrustment rating scales in an existing objective structured clinical examination. BMC MEDICAL EDUCATION 2019; 19:319. [PMID: 31438938 PMCID: PMC6704513 DOI: 10.1186/s12909-019-1736-2] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.4] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 03/21/2019] [Accepted: 07/29/2019] [Indexed: 06/10/2023]
Abstract
BACKGROUND The concept of EPAs is increasingly applied to assess trainees' workplace performance by means of entrustment ratings. OSCEs assess performance in a simulated setting, and it is unclear whether entrustment ratings can be integrated into these exams. This study explores the introduction of an entrustment rating scale into an existing OSCE. METHODS A 6-point entrustment scale was added to the standard ratings in an OSCE administered prior to students' final clerkship year in an undergraduate medical programme. Standard OSCE ratings assess clinical and communication skills. Assessors (n = 54) rated students' performance (n = 227) on a diverse set of clinical tasks and evaluated the addition of entrustment scales to OSCEs. Descriptive and inferential statistics were calculated for analyses. RESULTS Student performance varied across the stations, as reflected in both the standard OSCE ratings and the added entrustment ratings. Students received generally high standard OSCE ratings, whereas entrustment ratings were more widely distributed. All students passed the OSCE, and only a small proportion of students did not reach the expected pass threshold of 60% on the standard ratings in the single stations. The proportion of students who did not reach the expected entrustment level in the respective stations was noticeably higher. Both the clinical and communication skill ratings were related to the entrustment rating in most OSCE stations. A majority of the assessors positively evaluated the addition of entrustment ratings into the OSCE. DISCUSSION The findings provide an empirical basis to broaden our understanding of the potential use of entrustment ratings in existing OSCEs. They provide directions for future, more specific studies. The ratings might be used for formative feedback on students' readiness for workplace practice.
Collapse
Affiliation(s)
- Ylva Holzhausen
- Dieter Scheffner Center for Medical Education and Educational Research, Dean’s Office of Student Affairs, Charité – Universitätsmedizin Berlin, Free University of Berlin and Humboldt University of Berlin, Berlin, Germany
| | - Asja Maaz
- Dieter Scheffner Center for Medical Education and Educational Research, Dean’s Office of Student Affairs, Charité – Universitätsmedizin Berlin, Free University of Berlin and Humboldt University of Berlin, Berlin, Germany
| | - Maren März
- Office of Student Affairs, Dean’s Office of Student Affairs, Charité – Universitätsmedizin Berlin, Free University of Berlin and Humboldt University of Berlin, Berlin, Germany
| | - Victoria Sehy
- Office of Student Affairs, Dean’s Office of Student Affairs, Charité – Universitätsmedizin Berlin, Free University of Berlin and Humboldt University of Berlin, Berlin, Germany
| | - Harm Peters
- Dieter Scheffner Center for Medical Education and Educational Research, Dean’s Office of Student Affairs, Charité – Universitätsmedizin Berlin, Free University of Berlin and Humboldt University of Berlin, Berlin, Germany
| |
Collapse
|
13
|
Inayah AT, Anwer LA, Shareef MA, Nurhussen A, Alkabbani HM, Alzahrani AA, Obad AS, Zafar M, Afsar NA. Objectivity in subjectivity: do students' self and peer assessments correlate with examiners' subjective and objective assessment in clinical skills? A prospective study. BMJ Open 2017; 7:e012289. [PMID: 28487454 PMCID: PMC5623435 DOI: 10.1136/bmjopen-2016-012289] [Citation(s) in RCA: 14] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 04/14/2016] [Revised: 03/15/2017] [Accepted: 03/16/2017] [Indexed: 11/04/2022] Open
Abstract
OBJECTIVES The qualitative subjective assessment has been exercised either by self-reflection (self-assessment (SA)) or by an observer (peer assessment (PA)) and is considered to play an important role in students' development. The objectivity of PA and SA by students as well as those by faculty examiners has remained debated. This matters most when it comes to a high-stakes examination. We explored the degree of objectivity in PA, SA, as well as the global rating by examiners being Examiners' Subjective Assessment (ESA) compared with Objective Structured Clinical Examinations (OSCE). DESIGN Prospective cohort study. SETTING Undergraduate medical students at Alfaisal University, Riyadh. PARTICIPANTS All second-year medical students (n=164) of genders, taking a course to learn clinical history taking and general physical examination. MAIN OUTCOME MEASURES A Likert scale questionnaire was distributed among the participants during selected clinical skills sessions. Each student was evaluated randomly by peers (PA) as well as by himself/herself (SA). Two OSCEs were conducted where students were assessed by an examiner objectively as well as subjectively (ESA) for a global rating of confidence and well-preparedness. OSCE-1 had fewer topics and stations, whereas OSCE-2 was terminal and full scale. RESULTS OSCE-1 (B=0.10) and ESA (B=8.16) predicted OSCE-2 scores. 'No nervousness' in PA (r=0.185, p=0.018) and 'confidence' in SA (r=0.207, p=0.008) correlated with 'confidence' in ESA. In 'well-preparedness', SA correlated with ESA (r=0.234, p=0.003). CONCLUSIONS OSCE-1 and ESA predicted students' performance in the OSCE-2, a high-stakes evaluation, indicating practical 'objectivity' in ESA, whereas SA and PA had minimal predictive role. Certain components of SA and PA correlated with ESA, suggesting partial objectivity given the limited objectiveness of ESA. Such difference in 'qualitative' objectivity probably reflects experience. Thus, subjective assessment can be used with some degree of objectivity for continuous assessment.
Collapse
Affiliation(s)
| | - Lucman A Anwer
- College of Medicine, Alfaisal University, Riyadh, Saudi Arabia
- Mayo Clinic, Rochester, USA
| | - Mohammad Abrar Shareef
- College of Medicine, Alfaisal University, Riyadh, Saudi Arabia
- Mercy St. Vincent Medical Center, Toledo, USA
| | - Akram Nurhussen
- College of Medicine, Alfaisal University, Riyadh, Saudi Arabia
| | | | | | | | - Muhammad Zafar
- College of Medicine, Alfaisal University, Riyadh, Saudi Arabia
| | - Nasir Ali Afsar
- College of Medicine, Alfaisal University, Riyadh, Saudi Arabia
| |
Collapse
|