1
|
Kruppa C, Rudzki M, Baron DJ, Dudda M, Schildhauer TA, Herbstreit S. [Success factors and obstacles in the implementation of competence-oriented teaching in surgery]. CHIRURGIE (HEIDELBERG, GERMANY) 2024:10.1007/s00104-024-02107-9. [PMID: 38829547 DOI: 10.1007/s00104-024-02107-9] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Accepted: 05/03/2024] [Indexed: 06/05/2024]
Abstract
BACKGROUND AND OBJECTIVES For successful competence-oriented teaching at the medical faculties it is important to identify the factors that influence its implementation in order to benefit from the strengths and balance out weaknesses. The present study examined the success factors and obstacles of the implementation of competence-oriented teaching in the surgical discipline from the point of view of students and lecturers. METHODS After implementation of competence-oriented teaching based on the teaching goals of the NKLM, in clinical examination courses (bedside teaching and block internship, BP) at two hospitals, a qualitative content analysis and quantification of the answers were performed using focus group interviews and questionnaires with students (S) and lecturers (D). RESULTS During the summer semester 2022 a total of 31 students and 14 lecturers were interviewed in focus groups and 143 questionnaires (123 S, 20 D) were analyzed. For the students the presence of concrete competences/teaching goals, guidelines for the lesson, transparent goals and ability to demand teaching goals as well as structured lessons and mentoring were the main success factors. Lecturers on the other hand reported the presence of concrete goals, assistance for the lesson preparation and the activity of the students as success factors. The results of the questionnaires showed that the majority (88% S, 75% D) were informed about the teaching goals and considered them to be followed (84%S, 95% D). Obstacles were the factors "time", "mentoring" and "information". Factors that were between negative and positive (indifferent factors) were "uncertainty about competence-orientation" and "uncertainty how to examine the teaching success". DISCUSSION Transparent structure and teaching goals as well as a mentoring system are the success factors for the implementation of competence-oriented lessons and should be used as strengths. Indifferent factors represent weaknesses and need to be addressed by training and instruction. Restricted time and personnel resources are the immanent problems that hamper the implementation and require fulminant structural changes.
Collapse
Affiliation(s)
- C Kruppa
- Chirurgische Universitätsklinik und Poliklinik, Berufsgenossenschaftliches Universitätsklinikum Bergmannsheil, Ruhr-Universität Bochum, Bochum, Deutschland.
| | - M Rudzki
- Chirurgische Universitätsklinik und Poliklinik, Berufsgenossenschaftliches Universitätsklinikum Bergmannsheil, Ruhr-Universität Bochum, Bochum, Deutschland
| | - D J Baron
- Chirurgische Universitätsklinik und Poliklinik, Berufsgenossenschaftliches Universitätsklinikum Bergmannsheil, Ruhr-Universität Bochum, Bochum, Deutschland
| | - M Dudda
- Zentrum für Muskuloskelettale Chirurgie, Universitätsmedizin Essen, Universität Duisburg-Essen, Essen, Deutschland
- Klinik für Orthopädie und Unfallchirurgie, BG-Klinikum Duisburg, Universität Duisburg-Essen, Duisburg, Deutschland
| | - T A Schildhauer
- Chirurgische Universitätsklinik und Poliklinik, Berufsgenossenschaftliches Universitätsklinikum Bergmannsheil, Ruhr-Universität Bochum, Bochum, Deutschland
| | - S Herbstreit
- Zentrum für Muskuloskelettale Chirurgie, Universitätsmedizin Essen, Universität Duisburg-Essen, Essen, Deutschland
- Klinik für Orthopädie und Unfallchirurgie, BG-Klinikum Duisburg, Universität Duisburg-Essen, Duisburg, Deutschland
- Institut für Didaktik in der Medizin, Medizinische Fakultät, Universität Duisburg-Essen, Essen, Deutschland
| |
Collapse
|
2
|
Chang O, Holbrook AM, Lohit S, Deng J, Xu J, Lee M, Cheng A. Comparability of Objective Structured Clinical Examinations (OSCEs) and Written Tests for Assessing Medical School Students' Competencies: A Scoping Review. Eval Health Prof 2023; 46:213-224. [PMID: 36959750 PMCID: PMC10443966 DOI: 10.1177/01632787231165797] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 03/25/2023]
Abstract
Objective Structured Clinical Examinations (OSCEs) and written tests are commonly used to assess health professional students, but it remains unclear whether the additional human resources and expenses required for OSCEs, both in-person and online, are worthwhile for assessing competencies. This scoping review summarized literature identified by searching MEDLINE and EMBASE comparing 1) OSCEs and written tests and 2) in-person and online OSCEs, for assessing health professional trainees' competencies. For Q1, 21 studies satisfied inclusion criteria. The most examined health profession was medical trainees (19, 90.5%), the comparison was most frequently OSCEs versus multiple-choice questions (MCQs) (18, 85.7%), and 18 (87.5%) examined the same competency domain. Most (77.5%) total score correlation coefficients between testing methods were weak (r < 0.40). For Q2, 13 articles were included. In-person and online OSCEs were most used for medical trainees (9, 69.2%), checklists were the most prevalent evaluation scheme (7, 63.6%), and 14/17 overall score comparisons were not statistically significantly different. Generally low correlations exist between MCQ and OSCE scores, providing insufficient evidence as to whether OSCEs provide sufficient value to be worth their additional cost. Online OSCEs may be a viable alternative to in-person OSCEs for certain competencies where technical challenges can be met.
Collapse
Affiliation(s)
- Oswin Chang
- Clinical Pharmacology and Toxicology Research, St Joseph’s Healthcare Hamilton
- Faculty of Health Sciences, McMaster University
| | - Anne M. Holbrook
- Clinical Pharmacology and Toxicology Research, St Joseph’s Healthcare Hamilton
- Division of Clinical Pharmacology and Toxicology, McMaster University
| | - Simran Lohit
- Clinical Pharmacology and Toxicology Research, St Joseph’s Healthcare Hamilton
- Faculty of Health Sciences, McMaster University
| | - Jiawen Deng
- Clinical Pharmacology and Toxicology Research, St Joseph’s Healthcare Hamilton
- Faculty of Health Sciences, McMaster University
| | - Janice Xu
- Clinical Pharmacology and Toxicology Research, St Joseph’s Healthcare Hamilton
- Faculty of Health Sciences, McMaster University
| | - Munil Lee
- Schulich School of Medicine and Dentistry,University of Western Ontario
| | - Alan Cheng
- Clinical Pharmacology and Toxicology Research, St Joseph’s Healthcare Hamilton
- Faculty of Health Sciences, McMaster University
| |
Collapse
|
3
|
Kurz S, Buggenhagen H, Wachter N, Penzkofer L, Dietz SO, König TT, Heinemann MK, Neulen A, Hanke LI, Huber T. [Testing of practical surgical teaching at a distance-Experiences with a hybrid OSCE in surgery]. CHIRURGIE (HEIDELBERG, GERMANY) 2022; 93:976-982. [PMID: 35925137 PMCID: PMC9122243 DOI: 10.1007/s00104-022-01650-7] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Accepted: 04/06/2022] [Indexed: 11/24/2022]
Abstract
BACKGROUND The COVID-19 pandemic has changed medical teaching worldwide. Digital teaching and examinations are successful for cognitive learning objectives, whereas practical skills had to be carried out predominantly in person under strict hygiene standards. AIM This study presents the opportunities and challenges of using a presence objective structured clinical examination (OSCE) at a distance with digital support. METHOD Following surgical practical teaching an OSCE was conducted in presence, where students demonstrated practical skills in one room while the examiners were connected via videoconference from another room. Students were surveyed about the OSCE and sustained learning via a standardized online questionnaire after completion of the surgical teaching. Additionally, examiners were surveyed on their experiences. RESULTS In the online survey 40 students participated (25% of n = 157 students contacted) and 36 completed questionnaires were analyzed. Overall, the implementation of an OSCE even under pandemic conditions was perceived as very positive by the students (92% of students n = 33). In particular, the acquisition of practical skills was rated as very high. For 78% (n = 28) of the students, the acquisition of competencies through the practical examination was particularly sustainable. The vast majority of students and examiners felt safe regarding infection control because of the hygiene concept (92%, n = 33). Overall, 80 students achieved grade 1 (51%), 71 students grade 2 (45.2%) and 6 students grade 3 (3.8%) (grade 1 = very good, grade 6 = very bad). CONCLUSION Practical examinations are essential for checking practical learning objectives and can be implemented at a distance with a well-developed hygiene concept and digital support.
Collapse
Affiliation(s)
- S Kurz
- Rudolf Frey Lernklinik, Universitätsmedizin, Johannes Gutenberg-Universität Mainz, Mainz, Deutschland
| | - H Buggenhagen
- Rudolf Frey Lernklinik, Universitätsmedizin, Johannes Gutenberg-Universität Mainz, Mainz, Deutschland
| | - N Wachter
- Klinik für Allgemein‑, Viszeral- und Transplantationschirurgie, Universitätsmedizin, Johannes Gutenberg-Universität Mainz, Langenbeckstr. 1, 55131, Mainz, Deutschland
| | - L Penzkofer
- Klinik für Allgemein‑, Viszeral- und Transplantationschirurgie, Universitätsmedizin, Johannes Gutenberg-Universität Mainz, Langenbeckstr. 1, 55131, Mainz, Deutschland
| | - S O Dietz
- Zentrum für Orthopädie und Unfallchirurgie, Universitätsmedizin, Johannes Gutenberg-Universität Mainz, Mainz, Deutschland
| | - T T König
- Klinik und Poliklinik für Kinderchirurgie, Universitätsmedizin, Johannes Gutenberg-Universität Mainz, Mainz, Deutschland
| | - M K Heinemann
- Klinik und Poliklinik für Herz- und Gefäßchirurgie, Universitätsmedizin, Johannes Gutenberg-Universität Mainz, Mainz, Deutschland
| | - A Neulen
- Neurochirurgische Klinik und Poliklinik, Universitätsmedizin, Johannes Gutenberg-Universität Mainz, Mainz, Deutschland
| | - L I Hanke
- Klinik für Allgemein‑, Viszeral- und Transplantationschirurgie, Universitätsmedizin, Johannes Gutenberg-Universität Mainz, Langenbeckstr. 1, 55131, Mainz, Deutschland
| | - T Huber
- Klinik für Allgemein‑, Viszeral- und Transplantationschirurgie, Universitätsmedizin, Johannes Gutenberg-Universität Mainz, Langenbeckstr. 1, 55131, Mainz, Deutschland.
| |
Collapse
|
4
|
Cosker E, Favier V, Gallet P, Raphael F, Moussier E, Tyvaert L, Braun M, Feigerlova E. Tutor-Student Partnership in Practice OSCE to Enhance Medical Education. MEDICAL SCIENCE EDUCATOR 2021; 31:1803-1812. [PMID: 34956698 PMCID: PMC8651844 DOI: 10.1007/s40670-021-01421-9] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Accepted: 09/27/2021] [Indexed: 06/14/2023]
Abstract
BACKGROUND Training of examiners is essential to ensure the quality of objective structured clinical examination (OSCE). We aimed to study a perceived effectiveness of tutor-student partnership in a practice OSCE module by novice OSCE tutors and medical students. METHOD We implemented a practice OSCE at a medical faculty in France with novice tutors and third year medical students as partners. Each tutor (n = 44) served as a partner for the group of 5 students in the conception of the scenario and as an evaluator of the tutored station. Students (n = 303) were involved in the conception of a case and the roles of a physician, evaluator and a simulated patient. Data were obtained through self-assessment questionnaires. Descriptive statistics were used to analyze items of the questionnaires. Free-form answers were coded and analyzed thematically. RESULTS A total of 36 tutors (82%) and 185 students (61%) responded to the questionnaires. The intervention was well perceived. Thirty-two percent of the tutors reported some difficulties in the assessment of student performance and were disposed to receive further training. Fifty-five percent of the students considered the participation in the OSCE case development appropriate to their level of knowledge, and 70% perceived it as beneficial allowing them to set their learning goals. CONCLUSION This initiative provides a relevant method beneficial to OSCE tutors, medical students, and the faculty. Tutors learn how to assess student performance according to expected achievement levels. It allows students to be engaged as partners in co-creation of learning and teaching. SUPPLEMENTARY INFORMATION The online version contains supplementary material available at 10.1007/s40670-021-01421-9.
Collapse
Affiliation(s)
- Eve Cosker
- Pôle Hospitalo-Universitaire de psychiatrie d’adultes et d’addictologie du Grand Nancy, Centre Psychothérapique De Nancy, Laxou, F-54520 France
- Université de Strasbourg Unité de Physiopathologie et Médecine Translationnelle, INSERM U1114, Strasbourg, F-67000 France
| | - Valentin Favier
- Centre hospitalier, régionale et universitaire de Nancy, Otorhinolaryngology, Université de Lorraine, Nancy, F-54000 France
| | - Patrice Gallet
- Centre hospitalier, régionale et universitaire de Nancy, Otorhinolaryngology, Université de Lorraine, Nancy, F-54000 France
- Centre universitaire d’enseignement par simulation (CUESiM), Hôpital virtuel de Lorraine, Faculté de médecine, Nancy, F-54000 France
- Faculté de Médecine, Université de Lorraine,, Nancy, F-54000 France
| | - Francis Raphael
- Faculté de Médecine, Université de Lorraine,, Nancy, F-54000 France
| | | | - Louise Tyvaert
- Faculté de Médecine, Université de Lorraine,, Nancy, F-54000 France
- Centre hospitalier régionale et universitaire de Nancy, Department of Neuro, Université de Lorraine, Nancy, F-54000 France
| | - Marc Braun
- Centre universitaire d’enseignement par simulation (CUESiM), Hôpital virtuel de Lorraine, Faculté de médecine, Nancy, F-54000 France
- Faculté de Médecine, Université de Lorraine,, Nancy, F-54000 France
| | - Eva Feigerlova
- Centre universitaire d’enseignement par simulation (CUESiM), Hôpital virtuel de Lorraine, Faculté de médecine, Nancy, F-54000 France
- Faculté de Médecine, Université de Lorraine,, Nancy, F-54000 France
- Centre hospitalier régionale et universitaire de Nancy, Department of endocrinology, Université de Lorraine, Nancy, F-54000 France
- Université de Lorraine, Inserm, UMR S 1116 – DCAC, Nancy, F-54000 France
| |
Collapse
|
5
|
Tan JY, Ma IWY, Hunt JA, Kwong GPS, Farrell R, Bell C, Read EK. Video Recording in Veterinary Medicine OSCEs: Feasibility and Inter-rater Agreement between Live Performance Examiners and Video Recording Reviewing Examiners. JOURNAL OF VETERINARY MEDICAL EDUCATION 2021; 48:485-491. [PMID: 32758091 DOI: 10.3138/jvme-2019-0142] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/11/2023]
Abstract
The Objective Structured Clinical Examination (OSCE) is a valid, reliable assessment of veterinary students' clinical skills that requires significant examiner training and scoring time. This article seeks to investigate the utility of implementing video recording by scoring OSCEs in real-time using live examiners, and afterwards using video examiners from within and outside the learners' home institution. Using checklists, learners (n=33) were assessed by one live examiner and five video examiners on three OSCE stations: suturing, arthrocentesis, and thoracocentesis. When stations were considered collectively, there was no difference between pass/fail outcome between live and video examiners (χ2 = 0.37, p = .55). However, when considered individually, stations (χ2 = 16.64, p < .001) and interaction between station and type of examiner (χ2 = 7.13, p = .03) demonstrated a significant effect on pass/fail outcome. Specifically, learners being assessed on suturing with a video examiner had increased odds of passing the station as compared with their arthrocentesis or thoracocentesis stations. Internal consistency was fair to moderate (0.34-0.45). Inter-rater reliability measures varied but were mostly moderate to strong (0.56-0.82). Video examiners spent longer assessing learners than live raters (mean of 21 min/learner vs. 13 min/learner). Station-specific differences among video examiners may be due to intermittent visibility issues during video capture. Overall, video recording learner performances appears reliable and feasible, although there were time, cost, and technical issues that may limit its routine use.
Collapse
|
6
|
Kurz S, Lohse J, Buggenhagen H, Schmidtmann I, Laufenberg-Feldmann R, Engelhard K. Improving competence and safety in pain medicine: a practical clinical teaching strategy for students combining simulation and bedside teaching. BMC MEDICAL EDUCATION 2021; 21:133. [PMID: 33632210 PMCID: PMC7905916 DOI: 10.1186/s12909-021-02554-6] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 06/19/2020] [Accepted: 02/09/2021] [Indexed: 06/12/2023]
Abstract
BACKGROUND Pain is a devastating sensation and has to be treated immediately. Therefore, we developed a training program to improve the knowledge of medical students in the field of pain medicine. In the present study, the applicability and efficacy of this training program was tested. METHODS Half of the students attended first a training with simulated patients (SP) followed by bedside teaching (Group 1). Group 2 performed the training programs in reverse order. The evaluation based on standardized questionnaires completed by students (self-assessment) and all students took part in two practical examinations after the learning interventions. RESULTS This study included 35 students. The quality of the simulation was evaluated by the students with average grade 1.1 (1 = very good, 6 = very bad). The practical work on the ward with patients was rated with grade 1.4 of 6, the whole course with 1.1. Students of Group A were significantly better in the final examination (grade 1.7 vs. grade 2.2, p < 0.05). To rate the improvement of skills (self-assessment) we used a Likert Scale (1 = very certain, 5 = very uncertain). The following skills were similar in both groups and significantly better after the course: taking responsibility, expert knowledge, empathy, relationship building and communication. CONCLUSIONS Training with simulated patients in combination with small-group teaching at the bedside with real patients achieves a dramatic increase in student competence. Students prefer learning from the simulation before bedside teaching and propose to include simulation into the curricular teaching of pain medicine.
Collapse
Affiliation(s)
- Sandra Kurz
- University Medical Center of the Johannes Gutenberg-University, Langenbeckstraße 1, 55131, Mainz, Germany.
| | - Jana Lohse
- University Medical Center of the Johannes Gutenberg-University, Langenbeckstraße 1, 55131, Mainz, Germany
| | - Holger Buggenhagen
- University Medical Center of the Johannes Gutenberg-University, Langenbeckstraße 1, 55131, Mainz, Germany
| | - Irene Schmidtmann
- University Medical Center of the Johannes Gutenberg-University, Langenbeckstraße 1, 55131, Mainz, Germany
| | - Rita Laufenberg-Feldmann
- University Medical Center of the Johannes Gutenberg-University, Langenbeckstraße 1, 55131, Mainz, Germany
| | - Kristin Engelhard
- University Medical Center of the Johannes Gutenberg-University, Langenbeckstraße 1, 55131, Mainz, Germany
| |
Collapse
|
7
|
Zimmermann P, Kadmon M. Standardized examinees: development of a new tool to evaluate factors influencing OSCE scores and to train examiners. GMS JOURNAL FOR MEDICAL EDUCATION 2020; 37:Doc40. [PMID: 32685668 PMCID: PMC7346289 DOI: 10.3205/zma001333] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Subscribe] [Scholar Register] [Received: 10/15/2019] [Revised: 02/23/2020] [Accepted: 04/27/2020] [Indexed: 05/27/2023]
Abstract
Introduction: The Objective Structured Clinical Examination (OSCE) is an established format for practical clinical assessments at most medical schools and discussion is underway in Germany to make it part of future state medical exams. Examiner behavior that influences assessment results is described. Erroneous assessments of student performance can result, for instance, from systematic leniency, inconsistent grading, halo effects, and even a lack of differentiation between the tasks to be performed over the entire grading scale. The aim of this study was to develop a quality assurance tool that can monitor factors influencing grading in a real OSCE and enable targeted training of examiners. Material, Methods and Students: Twelve students at the Medical Faculty of the University of Heidelberg were each trained to perform a defined task for a particular surgical OSCE station. Definitions were set and operationalized for an excellent and a borderline performance. In a simulated OSCE during the first part of the study, the standardized student performances were assessed and graded by different examiners three times in succession; video recordings were made. Quantitative and qualitative analysis of the videos was also undertaken by the study coordinator. In the second part of the study, the videos were used to investigate the examiners' acceptance of standardized examinees and to analyze potential influences on scoring that stemmed from the examiners' experience. Results: In the first part of the study, the OSCE scores and subsequent video analysis showed that standardization for defined performance levels at different OSCE stations is generally possible. Individual deviations from the prescribed examinee responses were observed and occurred primarily with increased complexity of OSCE station content. In the second part of the study, inexperienced examiners assessed a borderline performance significantly lower than their experienced colleagues (13.50 vs. 15.15, p=0.035). No difference was seen in the evaluation of the excellent examinees. Both groups of examiners graded the item "ocial competence" - despite identical standardization - significantly lower for examinees with borderline performances than for excellent examinees (4.13 vs. 4.80, p<0.001). Conclusion: Standardization of examinees for previously defined performance levels is possible, making a new tool available in future not only for OSCE quality assurance, but also for training examiners. Detailed preparation of the OSCE checklists and intensive training of the examinees are essential. This new tool takes on a special importance if standardized OSCEs are integrated into state medical exams and, as such, become high-stakes assessments.
Collapse
Affiliation(s)
- Petra Zimmermann
- Ludwig-Maximilians-Universität München, Klinikum der Universität, Klinik für Allgemein-, Viszeral- und Transplantationschirurgie, München, Germany
| | - Martina Kadmon
- Universität Augsburg, Medizinische Fakultät, Gründungsdekanat, Augsburg, Germany
| |
Collapse
|
8
|
Prediger S, Schick K, Fincke F, Fürstenberg S, Oubaid V, Kadmon M, Berberat PO, Harendza S. Validation of a competence-based assessment of medical students' performance in the physician's role. BMC MEDICAL EDUCATION 2020; 20:6. [PMID: 31910843 PMCID: PMC6947905 DOI: 10.1186/s12909-019-1919-x] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 09/23/2019] [Accepted: 12/22/2019] [Indexed: 05/04/2023]
Abstract
BACKGROUND Assessing competence of advanced undergraduate medical students based on performance in the clinical context is the ultimate, yet challenging goal for medical educators to provide constructive alignment between undergraduate medical training and professional work of physicians. Therefore, we designed and validated a performance-based 360-degree assessment for competences of advanced undergraduate medical students. METHODS This study was conducted in three steps: 1) Ten facets of competence considered to be most important for beginning residents were determined by a ranking study with 102 internists and 100 surgeons. 2) Based on these facets of competence we developed a 360-degree assessment simulating a first day of residency. Advanced undergraduate medical students (year 5 and 6) participated in the physician's role. Additionally knowledge was assessed by a multiple-choice test. The assessment was performed twice (t1 and t2) and included three phases: a consultation hour, a patient management phase, and a patient handover. Sixty-seven (t1) and eighty-nine (t2) undergraduate medical students participated. 3) The participants completed the Group Assessment of Performance (GAP)-test for flight school applicants to assess medical students' facets of competence in a non-medical context for validation purposes. We aimed to provide a validity argument for our newly designed assessment based on Messick's six aspects of validation: (1) content validity, (2) substantive/cognitive validity, (3) structural validity, (4) generalizability, (5) external validity, and (6) consequential validity. RESULTS Our assessment proved to be well operationalised to enable undergraduate medical students to show their competences in performance on the higher levels of Bloom's taxonomy. Its generalisability was underscored by its authenticity in respect of workplace reality and its underlying facets of competence relevant for beginning residents. The moderate concordance with facets of competence of the validated GAP-test provides arguments of convergent validity for our assessment. Since five aspects of Messick's validation approach could be defended, our competence-based 360-degree assessment format shows good arguments for its validity. CONCLUSION According to these validation arguments, our assessment instrument seems to be a good option to assess competence in advanced undergraduate medical students in a summative or formative way. Developments towards assessment of postgraduate medical trainees should be explored.
Collapse
Affiliation(s)
- Sarah Prediger
- III. Department of Internal Medicine, University Medical Center Hamburg-Eppendorf, Hamburg, Germany
| | - Kristina Schick
- TUM Medical Education Center, School of Medicine, Technical University of Munich, Munich, Germany
| | - Fabian Fincke
- Department of Medical Education and Educational Research, Faculty of Medicine and Health Science, University of Oldenburg, Oldenburg, Germany
| | - Sophie Fürstenberg
- III. Department of Internal Medicine, University Medical Center Hamburg-Eppendorf, Hamburg, Germany
| | | | - Martina Kadmon
- Faculty of Medicine, University of Augsburg, Deanery, Augsburg, Germany
| | - Pascal O. Berberat
- TUM Medical Education Center, School of Medicine, Technical University of Munich, Munich, Germany
| | - Sigrid Harendza
- III. Department of Internal Medicine, University Medical Center Hamburg-Eppendorf, Hamburg, Germany
| |
Collapse
|
9
|
Sterz J, Bender B, Linßen S, Stefanescu MC, Höfer SH, Walcher F, Voss J, Seifert LB, Ruesseler M. Effects and Consequences of Being an OSCE Examiner in Surgery-A Qualitative Study. JOURNAL OF SURGICAL EDUCATION 2019; 76:433-439. [PMID: 30213735 DOI: 10.1016/j.jsurg.2018.08.003] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.2] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 02/27/2018] [Revised: 07/30/2018] [Accepted: 08/04/2018] [Indexed: 05/26/2023]
Abstract
OBJECTIVE Even though objective structured clinical examination (OSCE) is a well-investigated format for competency-based practical examination, only a few studies have explored the motivations of OSCE examiners and their opinions, both positive and negative, toward being an examiner. The aim of this study was to gain insights into the views of OSCE examiners using semi-structured interviews. DESIGN Surgical OSCE examiners were queried at two medical faculties in Germany via semi-structured interviews. The interviews were transcribed verbatim and analyzed using the techniques of structured qualitative content analysis. SETTING This study was conducted at the medical faculties of the Goethe University, Frankfurt, Germany and of the Otto-von-Guericke University, Magdeburg, Germany. PARTICIPANTS All of the study participants were surgeons working at the university hospital of one of the faculties. RESULTS A total of 29 examiners were queried until a saturation of content was achieved. A critical reflection of one's own teaching was described as a major benefit by most participants. Furthermore, they noted that the standards and competences examined during the OSCE boosted the detail of their teaching sessions in the wards. However, the examiners criticized missed operations due the examination and were not appreciated by superiors for being an examiner. Most of the examiners (22/29) preferred to be an examiner themselves rather than appointing student peer examiners. If they had appointed someone else, that would mean they would miss valuable experiences useful for their own teaching. CONCLUSIONS Being an OSCE examiner confers several advantages, notably the reflection of one's own teaching, which was described as highly valuable by the examiners.
Collapse
Affiliation(s)
- Jasmina Sterz
- University Hospital Frankfurt, Department of Trauma, Hand and Reconstructive Surgery, Frankfurt, Germany.
| | - Bernd Bender
- University Hospital Frankfurt, Department of Trauma, Hand and Reconstructive Surgery, Frankfurt, Germany.
| | - Svea Linßen
- University Hospital Frankfurt, Department of Trauma, Hand and Reconstructive Surgery, Frankfurt, Germany.
| | - Maria-Christina Stefanescu
- University Hospital Frankfurt, Department of Pediatric Surgery and Pediatric Urology, Frankfurt, Germany.
| | - Sebastian Herbert Höfer
- University Hospital Frankfurt, Department of Oral, Cranio-Maxillofacial and Facial Plastic Surgery, Frankfurt, Germany.
| | - Felix Walcher
- University Hospital Magdeburg, Department of Trauma Surgery, Magdeburg, Germany.
| | - Julia Voss
- University Hospital Magdeburg, Department of Trauma Surgery, Magdeburg, Germany.
| | - Lukas Benedikt Seifert
- University Hospital Frankfurt, Department of Oral, Cranio-Maxillofacial and Facial Plastic Surgery, Frankfurt, Germany.
| | - Miriam Ruesseler
- University Hospital Frankfurt, Department of Trauma, Hand and Reconstructive Surgery, Frankfurt, Germany.
| |
Collapse
|
10
|
[Examinations while studying medicine - more than simply grades]. Wien Med Wochenschr 2018; 169:126-131. [PMID: 30084089 DOI: 10.1007/s10354-018-0650-2] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/12/2018] [Accepted: 07/11/2018] [Indexed: 10/28/2022]
Abstract
Assessment drives learning. Examinations need to be aligned primarily with learning objectives, as well as teaching and assessment methods of the courses on offer. In doing so, various examination instruments are required to measure on levels of competency that build on one another. An appropriate mix is essential to reflect the variety of learning outcomes of a chosen curriculum. Furthermore, examinations also possess the characteristics of evaluation: They reflect the knowledge and abilities of students and assess the teaching at a defined location. Digital examinations in the form of multiple-choice-question (MCQ) testing enable a higher degree of automation and accelerate the processes of creation, implementation, and evaluation of the examination results. Thus, they enjoy increasing popularity, provided that the technical requirements for large semester cohorts are met. Shifting examination processes to computers or tablets entails not only a wealth of new challenges but also opportunities.
Collapse
|