1
|
Schleicher I, Leitner K, Juenger J, Moeltner A, Ruesseler M, Bender B, Sterz J, Schuettler KF, Koenig S, Kreuder JG. Examiner effect on the objective structured clinical exam - a study at five medical schools. BMC MEDICAL EDUCATION 2017; 17:71. [PMID: 28438196 PMCID: PMC5402669 DOI: 10.1186/s12909-017-0908-1] [Citation(s) in RCA: 15] [Impact Index Per Article: 2.1] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 09/12/2016] [Accepted: 04/06/2017] [Indexed: 05/25/2023]
Abstract
BACKGROUND The Objective Structured Clinical Examination (OSCE) is increasingly used at medical schools to assess practical competencies. To compare the outcomes of students at different medical schools, we introduced standardized OSCE stations with identical checklists. METHODS We investigated examiner bias at standardized OSCE stations for knee- and shoulder-joint examinations, which were implemented into the surgical OSCE at five different medical schools. The checklists for the assessment consisted of part A for knowledge and performance of the skill and part B for communication and interaction with the patient. At each medical faculty, one reference examiner also scored independently to the local examiner. The scores from both examiners were compared and analysed for inter-rater reliability and correlation with the level of clinical experience. Possible gender bias was also evaluated. RESULTS In part A of the checklist, local examiners graded students higher compared to the reference examiner; in part B of the checklist, there was no trend to the findings. The inter-rater reliability was weak, and the scoring correlated only weakly with the examiner's level of experience. Female examiners rated generally higher, but male examiners scored significantly higher if the examinee was female. CONCLUSIONS These findings of examiner effects, even in standardized situations, may influence outcome even when students perform equally well. Examiners need to be made aware of these biases prior to examining.
Collapse
Affiliation(s)
- Iris Schleicher
- Department of Orthopaedics, Trauma Surgery and Sportsmedicine, Agaplesion ev. Hospital Giessen, Paul-Zipp-Str.171, 35398 Giessen, Germany
| | - Karsten Leitner
- Department of Orthopaedics, Trauma Surgery and Sportsmedicine, Agaplesion ev. Hospital Giessen, Paul-Zipp-Str.171, 35398 Giessen, Germany
| | - Jana Juenger
- Department of Psychosomatic and General Internal Medicine, University of Heidelberg, 69120 Heidelberg, Germany
- Present address: Institute for medical and pharmaceutical tests, Große Langgasse 8, 55116 Mainz, Germany
| | - Andreas Moeltner
- Center of Excellence in Medical Assessment, Faculty of Medicine, University of Heidelberg, im Neuenheimer Feld 346, 69120 Heidelberg, Germany
| | - Miriam Ruesseler
- Department of Trauma, Hand and Reconstructive Surgery, University of Frankfurt, Theodor Stern Kai 7, 60590 Frankfurt am Main, Germany
| | - Bernd Bender
- Department of Trauma, Hand and Reconstructive Surgery, University of Frankfurt, Theodor Stern Kai 7, 60590 Frankfurt am Main, Germany
| | - Jasmina Sterz
- Department of General Surgery, University of Frankfurt, Theodor Stern Kai, 60590 Frankfurt am Main, Germany
| | - Karl-Friedrich Schuettler
- Center for Orthopaedics and Trauma Surgery, University Hospital Giessen and Marburg GmbH, Location Marburg, 35043 Marburg, Germany
| | - Sarah Koenig
- Department of General Surgery, University of Goettingen, Robert-Koch-Straße 40, 37075 Göttingen, Germany
- Present address: Institute for medical education and educational research, Josef-Schneider-Str. 2/D6, 97080 Würzburg, Germany
| | - Joachim Gerhard Kreuder
- Department of Orthopaedics, Trauma Surgery and Sportsmedicine, Agaplesion ev. Hospital Giessen, Paul-Zipp-Str.171, 35398 Giessen, Germany
| |
Collapse
|