The effect of images on item statistics in multiple choice anatomy examinations

Impact of gross anatomy laboratory on student written examination performance: A 3-year study of a large-enrollment undergraduate anatomy course

The efficacy of the various pedagogies that are used in human anatomy laboratories has been extensively debated. Nevertheless, an important question remains relatively unexamined-how the learning experience in the anatomy laboratory impacts students' mastery and application of anatomical knowledge beyond the laboratory setting. In this study, the effect of a prosection-based anatomy laboratory on overall comprehension and mastery of anatomical knowledge was evaluated in an upper division undergraduate anatomy curriculum that consists of a mandatory lecture course and an optional laboratory course. This flexible curricular structure permitted assessing the merit of laboratory learning on the written examination performance of the lecture course. In 2019 and 2022, the anatomy laboratory was taught in-person using prosections, while in 2021 due to the Covid-19 pandemic related regulations, it was taught remotely with live-streaming of prosections using document cameras. In both in-person and remote instructive formats, written examination scores of the lecture course were compared between two cohorts of students: Those enrolled in lecture only and those enrolled in both lecture and laboratory. Results showed that the cohort enrolled in both lecture and laboratory courses consistently outperformed the lecture-only cohort by one full letter grade. Furthermore, when the degrees of improvement on written examination scores were compared between the two instructive formats, in-person laboratory had a greater increase compared to remote laboratory. Altogether this study demonstrates that the prosection-based anatomy laboratory enhances students' mastery of anatomical knowledge beyond the laboratory setting by promoting comprehension of spatial relationships of anatomical structures.

Online Assessment of Applied Anatomy Knowledge: The Effect of Images on Medical Students' Performance

Anatomical examinations have been designed to assess topographical and/or applied knowledge of anatomy with or without the inclusion of visual resources such as cadaveric specimens or images, radiological images, and/or clinical photographs. Multimedia learning theories have advanced the understanding of how words and images are processed during learning. However, the evidence of the impact of including anatomical and radiological images within written assessments is sparse. This study investigates the impact of including images within clinically oriented single-best-answer questions on students' scores in a tailored online tool. Second-year medical students (n = 174) from six schools in the United Kingdom participated voluntarily in the examination, and 55 students provided free-text comments which were thematically analyzed. All questions were categorized as to whether their stimulus format was purely textual or included an associated image. The type (anatomical and radiological image) and deep structure of images (question referring to a bone or soft tissue on the image) were taken into consideration. Students scored significantly better on questions with images compared to questions without images (P < 0.001), and on questions referring to bones than to soft tissue (P < 0.001), but no difference was found in their performance on anatomical and radiological image questions. The coding highlighted areas of "test applicability" and "challenges faced by the students." In conclusion, images are critical in medical practice for investigating a patient's anatomy, and this study sets out a way to understand the effects of images on students' performance and their views in commonly employed written assessments.

Illustrated versus non-illustrated anatomical test items in anatomy course tests and German Medical Licensing examinations (M1)

Illustrated Multiple-choice questions (iMCQs) form an integral part of written tests in anatomy. In iMCQs, the written question refers to various types of figures, e. g. X-ray images, micrographs of histological sections, or drawings of anatomical structures. Since the inclusion of images in MCQs might affect item performance we compared characteristics of anatomical items tested with iMCQs and non-iMCQs in seven tests of anatomy courses and in two written parts of the first section of the German Medical Licensing Examination (M1). In summary, we compared 25 iMCQs and 163 non-iMCQs from anatomy courses, and 27 iMCQs and 130 non-iMCQs from the written part of the M1 using a nonparametric test for unpaired samples. As a result, there were no significant differences in difficulty and discrimination levels between iMCQs and non-iMCQs, the same applied to an analysis stratified for MCQ formats. We conclude that the illustrated item format by itself does not seem to affect item difficulty. The present results are consistent with previous retrospective studies which showed no significant differences of test or item characteristics between iMCQs and non-iMCQs.

Determinants of Difficulty and Discriminating Power of Image-based Test Items in Postgraduate Radiological Examinations

RATIONALE AND OBJECTIVES

The psychometric characteristics of image-based test items in radiological written examinations are not well known. In this study, we explored difficulty and discriminating power of these test items in postgraduate radiological digital examinations.

MATERIALS AND METHODS

We reviewed test items of seven Dutch Radiology Progress Tests (DRPTs) that were taken from October 2013 to April 2017. The DRPT is a semiannual formative examination, required for all Dutch radiology residents. We assessed several stimulus and response characteristics of test items. The response format of test items included true or false, single right multiple choice with 2, 3, 4, or ≥5 answer options, pick-N multiple-choice, drag-and-drop, and long-list-menu formats. We calculated item P values and item-rest-correlation (Rir) values to assess difficulty and discriminating power. We performed linear regression analysis in image-based test items to investigate whether P and Rir values were significantly related to stimulus and response characteristics. Also, we compared psychometric indices between image-based test items and text-alone items.

RESULTS

P and Rir values of image-based items (n = 369) were significantly related to the type of response format (P < .001), and not to which of the seven DRPTs the item was obtained from, radiological subspecialty domain, nonvolumetric or volumetric character of images, or context-rich or context-free character of the stimulus. When accounted for type of response format, difficulty and discriminating power of image-based items did not differ significantly from text-alone items (n = 881). Test items with a relatively large number of answer options were generally more difficult, and discriminated better among high- and low-performing candidates.

CONCLUSION

In postgraduate radiological written examinations, difficulty and discriminating power of image-based test items are related to the type of response format and are comparable to those of text-alone items. We recommend a response format with a relatively large number of answer options to optimize psychometric indices of radiological image-based test items.

Climbing Bloom's taxonomy pyramid: Lessons from a graduate histology course

Bloom's taxonomy was adopted to create a subject-specific scoring tool for histology multiple-choice questions (MCQs). This Bloom's Taxonomy Histology Tool (BTHT) was used to analyze teacher- and student-generated quiz and examination questions from a graduate level histology course. Multiple-choice questions using histological images were generally assigned a higher BTHT level than simple text questions. The type of microscopy technique (light or electron microscopy) used for these image-based questions did not result in any significant differences in their Bloom's taxonomy scores. The BTHT levels for teacher-generated MCQs correlated positively with higher discrimination indices and inversely with the percent of students answering these questions correctly (difficulty index), suggesting that higher-level Bloom's taxonomy questions differentiate well between higher- and lower-performing students. When examining BTHT scores for MCQs that were written by students in a Multiple-Choice Item Development Assignment (MCIDA) there was no significant correlation between these scores and the students' ability to answer teacher-generated MCQs. This suggests that the ability to answer histology MCQs relies on a different skill set than the aptitude to construct higher-level Bloom's taxonomy questions. However, students significantly improved their average BTHT scores from the midterm to the final MCIDA task, which indicates that practice, experience and feedback increased their MCQ writing proficiency. Anat Sci Educ 10: 456-464. © 2017 American Association of Anatomists.