A case study on multiple-choice testing in anatomical sciences

Medical dissertation basics: analysis of a course of study for medical students

Background

Although the majority of medical students in Germany pursue a doctorate, only a portion of them receive a standardized scientific training, which is reflected in the quality issues seen in medical doctoral theses. The course Medical Dissertation Basics was conceptualized and scientifically monitored in order to support medical doctoral students on the one hand and to improve the quality of their scientific work on the other.

Methodology

The course consists of three modules. Module I, which is an introductory module, covers time and writing management and addresses how to approach literature and the principles of scientific work as well as the chapters required in a dissertation and the dissertation presentation and defense. In the practical module II, doctoral students write sections of their dissertation chapters and receive feedback via peer and expert reviews. Module III includes training on dissertation presentations and their defense. For objective analysis purposes, a multiple-choice test was administered before and after module I. Medical students from semesters 2 to 6 served as a control group. Questionnaires were used to subjectively analyze the training and support functions of modules I-III.

Results

High participation rates and the fact that the modules were taught numerous times show that doctoral students accept the courses. The objective analysis of module I showed a highly significant knowledge acquisition of the course group (N=55) in contrast to the control group (N=34). The doctoral students rated the course modules I-III with grades between 1.0 and 1.25 (grade A+/A; N=20-65 SD=0-0.44), felt well supported and estimated their learning success as high.

Conclusion

The study indicates knowledge acquisition in module I and a high doctoral student satisfaction with all modules. For an objective analysis of modules II-III, a comparison of completed doctoral theses (course participants vs. non-participants) would be appropriate but would only make sense in a few years. Based on the results of our study, we recommend that other faculties implement similar courses.

Climbing Bloom's taxonomy pyramid: Lessons from a graduate histology course

Bloom's taxonomy was adopted to create a subject-specific scoring tool for histology multiple-choice questions (MCQs). This Bloom's Taxonomy Histology Tool (BTHT) was used to analyze teacher- and student-generated quiz and examination questions from a graduate level histology course. Multiple-choice questions using histological images were generally assigned a higher BTHT level than simple text questions. The type of microscopy technique (light or electron microscopy) used for these image-based questions did not result in any significant differences in their Bloom's taxonomy scores. The BTHT levels for teacher-generated MCQs correlated positively with higher discrimination indices and inversely with the percent of students answering these questions correctly (difficulty index), suggesting that higher-level Bloom's taxonomy questions differentiate well between higher- and lower-performing students. When examining BTHT scores for MCQs that were written by students in a Multiple-Choice Item Development Assignment (MCIDA) there was no significant correlation between these scores and the students' ability to answer teacher-generated MCQs. This suggests that the ability to answer histology MCQs relies on a different skill set than the aptitude to construct higher-level Bloom's taxonomy questions. However, students significantly improved their average BTHT scores from the midterm to the final MCIDA task, which indicates that practice, experience and feedback increased their MCQ writing proficiency. Anat Sci Educ 10: 456-464. © 2017 American Association of Anatomists.

The effect of images on item statistics in multiple choice anatomy examinations

Although multiple choice examinations are often used to test anatomical knowledge, these often forgo the use of images in favor of text-based questions and answers. Because anatomy is reliant on visual resources, examinations using images should be used when appropriate. This study was a retrospective analysis of examination items that were text based compared to the same questions when a reference image was included with the question stem. Item difficulty and discrimination were analyzed for 15 multiple choice items given across two different examinations in two sections of an undergraduate anatomy course. Results showed that there were some differences item difficulty but these were not consistent to either text items or items with reference images. Differences in difficulty were mainly attributable to one group of students performing better overall on the examinations. There were no significant differences for item discrimination for any of the analyzed items. This implies that reference images do not significantly alter the item statistics, however this does not indicate if these images were helpful to the students when answering the questions. Care should be taken by question writers to analyze item statistics when making changes to multiple choice questions, including ones that are included for the perceived benefit of the students. Anat Sci Educ 10: 68-78. © 2016 American Association of Anatomists.

Using Rasch measurement to score, evaluate, and improve examinations in an anatomy course

Any examination that involves moderate to high stakes implications for examinees should be psychometrically sound and legally defensible. Currently, there are two broad and competing families of test theories that are used to score examination data. The majority of instructors outside the high-stakes testing arena rely on classical test theory (CTT) methods. However, advances in item response theory software have made the application of these techniques much more accessible to classroom instructors. The purpose of this research is to analyze a common medical school anatomy examination using both the traditional CTT scoring method and a Rasch measurement scoring method to determine which technique provides more robust findings, and which set of psychometric indicators will be more meaningful and useful for anatomists looking to improve the psychometric quality and functioning of their examinations. Results produced by the more robust and meaningful methodology will undergo a rigorous psychometric validation process to evaluate construct validity. Implications of these techniques and additional possibilities for advanced applications are also discussed.

Reception learning and self-discovery learning in histology: students' perceptions and their implications for assessing the effectiveness of different learning modalities

Two questionnaires were used to investigate students' perceptions of their motivation to opt for reception learning (RL) or self-discovery learning (SDL) in histology and their choices of complementary learning strategies (CLS). The results demonstrated that the motivation to attend RL sessions was higher than the motivation to attend SDL to gain new knowledge (P < 0.01) and to apply this acquired knowledge to diagnosis (P < 0.01), therapy (P < 0.01), and research (P < 0.05). Students also showed a stronger preference for RL based on motivations related to leadership (P < 0.01) and competition (P < 0.01), although the rates were very low in both cases (≤ 1.9 ± 1.1). Statistically significant differences were found between male and female students for leadership (higher in males), responsibility (higher in females), and acquiring new knowledge (higher in females only in RL). This study's findings for students' preferred CLS strategies suggested a greater need for additional complementary resources after RL than after SDL (P < 0.01). In conclusion, RL was associated with a greater need for complementary training resources such as textbooks, atlases, the internet, audiovisual media, and tutorials, whereas SDL was associated with a greater need to orient teaching and training toward medical practice. These results suggest the need to reorient both types of learning processes to enhance their effectiveness in teaching histology, especially in the case of SDL, which should place more emphasis on clinically oriented knowledge.