Exploring the validity of assessment in anatomy: do images influence cognitive processes used in answering extended matching questions?

The pattern of reporting and presenting validity evidence of extended matching questions (EMQs) in health professions education: a systematic review

The Extended matching Questions (EMQs), or R-type questions, are format of selected-response. The validity evidence for this format is crucial, but there have been reports of misunderstandings about validity. It is unclear what kinds of evidence should be presented and how to present them to support their educational impact. This review explores the pattern and quality of reporting the sources of validity evidence of EMQs in health professions education, encompassing content, response process, internal structure, relationship to other variables, and consequences. A systematic search in the electronic databases including MEDLINE via PubMed, Scopus, Web of Science, CINAHL, and ERIC was conducted to extract studies that utilize EMQs. The framework for a unitary concept of validity was applied to extract data. A total of 218 titles were initially selected, the final number of titles was 19. The most reported pieces of evidence were the reliability coefficient, followed by the relationship to another variable. Additionally, the adopted definition of validity is mostly the old tripartite concept. This study found that reporting and presenting validity evidence appeared to be deficient. The available evidence can hardly provide a strong validity argument that supports the educational impact of EMQs. This review calls for more work on developing a tool to measure the reporting and presenting validity evidence.

Show them what they can't see! An evaluation of the use of customized 3D printed models in head and neck anatomy

Difficulty in visualizing anatomical structures has been identified as a challenge in anatomy learning and the emergence of three-dimensional printed models (3DPMs) offers a potential solution. This study evaluated the effectiveness of 3DPMs for learning the arterial supply of the head and neck region. One hundred eighty-four undergraduate medical students were randomly assigned to one of four learning modalities including wet specimen, digital model, 3DPM, and textbook image. Posttest scores indicated that all four modalities supported participants' knowledge acquisition, most significantly in the wet specimen group. While the participants rated 3DPMs lower for helping correct identification of structures than wet specimens, they praised 3DPMs for their ability to demonstrate topographical relationships between the arterial supply and adjacent structures. The data further suggested that the biggest limitation of the 3DPMs was their simplicity, thus making it more difficult for users to recognize the equivalent structures on the wet specimens. It was concluded that future designs of 3DPMs will need to consider the balance between the ease of visualization of anatomical structures and the degree of complexity required for successful transfer of learning. Overall, this study presented some conflicting evidence of the favorable outcomes of 3DPMs reported in other similar studies. While effective for anatomy learning as a standalone modality, educators must identify the position 3DPM models hold relative to other modalities in the continuum of undergraduate anatomy education in order to maximize their advantages for students.

Testing anatomy: Dissecting spatial and non-spatial knowledge in multiple-choice question assessment

Limited research has been conducted on the spatial ability of veterinary students and how this is evaluated within anatomy assessments. This study describes the creation and evaluation of a split design multiple-choice question (MCQ) assessment (totaling 30 questions divided into 15 non-spatial MCQs and 15 spatial MCQs). Two cohorts were tested, one cohort received a 2D teaching method in the academic year 2014/15 (male = 15/108, female 93/108), and the second a 3D teaching method in the academic year 2015/16 (male 14/98, female 84/98). The evaluation of the MCQ demonstrated strong reliability (KR-20 = 0.71 2D and 0.63 3D) meaning the MCQ consistently tests the same construct. Factor analysis of the MCQ provides evidence of validity of the split design of the assessment (RR = 1.11, p = 0.013). Neither cohort outperformed on the non-spatial questions (p > 0.05), however, the 3D cohort performed statistically significantly higher on the spatial questions (p = 0.013). The results of this research support the design of a new anatomy assessment aimed at testing both anatomy knowledge and the problem-solving aspects of anatomical spatial ability. Furthermore, a 3D teaching method was shown to increase students' performance on anatomy questions testing spatial ability.

The Future of Morphological Science Education: Learning and Teaching Anatomy in the Wake of the COVID-19 Pandemic

The COVID-19 Pandemic has conveyed an unprecedented worldwide challenge. Although there is much emphasis on caring for patients and communities, the high incidence of SARS-CoV-2 had seriously disturbed education and calls for prompt as well as serious consideration from educators in medical schools. The necessity to teach and prepare prospective medics, as well as clinicians, has certainly not been as intense as it is currently. The global effects of coronavirus disease 2019 may cause a permanent change in the education of future clinicians. The COVID-19 era presented logistical and practical obstacles and fears for the patients' well-being, taking into consideration the fact that students may be potential channels for the spread of the virus when asymptomatic and may get infected while being in training and attending lectures. This paper discusses the present state of morphological science education, depicting the effect of COVID-19 on learning environments, as well as highlights the probable effects of COVID-19 on medical instruction in the future.

Implications of introducing case based radiological images in anatomy on teaching, learning and assessment of medical students: a mixed-methods study

BACKGROUND

Introducing radiological anatomy in the preclinical curriculum can increase the understanding of Anatomy. Regardless of the integration when teaching anatomy, it is essential to maintain oversight as to what and how much is being taught. In addition, the knowledge requirements for preclinical students should be considered. The purpose of this kind of integration is that the student should be able to apply the knowledge which can help them better understand anatomy and not to make the course more challenging. This study aimed to understand whether adding radiological images would increase the difficulty level of the questions.

METHODS

We introduced radiological images, including X Rays, CT scans and MRIs, when teaching anatomy in the preclinical curriculum. A class of 99 students were tested using A-type MCQs (n = 84). All 84 questions were categorized on whether they were case-based with or without a radiological image. The item analysis of both groups of test questions was then compared based on their difficulty and discrimination index. A qualitative student perception regarding the inclusion of radiological images in anatomy was also measured using a questionnaire with a 5-point Likert scale.

RESULTS

The results showed that the performance level of the students was similar when comparing the test questions in both groups. The item analysis of the MCQs in the two groups revealed that by integrating radiological images when teaching anatomy, the various parameters in both groups of test questions were in the same range. More than 80% of the students felt that radiological images facilitate the achievement of learning outcomes and help to apply their knowledge in clinical contexts. The study's findings reported that the rate of satisfaction by including radiological images when teaching anatomy is high.

CONCLUSION

Recognition and interpretation of images are essential in an undergraduate medical program. Students found it helpful when radiological images were introduced to them when teaching anatomy. Since the students' performance in summative exams in both groups of questions was in the same range, the findings also point out that adding radiological images when teaching anatomy does not increase the difficulty of the subject.

Online Assessment of Applied Anatomy Knowledge: The Effect of Images on Medical Students' Performance

Anatomical examinations have been designed to assess topographical and/or applied knowledge of anatomy with or without the inclusion of visual resources such as cadaveric specimens or images, radiological images, and/or clinical photographs. Multimedia learning theories have advanced the understanding of how words and images are processed during learning. However, the evidence of the impact of including anatomical and radiological images within written assessments is sparse. This study investigates the impact of including images within clinically oriented single-best-answer questions on students' scores in a tailored online tool. Second-year medical students (n = 174) from six schools in the United Kingdom participated voluntarily in the examination, and 55 students provided free-text comments which were thematically analyzed. All questions were categorized as to whether their stimulus format was purely textual or included an associated image. The type (anatomical and radiological image) and deep structure of images (question referring to a bone or soft tissue on the image) were taken into consideration. Students scored significantly better on questions with images compared to questions without images (P < 0.001), and on questions referring to bones than to soft tissue (P < 0.001), but no difference was found in their performance on anatomical and radiological image questions. The coding highlighted areas of "test applicability" and "challenges faced by the students." In conclusion, images are critical in medical practice for investigating a patient's anatomy, and this study sets out a way to understand the effects of images on students' performance and their views in commonly employed written assessments.

Effects of image-based and text-based active learning exercises on student examination performance in a musculoskeletal anatomy course

Research on the benefits of visual learning has relied primarily on lecture-based pedagogy, but the potential benefits of combining active learning strategies with visual and verbal materials on learning anatomy has not yet been explored. In this study, the differential effects of text-based and image-based active learning exercises on examination performance were investigated in a functional anatomy course. Each class session was punctuated with an average of 12 text-based and image-based active learning exercises. Participation data from 231 students were compared with their examination performance on 262 questions associated with the in-class exercises. Students also rated the helpfulness and difficulty of the in-class exercises on a survey. Participation in the active learning exercises was positively correlated with examination performance (r = 0.63, P < 0.001). When controlling for other key demographics (gender, underrepresented minority status) and prior grade point average, participation in the image-based exercises was significantly correlated with performance on examination questions associated with image-based exercises (P < 0.001) and text-based exercises (P < 0.01), while participation in text-based exercises was not. Additionally, students reported that the active learning exercises were helpful for seeing images of key ideas (94%) and clarifying key course concepts (80%), and that the image-based exercises were significantly less demanding, less hard and required less effort than text-based exercises (P < 0.05). The findings confirm the positive effect of using images and active learning strategies on student learning, and suggest that integrating them may be especially beneficial for learning anatomy. Anat Sci Educ 10: 444-455. © 2017 American Association of Anatomists.

Model building to facilitate understanding of holliday junction and heteroduplex formation, and holliday junction resolution

Students frequently expressed difficulty in understanding the molecular mechanisms involved in chromosomal recombination. Therefore, we explored alternative methods for presenting the two concepts of the double-strand break model: Holliday junction and heteroduplex formation, and Holliday junction resolution. In addition to a lecture and computer-animated video, we included a model building activity using pipe cleaners. Biotechnology undergraduates (n = 108) used the model to simulate Holliday junction and heteroduplex formation, and Holliday junction resolution. Based on student perception, an average of 12.85 and 78.35% students claimed that they completely and partially understood the two concepts, respectively. A test conducted to ascertain their understanding about the two concepts showed that 66.1% of the students provided the correct response to the three multiple choice questions. A majority of the 108 students attributed the inclusion of model building to their better understanding of Holliday junction and heteroduplex formation, and Holliday junction resolution. This underlines the importance of incorporating model building, particularly in concepts that require spatial visualization. © 2016 by The International Union of Biochemistry and Molecular Biology, 44(4):381-390, 2016.

Remote histology learning from static versus dynamic microscopic images

Histology is the study of microscopic structures in normal tissue sections. Curriculum redesign in medicine has led to a decrease in the use of optical microscopes during practical classes. Other imaging solutions have been implemented to facilitate remote learning. With advancements in imaging technologies, learning material can now be digitized. Digitized microscopy images can be presented in either a static or dynamic format. This study of remote histology education identifies whether dynamic pictures are superior to static images for the acquisition of histological knowledge. Test results of two cohorts of second-year Bachelor in Medicine students at Ghent University were analyzed in two consecutive academic years: Cohort 1 (n = 190) and Cohort 2 (n = 174). Students in Cohort 1 worked with static images whereas students in Cohort 2 were presented with dynamic images. ANCOVA was applied to study differences in microscopy performance scores between the two cohorts, taking into account any possible initial differences in prior knowledge. The results show that practical histology scores are significantly higher with dynamic images as compared to static images (F (1,361) = 15.14, P < 0.01), regardless of student's gender and performance level. Several reasons for this finding can be explained in accordance with cognitivist learning theory. Since the findings suggest that knowledge construction with dynamic pictures is stronger as compared to static images, dynamic images should be introduced in a remote setting for microscopy education. Further implementation within a larger electronic learning management system needs to be explored in future research. Anat Sci Educ 9: 222-230. © 2015 American Association of Anatomists.

Medical student perceptions of radiology use in anatomy teaching

The use of radiology in the teaching of anatomy to medical students is gaining in popularity; however, there is wide variation in how and when radiology is introduced into the curriculum. The authors sought to investigate students' perceptions regarding methods used to depict and teach anatomy and effects of integrated radiology instruction on students' abilities to correctly identify imaging modalities and anatomical structures on radiological images. First-year medical students completed questionnaires at the beginning and end of the first academic year that incorporated ten hours of radiologic anatomy teaching in the anatomy curriculum. Questions used a combination of Likert scales, rankings, and binary options. Students were tested on their ability to identify radiology modalities and anatomical structures on radiology images. Preresponse and postresponse rates were 93% (157/168) and 85% (136/160), respectively. Postmodule, 96.3% of students wanted the same or more radiology integration. Furthermore, 92.4% premodule and 96.2% postmodule agreed that "Radiology is important in medical undergraduate teaching." Modality and structure identification scores significantly increased from 59.8% to 64.3% (P < 0.001) and from 47.4% to 71.2% (P < 0.001), respectively. The top three preferred teaching formats premodule and postmodule were (1) anatomy laboratory instruction, (2) interactive sessions combining radiology with anatomy, and (3) anatomy lectures. Postmodule, 38.3% of students were comfortable reviewing radiology images. Students were positive about integrating radiology into anatomy teaching and most students wanted at least the same level of assimilation but that it is used as an adjunct rather than primary method of teaching anatomy.

A meta-analysis of the educational effectiveness of three-dimensional visualization technologies in teaching anatomy

Many medical graduates are deficient in anatomy knowledge and perhaps below the standards for safe medical practice. Three-dimensional visualization technology (3DVT) has been advanced as a promising tool to enhance anatomy knowledge. The purpose of this review is to conduct a meta-analysis of the effectiveness of 3DVT in teaching and learning anatomy compared to all teaching methods. The primary outcomes were scores of anatomy knowledge tests expressed as factual or spatial knowledge percentage means. Secondary outcomes were perception scores of the learners. Thirty-six studies met the inclusion criteria including 28 (78%) randomized studies. Based on 2,226 participants including 2,128 from studies with comparison groups, 3DVTs (1) resulted in higher (d = 0.30, 95%CI: 0.02-0.62) factual knowledge, (2) yielded significant better results (d = 0.50, 95%CI: 0.20-0.80) in spatial knowledge acquisition, and (3) produced significant increase in user satisfaction (d = 0.28, 95%CI = 0.12-0.44) and in learners' perception of the effectiveness of the learning tool (d = 0.28, 95%CI = 0.14-0.43). The total mean scores (out of five) and ±SDs for QUESTS's Quality and Strength dimensions were 4.38 (±SD 1.3) and 3.3 (±SD 1.7), respectively. The results have high internal validity, for the improved outcomes of 3DVTs compared to other methods of anatomy teaching. Given that anatomy teaching and learning in the modern medical school appears to be approaching a crisis, 3DVT can be a potential solution to the problem of inadequate anatomy pedagogy.

Is a picture worth a thousand words: an analysis of the difficulty and discrimination parameters of illustrated vs. text-alone vignettes in histology multiple choice questions

BACKGROUND

Advances in cognitive load theory have led to greater understanding of how we process verbal and visual material during learning, but the evidence base with regard to the use of images within written assessments is still sparse. This study examines whether the inclusion of images within the stimulus format of multiple choice questions (MCQs) has a predictable or consistent influence on psychometric item properties, such as difficulty or discrimination.

METHODS

Item analysis data from three consecutive years of histology multiple choice examinations were included in this study. All items were reviewed and categorised according to whether their stem, or stimulus format, was purely textual or included an associated image.

RESULTS

A total of 195 MCQs were identified for inclusion and analysed using classical test theory; 95 used text alone and 100 included an image within the question stem. The number of students per examination ranged from 277 to 347, with a total of 60,850 student-question interactions. We initially examined whether the inclusion of an image within the item stem altered the item difficulty using Mann-Whitney U. The median item difficulty for images with purely textual stems was 0.77, while that for items incorporating an appropriate image was 0.80; this difference was not significant (0.77 vs. 0.80; p = 0.862, Mann-Whitney-U = 4818.5). Mean values showed that the Item Discrimination Index appeared unaffected by the inclusion of an image within the stem, and Item point biserial correlation also showed no difference in means between these two groups (Independent samples t-test; 2-tailed).

CONCLUSION

We demonstrate that the addition of illustrations within undergraduate histology Multiple Choice Question stems has no overall influence on item difficulty, or measures of item discrimination. We conclude that the use of images in this context is statistically uncritical, and suggest that their inclusion within item stems should be based upon the principles of constructive alignment. However, further research with respect to the effect of images within item stems on cognitive processing, particularly with regard to image complexity or type, would enable the development of more informed guidelines for their use.