1
|
Harendza S, Bußenius L, Gärtner J, Heuser M, Ahles J, Prediger S. "Fit for the finals" - project report on a telemedical training with simulated patients, peers, and assessors for the licensing exam. GMS JOURNAL FOR MEDICAL EDUCATION 2023; 40:Doc17. [PMID: 37361248 PMCID: PMC10285374 DOI: 10.3205/zma001599] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Figures] [Subscribe] [Scholar Register] [Received: 11/06/2022] [Revised: 01/11/2023] [Accepted: 02/06/2023] [Indexed: 06/28/2023]
Abstract
Background Undergraduate medical students take the licensing exam (M3) as a two-day oral-practical examination. The main requirements are to demonstrate history taking skills and coherent case presentations. The aim of this project was to establish a training in which students can test their communication skills during history taking and their clinical reasoning skills in focused case presentations. Methods In the newly developed training, final-year students took four telemedical histories in the role of physicians from simulated patients (SP). They received further findings for two SPs and presented these in a handover, in which they also received a handover of two SPs which they had not seen themselves. Each student presented one of the two received SPs in a case discussion with a senior physician. Feedback was given to the participants on their communication and interpersonal skills by the SPs with the ComCare questionnaire and on the case presentation by the senior physician. Sixty-two students from the universities of Hamburg and Freiburg in their final year participated in September 2022 and evaluated the training. Results Participants felt that the training was very appropriate for exam preparation. The SPs' feedback on communication and the senior physician's feedback on clinical reasoning skills received the highest ratings in importance to the students. Participants highly valued the practice opportunity for structured history taking and case presentation and would like to have more such opportunities in the curriculum. Conclusion Essential elements of the medical licensing exam can be represented, including feedback, in this telemedical training and it can be offered independent of location.
Collapse
Affiliation(s)
- Sigrid Harendza
- Universitätsklinikum Hamburg-Eppendorf, III. Medizinische Klinik, Hamburg, Germany
| | - Lisa Bußenius
- Universitätsklinikum Hamburg-Eppendorf, III. Medizinische Klinik, Hamburg, Germany
| | - Julia Gärtner
- Universitätsklinikum Hamburg-Eppendorf, III. Medizinische Klinik, Hamburg, Germany
| | - Miriam Heuser
- Albert-Ludwigs-Universität Freiburg, Medizinische Fakultät, Studiendekanat, Freiburg, Germany
| | - Jonathan Ahles
- Albert-Ludwigs-Universität Freiburg, Medizinische Fakultät, Studiendekanat, Freiburg, Germany
| | - Sarah Prediger
- Universitätsklinikum Hamburg-Eppendorf, III. Medizinische Klinik, Hamburg, Germany
| |
Collapse
|
2
|
Fischetti C, Bhatter P, Frisch E, Sidhu A, Helmy M, Lungren M, Duhaime E. The Evolving Importance of Artificial Intelligence and Radiology in Medical Trainee Education. Acad Radiol 2022; 29 Suppl 5:S70-S75. [PMID: 34020872 DOI: 10.1016/j.acra.2021.03.023] [Citation(s) in RCA: 8] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/21/2020] [Revised: 03/11/2021] [Accepted: 03/20/2021] [Indexed: 11/16/2022]
Abstract
Radiology education is understood to be an important component of medical school and resident training, yet lacks a standardization of instruction. The lack of uniformity in both how radiology is taught and learned has afforded opportunities for new technologies to intervene. Now with the integration of artificial intelligence within medicine, it is likely that the current medical trainee curricula will experience the impact it has to offer both for education and medical practice. In this paper, we seek to investigate the landscape of radiologic education within the current medical trainee curricula, and also to understand how artificial intelligence may potentially impact the current and future radiologic education model.
Collapse
Affiliation(s)
- Chanel Fischetti
- Brigham and Women's Department of Emergency Medicine, 75 Francis St.Neville House, Boston, MA 02115.
| | | | - Emily Frisch
- UC Irvine School of Medicine, Irvine, California
| | - Amreet Sidhu
- Department of Internal Medicine, St. Mary Mercy Hospital, Livonia, Michigan
| | - Mohammad Helmy
- Department of Radiology, UC Irvine School of Medicine, Irvine, California
| | - Matt Lungren
- Department of Radiology, Stanford Center for Artificial Intelligence in Medicine and Imaging and Stanford University Medical Center, Stanford, California
| | - Erik Duhaime
- Centaur Labs Diagnostics, Inc., Boston, Massachusetts
| |
Collapse
|
3
|
Monrad SU, Bibler Zaidi NL, Grob KL, Kurtz JB, Tai AW, Hortsch M, Gruppen LD, Santen SA. What faculty write versus what students see? Perspectives on multiple-choice questions using Bloom's taxonomy. MEDICAL TEACHER 2021; 43:575-582. [PMID: 33590781 DOI: 10.1080/0142159x.2021.1879376] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/12/2023]
Abstract
BACKGROUND Using revised Bloom's taxonomy, some medical educators assume they can write multiple choice questions (MCQs) that specifically assess higher (analyze, apply) versus lower-order (recall) learning. The purpose of this study was to determine whether three key stakeholder groups (students, faculty, and education assessment experts) assign MCQs the same higher- or lower-order level. METHODS In Phase 1, stakeholders' groups assigned 90 MCQs to Bloom's levels. In Phase 2, faculty wrote 25 MCQs specifically intended as higher- or lower-order. Then, 10 students assigned these questions to Bloom's levels. RESULTS In Phase 1, there was low interrater reliability within the student group (Krippendorf's alpha = 0.37), the faculty group (alpha = 0.37), and among three groups (alpha = 0.34) when assigning questions as higher- or lower-order. The assessment team alone had high interrater reliability (alpha = 0.90). In Phase 2, 63% of students agreed with the faculty as to whether the MCQs were higher- or lower-order. There was low agreement between paired faculty and student ratings (Cohen's Kappa range .098-.448, mean .256). DISCUSSION For many questions, faculty and students did not agree whether the questions were lower- or higher-order. While faculty may try to target specific levels of knowledge or clinical reasoning, students may approach the questions differently than intended.
Collapse
Affiliation(s)
- Seetha U Monrad
- Division of Rheumatology, Department of Internal Medicine, University of Michigan Medical School (UMMS), Ann Arbor, MA, USA
| | | | - Karri L Grob
- Office of Medical School Education, University of Michigan Medical School, Ann Arbor, MA, USA
| | - Joshua B Kurtz
- University of Michigan Medical School, Ann Arbor, MA, USA
| | - Andrew W Tai
- Division of Gastroenterology, Department of Internal Medicine, University of Michigan Medical School, Ann Arbor, MA, USA
| | - Michael Hortsch
- Department of Cell and Developmental Biology, University of Michigan Medical School, Ann Arbor, MA, USA
| | - Larry D Gruppen
- Department of Learning Health Sciences, University of Michigan Medical School, Ann Arbor, MA, USA
| | - Sally A Santen
- Department of Emergency Medicine, Virginia Commonwealth University School of Medicine, Richmond, VA, USA
| |
Collapse
|
4
|
Krasne S, Stevens CD, Kellman PJ, Niemann JT. Mastering Electrocardiogram Interpretation Skills Through a Perceptual and Adaptive Learning Module. AEM EDUCATION AND TRAINING 2021; 5:e10454. [PMID: 33796803 PMCID: PMC7995930 DOI: 10.1002/aet2.10454] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 01/25/2020] [Revised: 04/08/2020] [Accepted: 04/08/2020] [Indexed: 06/12/2023]
Abstract
Although accurate interpretation of the standard 12-lead electrocardiogram (ECG) is fundamental to diagnosing heart disease, several prior studies report low accuracy rates among medical students, residents, and practicing physicians. The objective of this study was to determine if an online ECG Perceptual and Adaptive Learning Module (ECG PALM) is an efficient instrument to teach ECG interpretation. The ECG PALM consists of 415 unique ECG tracings with associated pretest, posttest, and delayed tests, each using 30 additional ECGs to gauge the effectiveness and durability of training. Between 2013 and 2015, a total of 113 third-year and 156 fourth-year medical students and 34 first-year, 41 second-year, and 37 third-year emergency medicine residents completed the PALM and associated tests. We measured two mastery criteria: accuracy, the percentage of correct interpretations, and fluency, the percentage of images interpreted accurately within 15 seconds. The ECG PALM produced statistically significant improvements (0.0001 < p < 0.0045) in student and resident performance for both accuracy (effect size = 0.9 to 3.2) and fluency (effect size = 2.5 to 3.1) following training ranging from 46 ± 24 minutes (R3s) to 88 ± 32 minutes (third-year medical students). Medical students and residents performed significantly better on a test the year following training (delayed test) than those without prior ECG PALM training (pretest). The fluency of R3 residents in classifying the 15 diagnostic categories was less than 60% for nine of the 15 diagnoses and greater than 80% for only one. Following PALM training, fluency was higher than 80% for seven of the 15 categories and less than 60% for only two categories. Accuracy in recognizing ST-elevation myocardial infarctions (STEMIs) was high both before and after PALM training for R3s, but fluency was only 64% for anterior STEMIs on the pretest, increasing to 93% following PALM training. These observations suggest that the ECG PALM is an effective and durable supplemental tool for developing mastery in interpreting common ECG abnormalities.
Collapse
Affiliation(s)
- Sally Krasne
- Department of PhysiologyDavid Geffen School of MedicineUniversity of CaliforniaLos AngelesCA
| | - Carl D. Stevens
- Department of Emergency MedicineDavid Geffen School of MedicineUniversity of CaliforniaLos AngelesCA
- Present address:
CareOregonPortlandOR
| | | | - James T. Niemann
- Department of Emergency MedicineHarbor‐UCLA Medical CenterTorranceCA
| |
Collapse
|
5
|
Yazdani S, Hoseini Abardeh M. Five decades of research and theorization on clinical reasoning: a critical review. ADVANCES IN MEDICAL EDUCATION AND PRACTICE 2019; 10:703-716. [PMID: 31695548 PMCID: PMC6717718 DOI: 10.2147/amep.s213492] [Citation(s) in RCA: 26] [Impact Index Per Article: 5.2] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Received: 04/26/2019] [Accepted: 07/22/2019] [Indexed: 05/26/2023]
Abstract
Clinical reasoning is a complex cognitive process that is essential to evaluate and manage a patient's medical problem. The aim of this paper was to provide a critical review of the research literature on clinical reasoning theories and models. To conduct our study, we applied the process of conducting a literature review in four stages in accordance with the approach of Carnwell and Daly. First, we defined the scope of the review as being limited to clinical reasoning theories and models in medical education. In the second stage, we conducted a search based on related words in PubMed, Google Scholar, PsycINFO, ERIC, ScienceDirect and Web of Science databases. In the third stage, we classified the results of the review into three categories, and in the fourth stage, we concluded and informed further studies. Based on the inclusion and exclusion criteria, 31 articles were eligible to be reviewed. Three theories and two models were recognized and classified into three categories. Several theories and models have been proposed in relation to clinical reasoning, but it seems that these theories and models could only explain part of this complex process and not the whole process. Therefore, to fulfill this gap, it may be helpful to build a Meta-model or Meta-theory, which unified all the models, and theories of clinical reasoning.
Collapse
Affiliation(s)
- Shahram Yazdani
- Department of Medical Education, School of Medical Education, Shahid Beheshti University of Medical Sciences, Tehran, Iran
| | - Maryam Hoseini Abardeh
- Department of Medical Education, School of Medical Education, Shahid Beheshti University of Medical Sciences, Tehran, Iran
| |
Collapse
|
6
|
[Competence-based assessment in the national licensing examination in Germany]. Bundesgesundheitsblatt Gesundheitsforschung Gesundheitsschutz 2019; 61:171-177. [PMID: 29230515 DOI: 10.1007/s00103-017-2668-9] [Citation(s) in RCA: 32] [Impact Index Per Article: 6.4] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/18/2022]
Abstract
In Germany, future physicians have to pass a national licensing examination at the end of their medical studies. Passing this examination is the requirement for the license to practice medicine. The Masterplan Medizinstudium 2020 with its 41 measures aims to shift the paradigm in medical education and medical licensing examinations.The main goals of the Masterplan include the development towards competency-based and practical medical education and examination as well as the strengthening of general medicine. The healthcare policy takes into account social developments, which are very important for the medical education and licensing examination.Seven measures of the Masterplan relate to the realignment of the licensing examinations. Their function to drive learning should better support students in achieving the study goal defined in the German Medical Licensure Act: to educate a medical doctor scientifically and practically who is qualified for autonomous and independent professional practice, postgraduate education and continuous training.
Collapse
|
7
|
Leo J, Kurdi G, Matentzoglu N, Parsia B, Sattler U, Forge S, Donato G, Dowling W. Ontology-Based Generation of Medical, Multi-term MCQs. INTERNATIONAL JOURNAL OF ARTIFICIAL INTELLIGENCE IN EDUCATION 2019. [DOI: 10.1007/s40593-018-00172-w] [Citation(s) in RCA: 11] [Impact Index Per Article: 2.2] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/27/2022]
|
8
|
Harendza S, Soll H, Prediger S, Kadmon M, Berberat PO, Oubaid V. Assessing core competences of medical students with a test for flight school applicants. BMC MEDICAL EDUCATION 2019; 19:9. [PMID: 30616684 PMCID: PMC6322305 DOI: 10.1186/s12909-018-1438-1] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.4] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 03/30/2018] [Accepted: 12/26/2018] [Indexed: 05/21/2023]
Abstract
BACKGROUND Important competences of physicians regarding patient safety include communication, leadership, stress resistance, adherence to procedures, awareness, and teamwork. Similarly, while selected, prospective flight school applicants are tested for the same set of skills. The aim of our study was to assess these core competences in advanced undergraduate medical students from different medical schools. METHODS In 2017, 67 medical students (year 5 and 6) from the universities of Hamburg, Oldenburg, and TU Munich, Germany, participated in the verified Group Assessment Performance (GAP)-Test at the German Aerospace Center (DLR) in Hamburg. All participants were rated by DLR assessment observers with a set of empirically derived behavioural checklists. This lists consisted of 6-point rating scales (1: very low occurrence to 6: very high occurrence) and included the competences leadership, teamwork, stress resistance, communication, awareness, and adherence to procedures. Medical students' scores were compared with the results of 117 admitted flight school applicants. RESULTS Medical students showed significantly higher scores than admitted flight school applicants for adherence to procedures (p < .001, d = .63) and communication (p < .01, d = .62). They reached significantly lower ratings for teamwork (p < .001, d = .77), stress resistance (p < 0.001, d = .70), and awareness (p < .001, d = 1.31). Students in semester 10 showed significantly (p < .02, d = .58) higher scores in domain awareness compared to the final year students. On average, flight school entrance level was not reached by either group for this domain. CONCLUSIONS Advanced medical students' low results for awareness are alarming as awareness is essential and integrative for clinical reasoning and patient safety. Further studies should elucidate and discuss whether awareness needs to be included in medical student selection or integrated into the curriculum in training units.
Collapse
Affiliation(s)
- Sigrid Harendza
- III. Department of Internal Medicine, University Medical Center Hamburg-Eppendorf, Hamburg, Germany
- III. Medizinische Klinik, Universitätsklinikum Hamburg-Eppendorf, Martinistr. 52, D-20246 Hamburg, Germany
| | | | - Sarah Prediger
- III. Department of Internal Medicine, University Medical Center Hamburg-Eppendorf, Hamburg, Germany
| | - Martina Kadmon
- Medical Faculty, Deanery, University of Augsburg, Augsburg, Germany
| | - Pascal O. Berberat
- TUM Medical Education Center, School of Medicine, Technical University of Munich, Munich, Germany
| | | |
Collapse
|
9
|
Zaidi NLB, Grob KL, Monrad SM, Kurtz JB, Tai A, Ahmed AZ, Gruppen LD, Santen SA. Pushing Critical Thinking Skills With Multiple-Choice Questions: Does Bloom's Taxonomy Work? ACADEMIC MEDICINE : JOURNAL OF THE ASSOCIATION OF AMERICAN MEDICAL COLLEGES 2018; 93:856-859. [PMID: 29215375 DOI: 10.1097/acm.0000000000002087] [Citation(s) in RCA: 9] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/07/2023]
Abstract
Medical school assessments should foster the development of higher-order thinking skills to support clinical reasoning and a solid foundation of knowledge. Multiple-choice questions (MCQs) are commonly used to assess student learning, and well-written MCQs can support learner engagement in higher levels of cognitive reasoning such as application or synthesis of knowledge. Bloom's taxonomy has been used to identify MCQs that assess students' critical thinking skills, with evidence suggesting that higher-order MCQs support a deeper conceptual understanding of scientific process skills. Similarly, clinical practice also requires learners to develop higher-order thinking skills that include all of Bloom's levels. Faculty question writers and examinees may approach the same material differently based on varying levels of knowledge and expertise, and these differences can influence the cognitive levels being measured by MCQs. Consequently, faculty question writers may perceive that certain MCQs require higher-order thinking skills to process the question, whereas examinees may only need to employ lower-order thinking skills to render a correct response. Likewise, seemingly lower-order questions may actually require higher-order thinking skills to respond correctly. In this Perspective, the authors describe some of the cognitive processes examinees use to respond to MCQs. The authors propose that various factors affect both the question writer and examinee's interaction with test material and subsequent cognitive processes necessary to answer a question.
Collapse
Affiliation(s)
- Nikki L Bibler Zaidi
- N.L.B. Zaidi is associate director of evaluation and assessment, Office of Medical Student Education, University of Michigan Medical School, Ann Arbor, Michigan. K.L. Grob is assistant director of evaluation and assessment, Office of Medical Student Education, University of Michigan Medical School, Ann Arbor, Michigan. S.M. Monrad is science and clinical trunk director, Office of Medical Student Education, and clinical associate professor of internal medicine and learning health sciences, University of Michigan Medical School, Ann Arbor, Michigan. J.B. Kurtz is a second-year medical student, University of Michigan Medical School, Ann Arbor, Michigan. A. Tai is assistant professor of internal medicine and of microbiology and immunology, University of Michigan Medical School, Ann Arbor, Michigan. A.Z. Ahmed is assistant professor of internal medicine, University of Michigan Medical School, Ann Arbor, Michigan. L.D. Gruppen is professor, Department of Learning Health Sciences, University of Michigan Medical School, Ann Arbor, Michigan. S.A. Santen is assistant dean for educational research and quality improvement, Office of Medical Student Education, and associate professor and chair of education, Department of Emergency Medicine, University of Michigan Medical School, Ann Arbor, Michigan
| | | | | | | | | | | | | | | |
Collapse
|
10
|
Harendza S, Krenz I, Klinge A, Wendt U, Janneck M. Implementation of a Clinical Reasoning Course in the Internal Medicine trimester of the final year of undergraduate medical training and its effect on students' case presentation and differential diagnostic skills. GMS JOURNAL FOR MEDICAL EDUCATION 2017; 34:Doc66. [PMID: 29226234 PMCID: PMC5704605 DOI: 10.3205/zma001143] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.4] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Subscribe] [Scholar Register] [Received: 04/23/2017] [Revised: 06/18/2017] [Accepted: 08/18/2017] [Indexed: 05/26/2023]
Abstract
Background: Clinical reasoning, comprising the processes of clinical thinking, which form the basis of medical decisions, constitutes a central competence in the clinical routine on which diagnostic and therapeutic steps are based. In medical curricula in Germany, clinical reasoning is currently taught explicitly only to a small extend. Therefore, the aim of this project was to develop and implement a clinical reasoning course in the final year of undergraduate medical training. Project description: A clinical reasoning course with six learning units and 18 learning objectives was developed, which was taught by two to four instructors on the basis of 32 paper cases from the clinical practice of the instructors. In the years 2011 to 2013, the course of eight weeks with two hours per week was taught seven times. Before the first and after the last seminar, the participating students filled out a self-assessment questionnaire with a 6-point Likert scale regarding eight different clinical reasoning skills. At the same times, they received a patient case with the assignment to prepare a case presentation and differential diagnoses. Results: From 128 participating students altogether, 42 complete data sets were available. After the course, participants assessed themselves significantly better than before the course in all eight clinical reasoning skills, for example in "Summarizing and presentation of a paper case" or in the "Skill to enumerate differential diagnoses" (p<0.05). The greatest increase occurred in the skill to recognize typical cognitive errors in medicine and to identify risk situations for their occurrence (pre: 2.98±0.92 and retro-pre: 2.64±1.01, respectively, versus post: 4.38±0.88). Based on the ratio of number of words used per keywords used the problem presentation of the paper case was significantly more focused after the course (p=0.011). A significant increase in the number of gathered differential diagnoses was not detected after the course. Conclusion: The newly developed and established Clinical Reasoning Course leads to a gain in the desired skills from the students' self-assessment perspective and to a more structured case presentation. To establish better options to exercise clinical reasoning, a longitudinal implementation in the medical curriculum seems to be desirable. Faculty training would be useful to implement the concept as standardized as possible.
Collapse
Affiliation(s)
- Sigrid Harendza
- University Medical Centre Hamburg-Eppendorf, III. Department of Internal Medicine, Hamburg, Germany
| | - Ingo Krenz
- Praxis Blankenese, Consultant in Internal Medicine/Nephrology/Hypertensiology, Hamburg, Germany
| | - Andreas Klinge
- Diabetes Schwerpunktpraxis Eidelstedt, Consultant in Internal Medicine/ Diabetology, Hamburg, Germany
| | - Ulrike Wendt
- Bethesda Hospital Bergedorf, Department of Psychiatry and Psychotherapy, Hamburg, Germany
| | - Matthias Janneck
- University Medical Centre Hamburg-Eppendorf, III. Department of Internal Medicine, Hamburg, Germany
| |
Collapse
|
11
|
Rodríguez-Díez MC, Alegre M, Díez N, Arbea L, Ferrer M. Technical flaws in multiple-choice questions in the access exam to medical specialties ("examen MIR") in Spain (2009-2013). BMC MEDICAL EDUCATION 2016; 16:47. [PMID: 26846526 PMCID: PMC4741005 DOI: 10.1186/s12909-016-0559-7] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.1] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 02/14/2015] [Accepted: 01/26/2016] [Indexed: 06/05/2023]
Abstract
BACKGROUND The main factor that determines the selection of a medical specialty in Spain after obtaining a medical degree is the MIR ("médico interno residente", internal medical resident) exam. This exam consists of 235 multiple-choice questions with five options, some of which include images provided in a separate booklet. The aim of this study was to analyze the technical quality of the multiple-choice questions included in the MIR exam over the last five years. METHODS All the questions included in the exams from 2009 to 2013 were analyzed. We studied the proportion of questions including clinical vignettes, the number of items related to an image and the presence of technical flaws in the questions. For the analysis of technical flaws, we adapted the National Board of Medical Examiners (NBME) guidelines. We looked for 18 different issues included in the manual, grouped into two categories: issues related to testwiseness and issues related to irrelevant difficulties. RESULTS The final number of questions analyzed was 1,143. The percentage of items based on clinical vignettes increased from 50% in 2009 to 56-58% in the following years (2010-2013). The percentage of items based on an image increased progressively from 10% in 2009 to 15% in 2012 and 2013. The percentage of items with at least one technical flaw varied between 68 and 72%. We observed a decrease in the percentage of items with flaws related to testwiseness, from 30% in 2009 to 20% in 2012 and 2013. While most of these issues decreased dramatically or even disappeared (such as the imbalance in the correct option numbers), the presence of non-plausible options remained frequent. With regard to technical flaws related to irrelevant difficulties, no improvement was observed; this is especially true with respect to negative stem questions and "hinged" questions. CONCLUSION The formal quality of the MIR exam items has improved over the last five years with regard to testwiseness. A more detailed revision of the items submitted, checking systematically for the presence of technical flaws, could improve the validity and discriminatory power of the exam, without increasing its difficulty.
Collapse
Affiliation(s)
| | - Manuel Alegre
- Medical Education Department, University of Navarra, School of Medicine, C/Irunlarrea n°1, 31008 Pamplona, Spain
| | - Nieves Díez
- Medical Education Department, University of Navarra, School of Medicine, C/Irunlarrea n°1, 31008 Pamplona, Spain
| | - Leire Arbea
- Medical Education Department, University of Navarra, School of Medicine, C/Irunlarrea n°1, 31008 Pamplona, Spain
| | - Marta Ferrer
- Medical Education Department, University of Navarra, School of Medicine, C/Irunlarrea n°1, 31008 Pamplona, Spain
| |
Collapse
|