Patients don't come with multiple choice options: essay-based assessment in UME

Question banks: credit? Or debit? A qualitative exploration of their use among medical students

BACKGROUND

Online question banks are the most widely used education resource amongst medical students. Despite this there is an absence of literature outlining how and why they are used by students. Drawing on Deci and Ryan's self-determination theory, our study aimed to explore why and how early-stage medical students use question banks in their learning and revision strategies.

METHODS

The study was conducted at Newcastle University Medical School (United Kingdom and Malaysia). Purposive, convenience and snowball sampling of year two students were employed. Ten interviews were conducted. Thematic analysis was undertaken iteratively, enabling exploration of nascent themes. Data collection ceased when no new perspectives were identified.

RESULTS

Students' motivation to use question banks was predominantly driven by extrinsic motivators, with high-stakes exams and fear of failure being central. Their convenience and perceived efficiency promoted autonomy and thus motivation. Rapid feedback cycles and design features consistent with gamification were deterrents to intrinsic motivation. Potentially detrimental patterns of question bank use were evident: cueing, avoidance and memorising. Scepticism regarding veracity of question bank content was absent.

CONCLUSIONS

We call on educators to provide students with guidance about potential pitfalls associated with question banks and to reflect on potential inequity of access to these resources.

Giving a Voice to Patients With Smell Disorders Associated With COVID-19: Cross-Sectional Longitudinal Analysis Using Natural Language Processing of Self-Reports

BACKGROUND

Smell disorders are commonly reported with COVID-19 infection. The smell-related issues associated with COVID-19 may be prolonged, even after the respiratory symptoms are resolved. These smell dysfunctions can range from anosmia (complete loss of smell) or hyposmia (reduced sense of smell) to parosmia (smells perceived differently) or phantosmia (smells perceived without an odor source being present). Similar to the difficulty that people experience when talking about their smell experiences, patients find it difficult to express or label the symptoms they experience, thereby complicating diagnosis. The complexity of these symptoms can be an additional burden for patients and health care providers and thus needs further investigation.

OBJECTIVE

This study aims to explore the smell disorder concerns of patients and to provide an overview for each specific smell disorder by using the longitudinal survey conducted in 2020 by the Global Consortium for Chemosensory Research, an international research group that has been created ad hoc for studying chemosensory dysfunctions. We aimed to extend the existing knowledge on smell disorders related to COVID-19 by analyzing a large data set of self-reported descriptive comments by using methods from natural language processing.

METHODS

We included self-reported data on the description of changes in smell provided by 1560 participants at 2 timepoints (second survey completed between 23 and 291 days). Text data from participants who still had smell disorders at the second timepoint (long-haulers) were compared with the text data of those who did not (non-long-haulers). Specifically, 3 aims were pursued in this study. The first aim was to classify smell disorders based on the participants' self-reports. The second aim was to classify the sentiment of each self-report by using a machine learning approach, and the third aim was to find particular food and nonfood keywords that were more salient among long-haulers than those among non-long-haulers.

RESULTS

We found that parosmia (odds ratio [OR] 1.78, 95% CI 1.35-2.37; P<.001) as well as hyposmia (OR 1.74, 95% CI 1.34-2.26; P<.001) were more frequently reported in long-haulers than in non-long-haulers. Furthermore, a significant relationship was found between long-hauler status and sentiment of self-report (P<.001). Finally, we found specific keywords that were more typical for long-haulers than those for non-long-haulers, for example, fire, gas, wine, and vinegar.

CONCLUSIONS

Our work shows consistent findings with those of previous studies, which indicate that self-reports, which can easily be extracted online, may offer valuable information to health care and understanding of smell disorders. At the same time, our study on self-reports provides new insights for future studies investigating smell disorders.

Efficacy of Google Form-based MCQ tests for formative assessment in medical biochemistry education

BACKGROUND

Biochemistry, being a vast and complex subject, can be challenging for Phase I MBBS students to comprehend and retain. Embracing rapidly evolving technology can facilitate a more accessible learning experience. In this study, we investigated the effectiveness of using Google Form-based multiple-choice question (MCQ) tests as a formative assessment tool after each biochemistry lecture series. The aim was to assess the improvement and gather feedback of Phase I MBBS students on the utility of this assessment tool.

MATERIALS AND METHODS

This educational prospective longitudinal study was conducted by the Department of Biochemistry at a university-affiliated medical college and tertiary care hospital. The study included 150 Phase I MBBS students as participants. Google Form-based MCQ tests were implemented as educational interventions after each lecture series during the study period. The study compared the internal assessment (IA) MCQ marks of students before and after the implementation of the intervention. In addition, feedback questionnaires were collected from the students.

RESULTS

There was a significant improvement in students' scores between the first IA (mean ± standard deviation [SD], 8.16 ± 3.08) and second IA (mean ± SD, 17.64 ± 2.02) (P < 0.0001). According to students' feedback, 149 out of 150 (99.3%) students found the use of Google Form-based MCQ tests as a formative assessment tool in the teaching-learning process to be highly beneficial and motivated them to engage in their biochemistry studies.

CONCLUSION

With the shift toward competency-based medical education (CBME) in India, it is crucial for educators to embrace novel teaching-learning and evaluation approaches. Our study highlighted the efficacy of employing Google Form-based MCQ tests in enhancing students' comprehension of the biochemistry subject, evaluating their scores and improving the overall quality of learning. Through this mode of assessment, teachers were able to provide targeted feedback on areas that required improvement, thereby enhancing the learning experience.

What have we learned about constructed response short-answer questions from students and faculty? A multi-institutional study

PURPOSE

The purpose of this study was to enrich understanding about the perceived benefits and drawbacks of constructed response short-answer questions (CR-SAQs) in preclerkship assessment using Norcini's criteria for good assessment as a framework.

METHODS

This multi-institutional study surveyed students and faculty at three institutions. A survey using Likert scale and open-ended questions was developed to evaluate faculty and student perceptions of CR-SAQs using the criteria of good assessment to determine the benefits and drawbacks. Descriptive statistics and Chi-square analyses are presented, and open responses were analyzed using directed content analysis to describe benefits and drawbacks of CR-SAQs.

RESULTS

A total of 260 students (19%) and 57 faculty (48%) completed the survey. Students and faculty report that the benefits of CR-SAQs are authenticity, deeper learning (educational effect), and receiving feedback (catalytic effect). Drawbacks included feasibility, construct validity, and scoring reproducibility. Students and faculty found CR-SAQs to be both acceptable (can show your reasoning, partial credit) and unacceptable (stressful, not USMLE format).

CONCLUSIONS

CR-SAQs are a method of aligning innovative curricula with assessment and could enrich the assessment toolkit for medical educators.

Development of a Clinical Simulation Video to Evaluate Multiple Domains of Clinical Competence: Cross-Sectional Study

BACKGROUND

Medical students in Japan undergo a 2-year postgraduate residency program to acquire clinical knowledge and general medical skills. The General Medicine In-Training Examination (GM-ITE) assesses postgraduate residents' clinical knowledge. A clinical simulation video (CSV) may assess learners' interpersonal abilities.

OBJECTIVE

This study aimed to evaluate the relationship between GM-ITE scores and resident physicians' diagnostic skills by having them watch a CSV and to explore resident physicians' perceptions of the CSV's realism, educational value, and impact on their motivation to learn.

METHODS

The participants included 56 postgraduate medical residents who took the GM-ITE between January 21 and January 28, 2021; watched the CSV; and then provided a diagnosis. The CSV and GM-ITE scores were compared, and the validity of the simulations was examined using discrimination indices, wherein ≥0.20 indicated high discriminatory power and >0.40 indicated a very good measure of the subject's qualifications. Additionally, we administered an anonymous questionnaire to ascertain participants' views on the realism and educational value of the CSV and its impact on their motivation to learn.

RESULTS

Of the 56 participants, 6 (11%) provided the correct diagnosis, and all were from the second postgraduate year. All domains indicated high discriminatory power. The (anonymous) follow-up responses indicated that the CSV format was more suitable than the conventional GM-ITE for assessing clinical competence. The anonymous survey revealed that 12 (52%) participants found the CSV format more suitable than the GM-ITE for assessing clinical competence, 18 (78%) affirmed the realism of the video simulation, and 17 (74%) indicated that the experience increased their motivation to learn.

CONCLUSIONS

The findings indicated that CSV modules simulating real-world clinical examinations were successful in assessing examinees' clinical competence across multiple domains. The study demonstrated that the CSV not only augmented the assessment of diagnostic skills but also positively impacted learners' motivation, suggesting a multifaceted role for simulation in medical education.

A Classroom Practice and Assessment Resource for Dental Clinical Treatment Planning

Introduction

Dental students' opportunities to practice clinical treatment planning prior to the clinical experience are often limited. This resource permits students to practice treatment planning in the classroom and allows the instructor to efficiently provide feedback to a large number of students.

Methods

I developed a computer-based treatment plan worksheet (Microsoft Excel, Google Sheets, or Qualtrics) consisting of four sequence steps and a list of Common Dental Terminology (CDT) codes. I implemented this educational activity in the Case Management and Treatment Planning course (second year, spring quarter). The course included 10 weekly sessions of 1 hour each. I gave students a practice case at the beginning of the course. They submitted comprehensive sequenced treatment plans as assignments using the Qualtrics treatment plan worksheet 2 days after the interdisciplinary treatment planning didactic session. In the subsequent debrief session, I discussed the case with the entire class and focused on themes observed from the students' performance on the assignments.

Results

I graded all students' submissions (n = 87) using CDT codes both with and without consideration of sequence steps and observed variations in students' performance. Many students (n = 36, 41%) opted to give feedback through an anonymous survey, with 88%-96% of them agreeing or strongly agreeing that the educational activity allowed them to practice, demonstrate their knowledge, and improve their skills in treatment planning.

Discussion

This computer-based worksheet lets students practice clinical treatment planning in the classroom and allows the instructor to effectively assess a large number of students' assignments.

A Multi-institutional Study of the Feasibility and Reliability of the Implementation of Constructed Response Exam Questions

PROBLEM

Some medical schools have incorporated constructed response short answer questions (CR-SAQs) into their assessment toolkits. Although CR-SAQs carry benefits for medical students and educators, the faculty perception that the amount of time required to create and score CR-SAQs is not feasible and concerns about reliable scoring may impede the use of this assessment type in medical education.

INTERVENTION

Three US medical schools collaborated to write and score CR-SAQs based on a single vignette. Study participants included faculty question writers (N = 5) and three groups of scorers: faculty content experts (N = 7), faculty non-content experts (N = 6), and fourth-year medical students (N = 7). Structured interviews were performed with question writers and an online survey was administered to scorers to gather information about their process for creating and scoring CR-SAQs. A content analysis was performed on the qualitative data using Bowen's model of feasibility as a framework. To examine inter-rater reliability between the content expert and other scorers, a random selection of fifty student responses from each site were scored by each site's faculty content experts, faculty non-content experts, and student scorers. A holistic rubric (6-point Likert scale) was used by two schools and an analytic rubric (3-4 point checklist) was used by one school. Cohen's weighted kappa (κw) was used to evaluate inter-rater reliability.

CONTEXT

This research study was implemented at three US medical schools that are nationally dispersed and have been administering CR-SAQ summative exams as part of their programs of assessment for at least five years. The study exam question was included in an end-of-course summative exam during the first year of medical school.

IMPACT

Five question writers (100%) participated in the interviews and twelve scorers (60% response rate) completed the survey. Qualitative comments revealed three aspects of feasibility: practicality (time, institutional culture, teamwork), implementation (steps in the question writing and scoring process), and adaptation (feedback, rubric adjustment, continuous quality improvement). The scorers' described their experience in terms of the need for outside resources, concern about lack of expertise, and value gained through scoring. Inter-rater reliability between the faculty content expert and student scorers was fair/moderate (κw=.34-.53, holistic rubrics) or substantial (κw=.67-.76, analytic rubric), but much lower between faculty content and non-content experts (κw=.18-.29, holistic rubrics; κw=.59-.66, analytic rubric).

LESSONS LEARNED

Our findings show that from the faculty perspective it is feasible to include CR-SAQs in summative exams and we provide practical information for medical educators creating and scoring CR-SAQs. We also learned that CR-SAQs can be reliably scored by faculty without content expertise or senior medical students using an analytic rubric, or by senior medical students using a holistic rubric, which provides options to alleviate the faculty burden associated with grading CR-SAQs.

Natural language processing for automatic evaluation of free-text answers - a feasibility study based on the European Diploma in Radiology examination

BACKGROUND

Written medical examinations consist of multiple-choice questions and/or free-text answers. The latter require manual evaluation and rating, which is time-consuming and potentially error-prone. We tested whether natural language processing (NLP) can be used to automatically analyze free-text answers to support the review process.

METHODS

The European Board of Radiology of the European Society of Radiology provided representative datasets comprising sample questions, answer keys, participant answers, and reviewer markings from European Diploma in Radiology examinations. Three free-text questions with the highest number of corresponding answers were selected: Questions 1 and 2 were "unstructured" and required a typical free-text answer whereas question 3 was "structured" and offered a selection of predefined wordings/phrases for participants to use in their free-text answer. The NLP engine was designed using word lists, rule-based synonyms, and decision tree learning based on the answer keys and its performance tested against the gold standard of reviewer markings.

RESULTS

After implementing the NLP approach in Python, F1 scores were calculated as a measure of NLP performance: 0.26 (unstructured question 1, n = 96), 0.33 (unstructured question 2, n = 327), and 0.5 (more structured question, n = 111). The respective precision/recall values were 0.26/0.27, 0.4/0.32, and 0.62/0.55.

CONCLUSION

This study showed the successful design of an NLP-based approach for automatic evaluation of free-text answers in the EDiR examination. Thus, as a future field of application, NLP could work as a decision-support system for reviewers and support the design of examinations being adjusted to the requirements of an automated, NLP-based review process.

CLINICAL RELEVANCE STATEMENT

Natural language processing can be successfully used to automatically evaluate free-text answers, performing better with more structured question-answer formats. Furthermore, this study provides a baseline for further work applying, e.g., more elaborated NLP approaches/large language models.

KEY POINTS

• Free-text answers require manual evaluation, which is time-consuming and potentially error-prone. • We developed a simple NLP-based approach - requiring only minimal effort/modeling - to automatically analyze and mark free-text answers. • Our NLP engine has the potential to support the manual evaluation process. • NLP performance is better on a more structured question-answer format.

Consolidation Carnival: A Case-Based Approach to Reviewing Musculoskeletal Anatomy in an Undergraduate Medical Curriculum

Introduction

Recent changes in undergraduate medical curricula have resulted in time constraints that are particularly challenging, especially when students must learn large blocks of musculoskeletal anatomy content. Consequently, students have resorted to rote memorization to cope, which counteracts our established collaborative self-directed learning model.

Methods

For a 6-week musculoskeletal anatomy course, two structured case-based review sessions are described, each following the completion of two five-hour lab sessions, two on the upper extremities and two on the lower extremities. These largely self-directed review sessions consisted of 6 students rotating through 7 to 8 stations every 10 minutes where clinical cases with follow-up questions were projected on large screens. The students were expected to work collaboratively to solve the cases utilizing the prosected specimens provided and discuss the accompanying answers at the end of each case.

Results

Ninety-four per cent of the students who participated in this study agreed that the case-based review sessions provided a helpful overview of musculoskeletal anatomy content. Student performance on the open-ended, case-based musculoskeletal examination questions showed no significant difference in performance on shoulder, hand, hip, thigh, and leg questions. There was, however, a statistically significant decrease in the students' scores on a forearm question in 2021 compared to 2019.

Conclusion

This paper describes our integrated, collaborative musculoskeletal course, including case-based review sessions, which was positively received by students as having value in reviewing the musculoskeletal content though it was not found to improve examination performance.

Integration of physiology in a curriculum on human structure: a snapshot of the cardiovascular block

With the gradual shift from discipline-based to competency-based medical education, the integrated curriculum has become a popular model for connecting basic science and clinical content in undergraduate medical education. Despite its popularity, there are concerns that important physiological concepts are not adequately addressed. We describe the spiral integration of physiology content in the 5-week Cardiovascular block of our Homeostasis course at the Zucker School of Medicine. We also describe our approach to incorporating physiology into an integrated, constructed response, short-answer assessment format. Our approach to spiral integration consists of rotating lab stations that highlight the distinction between normal and abnormal states, linked with appropriate clinical interventions. Physiology is at the core of integration in any curriculum and the basis of all applied fields of medicine, hence our approach is that teaching structural relationships would not be valuable without consideration of its functions, which can then be utilized in discussion of clinical presentations, imaging, and relevant pathologies. Likewise, our integrated assessments require the students to compose their answers to the questions from scratch, which creates a shift in mode of students' preparation from rote memorizations to more cognitive processing that enhances critical thinking.

Evidence for commonly used teaching, learning and assessment methods in contact lens clinical skills education

INTRODUCTION

Evidence based practice is now an important part of healthcare education. The aim of this narrative literature review was to determine what evidence exists on the efficacy of commonly used teaching and learning and assessment methods in the realm of contact lens skills education (CLE) in order to provide insights into best practice. A summary of the global regulation and provision of postgraduate learning and continuing professional development in CLE is included.

METHOD

An expert panel of educators was recruited and completed a literature review of current evidence of teaching and learning and assessment methods in healthcare training, with an emphasis on health care, general optometry and CLE.

RESULTS

No direct evidence of benefit of teaching and learning and assessment methods in CLE were found. There was evidence for the benefit of some teaching and learning and assessment methods in other disciplines that could be transferable to CLE and could help students meet the intended learning outcomes. There was evidence that the following teaching and learning methods helped health-care and general optometry students meet the intended learning outcomes; clinical teaching and learning, flipped classrooms, clinical skills videos and clerkships. For assessment these methods were; essays, case presentations, objective structured clinical examinations, self-assessment and formative assessment. There was no evidence that the following teaching and learning methods helped health-care and general optometry students meet the intended learning outcomes; journal clubs and case discussions. Nor was any evidence found for the following assessment methods; multiple-choice questions, oral examinations, objective structured practical examinations, holistic assessment, and summative assessment.

CONCLUSION

Investigation into the efficacy of common teaching and learning and assessment methods in CLE are required and would be beneficial for the entire community of contact lens educators, and other disciplines that wish to adapt this approach of evidence-based teaching.

Very Short Answer Questions in Team-Based Learning: Limited Effect on Peer Elaboration and Memory

INTRODUCTION

In team-based learning (TBL), single best answer questions (SBAQs) are traditionally used because immediate computer-assisted feedback facilitates team discussions. Recent improvements in digital marking systems and criticisms of non-analytical strategies in SBAQs have prompted the consideration of very short answer questions (VSAQs) as an alternative to SBAQs. We aim to compare the effect of VSAQs and SBAQs on peer elaboration and knowledge retention in TBL.

MATERIALS AND METHODS

Twenty-four second-year students from the Lee Kong Chian School of Medicine participated in a mixed-methods study that included a randomised controlled crossover trial with two intervention arms (TBL-VSAQs and TBL-SBAQs). Two TBL sessions were conducted, with one topic covered each. Students were randomly allocated into six teams of four members. Individual and team scores and completion times were measured, and students were surveyed on their TBL experience. A follow-up quiz on the same topics was administered two weeks later to assess knowledge retention.

RESULTS

Individuals scored lower for VSAQs than SBAQs in the second TBL topic (7.17 ± 1.52 versus 8.25 ± 1.48; p = 0.046), while findings in other metrics were non-significant. Follow-up quiz scores showed no significant difference in knowledge retention, although effect size and power were low. Students perceived VSAQs as more authentic and challenging, though most preferred the continued use of SBAQs for TBL.

DISCUSSION

VSAQs have a limited impact on peer elaboration and knowledge retention versus SBAQs in TBL.

CONCLUSIONS

We should be circumspect about implementing VSAQs in TBL especially for medical students at the pre-clinical level.

SUPPLEMENTARY INFORMATION

The online version contains supplementary material available at 10.1007/s40670-022-01716-5.

Fostering uncertainty tolerance in anatomy education: Lessons learned from how humanities, arts and social science (HASS) educators develop learners' uncertainty tolerance

Uncertainty tolerance, individuals' perceptions/responses to uncertain stimuli, is increasingly recognized as critical to effective healthcare practice. While the Covid-19 pandemic generated collective uncertainty, healthcare-related uncertainty is omnipresent. Correspondingly, there is increasing focus on uncertainty tolerance as a health professional graduate "competency," and a concomitant interest in identifying pedagogy fostering learners' uncertainty tolerance. Despite these calls, practical guidelines for educators are lacking. There is some initial evidence that anatomy education can foster medical students' uncertainty tolerance (e.g., anatomical variation and dissection novelty), however, there remains a knowledge gap regarding robust curriculum-wide uncertainty tolerance teaching strategies. Drawing upon humanities, arts and social sciences (HASS) educators' established uncertainty tolerance pedagogies, this study sought to learn from HASS academics' experiences with, and teaching practices related to, uncertainty pedagogy using a qualitative, exploratory study design. Framework analysis was undertaken using an abductive approach, wherein researchers oscillate between inductive and deductive coding (comparing to the uncertainty tolerance conceptual model). During this analysis, the authors analyzed ~386 min of data from purposively sampled HASS academics' (n = 14) discussions to address the following research questions: (1) What teaching practices do HASS academics' perceive as impacting learners' uncertainty tolerance, and (2) How do HASS academics execute these teaching practices? The results extend current understanding of the moderating effects of education on uncertainty tolerance and supports prior findings that the anatomy learning environment is ripe for supporting learner uncertainty tolerance development. This study adds to growing literature on the powerful moderating effect education has on uncertainty tolerance and proposes translation of HASS uncertainty tolerance teaching practices to enhance anatomy education.

Examining the Threat of ChatGPT to the Validity of Short Answer Assessments in an Undergraduate Medical Program

OBJECTIVES

ChatGPT is an artificial intelligence model that can interpret free-text prompts and return detailed, human-like responses across a wide domain of subjects. This study evaluated the extent of the threat posed by ChatGPT to the validity of short-answer assessment problems used to examine pre-clerkship medical students in our undergraduate medical education program.

METHODS

Forty problems used in prior student assessments were retrieved and stratified by levels of Bloom's Taxonomy. Thirty of these problems were submitted to ChatGPT-3.5. For the remaining 10 problems, we retrieved past minimally passing student responses. Six tutors graded each of the 40 responses. Comparison of performance between student-generated and ChatGPT-generated answers aggregated as a whole and grouped by Bloom's levels of cognitive reasoning, was done using t-tests, ANOVA, Cronbach's alpha, and Cohen's d. Scores for ChatGPT-generated responses were also compared to historical class average performance.

RESULTS

ChatGPT-generated responses received a mean score of 3.29 out of 5 (n = 30, 95% CI 2.93-3.65) compared to 2.38 for a group of students meeting minimum passing marks (n = 10, 95% CI 1.94-2.82), representing higher performance (P = .008, η2 = 0.169), but was outperformed by historical class average scores on the same 30 problems (mean 3.67, P = .018) when including all past responses regardless of student performance level. There was no statistically significant trend in performance across domains of Bloom's Taxonomy.

CONCLUSION

While ChatGPT was able to pass short answer assessment problems spanning the pre-clerkship curriculum, it outperformed only underperforming students. We remark that tutors in several cases were convinced that ChatGPT-produced responses were produced by students. Risks to assessment validity include uncertainty in identifying struggling students and inability to intervene in a timely manner. The performance of ChatGPT on problems requiring increasing demands of cognitive reasoning warrants further research.

Considering medical students' perception, concerns and needs for e-exam during COVID-19: a promising approach to improve subject specific e-exams

The COVID-19 pandemic forced a rapid shift to digital strategies including e-exams in medical schools. However, there are significant concerns, predominately from student perspectives, and further data is required to successfully establish e-assessment in the medical curricula. The objective of the study was to examine medical students' perceptions, concerns, and needs regarding e-assessment to establish a comprehensive e-exam based on these and previous findings and to evaluate its effectiveness in terms of examinee perceptions and further needs. During the 2021 summer term, a cross-sectional study using qualitative and quantitative methods was conducted among all 1077 students at the School of Medicine, Technical University of Munich. They were asked to provide information regarding their characteristics, preferred exam format, e-assessment perception, concerns, and needs in an online questionnaire. Based on these findings, a pilot e-exam including an e-exam preparation for the students were established and subsequently evaluated among 125 pilot e-exam examinees under study consideration via an online-questionnaire. Of the 317 pre-exam participants (73.2% female), 70.3% preferred in-person exams and showed concerns about the technological framework, privacy, and examination requirements. Qualitative analysis showed that these concerns lead to additional exam stress and fear of failure. The 34 (79.4% female) participants who participated in the evaluation survey showed a significantly more positive e-exam perception. The fairness of the platform, the independence from an internet connection, the organization including the e-exam preparation, and the consideration of participant needs were discussed as particularly positive in the open-ended comments. In both surveys, participants requested uniform platforms and processes for all subjects. This study provides evidence for a positive, complementary role of student participation in a successful e-exam implementation. Furthermore, when establishing an e-exam format in the medical curricula, e-exam training, equal accessibility, availability offline, and all-round fairness should be considered.

Medical education across three colleges of medicine: perspectives of medical students

Aim

This study aimed to explore and evaluate various components of the medical education process (lectures, labs, small-group discussions, clinical rotations, and undergraduate research) in three colleges of medicine in Jordan.

Methods

This cross-sectional questionnaire-based study included 849 undergraduate students from three main medical colleges in Jordan. Statically valid responses were considered for 684 students. The participants were from Jordan University of Science and Technology, Yarmouk University, and the University of Jordan.

Results

The distribution of students according to their admission status was 276 (40%) regular, 266 (38.9%) parallel, and 142 (20.8%) international programs. Personal interest and self-initiation were the major motives for studying medicine in 66.1%. Regarding the frequency of attending classes, University of Jordan students reported the highest rate of regular classes' attendance (93%). The study also reported that lecture notes and textbooks were the main sources of learning for medical students. The study also reported superior academic performance of students in the regular program compared to students in the parallel and international programs. Participants of the study criticized the medical curricula in the three colleges mentioned above because of the lack of active research programs. Most of the students (40%–56%) also complained that the lectures within the modules were not well-integrated, and they felt that the academic environment was moderate (48–59%). In addition, most students in the clinical phase complained of overcrowding in hospital wards during clinical rotation.

Conclusions

Based on students' feedback, multiple aspects of the medical education process require substantial reform to meet the expectations of medical students in Jordan.

Should multiple-choice questions get the SAQ? Development of a short-answer question writing rubric

INTRODUCTION

Short-answer questions (SAQs) are often used to assess pharmacy student competency. However, the literature lacks guidance on SAQ development strategies, resulting in varying practices between SAQ writers. Understanding student and faculty perceptions of what constitutes a high-quality SAQ can identify best practices for SAQ development.

METHODS

We surveyed second-year pharmacy students at the University of California San Francisco (UCSF) to assess their perceptions of SAQs. Likert-type data were descriptively analyzed, and open-ended responses were analyzed using thematic analysis; we used these results to draft an initial SAQ checklist. We then conducted focus groups of UCSF pharmacy faculty to explore their experiences writing SAQs. Transcripts were analyzed using the survey codebook and de novo codes to generate themes. We used the focus group findings to finalize the checklist.

RESULTS

Seventy-five students (82%) completed the survey. Students identified "structure" (organizing into sections/lists) and "content" (clearly delineating student's task) as two ways to improve SAQ quality. Eight faculty participated in focus groups of two to three participants each. Faculty expanded on these previous themes and also identified a new theme, "process." This included peer review of SAQs as well as the iterative process involved in writing the SAQ, model answer, and grading rubric.

CONCLUSIONS

Content, structure, and process were the three areas identified for the improvement of SAQ quality at our institution. A checklist outlining best practices in these areas may be best implemented and adopted within the SAQ peer-review process.

Multiple-choice questions in assessment: Perceptions of medical students from low-resource setting

BACKGROUND

In curriculum reforms under Competency-Based Medical Education (CBME) introduced by the National Medical Commission, medical students are recognized as one of the stake holders. The well-designed multiple-choice questions (MCQs) help students in assessment as well as promote deep learning. The objective of this study is to assess the perceptions of the Indian medical students towards the examination patterns in their undergraduate curriculum with particular reference to MCQs.

MATERIALS AND METHODS

An observational cross-sectional study was conducted among the medical colleges in India. A total of 7875 students took part in the study. The study instrument was a validated pretested self-administered questionnaire. The study was conducted as an online survey using the Google Forms platform. Inclusion criteria were students from all professional years as well as interns and those who were willing to participate in the study.

RESULTS

For the question of initiation of MCQs as a tool for the assessment, many of the students (46%) expressed that it should start from the 1st MBBS itself. The majority of the students (85%) felt that MCQ oriented preparation will be helpful for MBBS professional (theory and practical) examinations. Most of the students (90.9%) preferred that the MCQs should be concept and case scenario based. To the question of whether the current system of the university theory examinations of essays and short notes be replaced in total with MCQs so that they are more prepared for NEXT and USMLE examination, 65.5% responded with an answer of yes.

CONCLUSION

The findings of this study provide input to the regulatory authorities and other stakeholders responsible for designing and implementing the medical curriculum.

Off the mark: Repetitive marking undermines essay evaluations due to boredom

Essay-style assessment is widespread in education. Nonetheless, research shows that this tool can suffer from low reliability and validity. We attribute this problem partly to the boredom that marking multiple essays causes. Specifically, we propose that boredom in markers is associated with systematically lower marks on essays. To test this, we asked participants (N = 100) with an undergraduate degree to mark essays. The majority of these participants had at least some experience with marking. After marking each essay, participants indicated how bored they were. We found an increase in boredom over time and that higher boredom was associated with lower marks. Furthermore, offering a marking rubric did not prevent this problematic impact of boredom. These findings have implications for the validity of essays as an assessment tool and raise concerns about repetitive marking practices in general.

Assessing Anatomy Education: A Perspective from Design

Medical and healthcare practice is likely to see fundamental changes in the future that will require a different approach to the way in which we educate, train, and assess the next generation of healthcare professionals. The anatomical sciences will need to be part of that challenge so they continue to play a full role in preparing students with the knowledge and ever increasingly the skills and competencies that will contribute to the fundamentals of their future capacity to practice effectively. Although there have been significant advances in anatomical science pedagogy, by reviewing learning and assessment in an apparently unrelated field, provides an opportunity to bring a different perspective and enable appropriate challenge of the current approaches in anatomy. Design learning has had to continually reimagine itself in response to the shifting landscape in design practice and the threats associated with technology and societal change. Design learning has also long used a student-centric active pedagogy and allied authentic assessment methods and, therefore, provides an ideal case study to help inform future changes required in anatomical learning and assessment.

The design and implementation of a longitudinal social medicine curriculum at the University of Vermont's Larner College of Medicine

BACKGROUND

Despite an abundant literature advocating that social determinants of health (SDH) be taught during undergraduate medical education, there are few detailed descriptions of how to design and implement longitudinal core curricula that is delivered to all students and accomplishes this goal.

METHODS

In this paper, we describe the design and implementation of a social medicine curriculum at the University of Vermont's Larner College of Medicine (UVM Larner). Using Kern's principles, we designed a longitudinal curriculum that extends through both preclinical and clinical training for all students and focused on integrating SDH material directly into basic science and clinical training.

RESULTS

We successfully developed and implemented two primary tools, a "Social Medicine Theme of the Week" (SMTW) in preclinical training, and SDH rounds in the clinical setting to deliver SDH content to all learners at UVM Larner.

CONCLUSIONS

Extensive student-faculty partnerships, robust needs assessment, and focusing on longitudinal and integrated SDH content delivery to all students were key features that contributed to successful design and implementation.