Summative assessments are more powerful drivers of student learning than resource intensive teaching formats

Managing assessment during curriculum change: Ottawa Consensus Statement

Curriculum change is relatively frequent in health professional education. Formal, planned curriculum review must be conducted periodically to incorporate new knowledge and skills, changing teaching and learning methods or changing roles and expectations of graduates. Unplanned curriculum evolution arguably happens continually, usually taking the form of "minor" changes that in combination over time may produce a substantially different programme. However, reviewing assessment practices is less likely to be a major consideration during curriculum change, overlooking the potential for unintended consequences for learning. This includes potentially undermining or negating the impact of even well-designed and important curriculum changes. Changes to any component of the curriculum "ecosystem "- graduate outcomes, content, delivery or assessment of learning - should trigger an automatic review of the whole ecosystem to maintain constructive alignment. Consideration of potential impact on assessment is essential to support curriculum change. Powerful contextual drivers of a curriculum include national examinations and programme accreditation, so each assessment programme sits within its own external context. Internal drivers are also important, such as adoption of new learning technologies and learning preferences of students and faculty. Achieving optimal and sustainable outcomes from a curriculum review requires strong governance and support, stakeholder engagement, curriculum and assessment expertise and internal quality assurance processes. This consensus paper provides guidance on managing assessment during curriculum change, building on evidence and the contributions of previous consensus papers.

What guides student learning in the clinical years: A mixed methods study exploring study behaviours prior to the UK Medical Licensing Assessment (UKMLA)

PURPOSE

Student study behaviours that prioritise the UKMLA content map over the local curriculum are a significant risk for UK medical education. To mitigate this, we describe a student-centred faculty process to improve local curriculum guidance based on an evaluation of student study behaviours, concerns and needs. Responses informed the build of an online curriculum map.

METHODS

A mixed methods approach was adopted, including an online anonymous survey exploring student study behaviours and preferences for curricular guidance. This was followed by student-led focus groups to explore emergent themes further. Qualitative data underwent reflexive thematic analysis.

RESULTS

121 students responded to the survey, of which 12 consented to participate in two student-led focus groups. Five key themes emerged, including motivation for learning, student use of the intended curriculum, student experience of the enacted curriculum, the hidden curriculum, and expectations of an online curriculum map.

CONCLUSIONS

A participatory framework enabled shared aims and responsive outcomes for curricular development in the run up to the UKMLA. Student responses led to clarification of guidance, reorganisation of learning resources and optimal design of an online curriculum map which linked all content in a visible, UKMLA aligned framework, accessible to all students and teachers.

Knowledge Transfer and Networking Upon Implementation of a Transdisciplinary Digital Health Curriculum in a Unique Digital Health Training Culture: Prospective Analysis

Background

Digital health has been taught at medical faculties for a few years. However, in general, the teaching of digital competencies in medical education and training is still underrepresented.

Objective

This study aims to analyze the objective acquisition of digital competencies through the implementation of a transdisciplinary digital health curriculum as a compulsory elective subject at a German university. The main subject areas of digital leadership and management, digital learning and didactics, digital communication, robotics, and generative artificial intelligence were developed and taught in a transdisciplinary manner over a period of 1 semester.

Methods

The participants evaluated the relevant content of the curriculum regarding the competencies already taught in advance during the study, using a Likert scale. The participants' increase in digital competencies were examined with a pre-post test consisting of 12 questions. Statistical analysis was performed using an unpaired 2-tailed Student t test. A P value of <.05 was considered statistically significant. Furthermore, an analysis of the acceptance of the transdisciplinary approach as well as the application of an alternative examination method (term paper instead of a test with closed and open questions) was carried out.

Results

In the first year after the introduction of the compulsory elective subject, students of human medicine (n=15), dentistry (n=3), and medical biotechnology (n=2) participated in the curriculum. In total, 13 participants were women (7 men), and 61.1% (n=11) of the participants in human medicine and dentistry were in the preclinical study stage (clinical: n=7, 38.9%). All the aforementioned learning objectives were largely absent in all study sections (preclinical: mean 4.2; clinical: mean 4.4; P=.02). The pre-post test comparison revealed a significant increase of 106% in knowledge (P<.001) among the participants.

Conclusions

The transdisciplinary teaching of a digital health curriculum, including digital teaching methods, considers perspectives and skills from different disciplines. Our new curriculum facilitates an objective increase in knowledge regarding the complex challenges of the digital transformation of our health care system. Of the 16 student term papers arising from the course, robotics and artificial intelligence attracted the most interest, accounting for 9 of the submissions.

Reflecting on gamified learning in medical education: a systematic literature review grounded in the Structure of Observed Learning Outcomes (SOLO) taxonomy 2012-2022

BACKGROUND

The acquisition of in-depth medical knowledge, skills, and competencies is of utmost importance when training future medical professionals. This systematic literature review delves into the empirical connection between gamified learning in medical education and the Structure of Observed Learning Outcomes (SOLO) taxonomy.

METHODS

Following PRISMA guidelines, a systematic literature review was conducted in seven databases to identify empirical studies related to gamification and medical education. The literature search was limited to peer-reviewed articles published between January 2012 and December 2022. Articles focusing on games or learning technologies in a broader sense as well as research focusing on nursing or pharmacy education were excluded.

RESULTS

Upon reviewing 23 qualified empirical studies that applied gamified learning strategies in medical education, 18 (78%) studies are associated with the second lowest level (uni-structural) of the SOLO taxonomy. The mid-level (multi-structural) learning outcomes are associated with three (13%) of the reviewed studies. There are five (22%) studies focusing on the second highest (relational) level of the SOLO. Only one study (4%) is associated with the highest SOLO level (extended-abstract). Finally, three (13%) studies were identified with multiple levels of the SOLO. In addition to the SOLO levels, the review found six (26%) studies emphasizing motivational and engagement support of gamified learning strategies in facilitating intended learning outcome attainment. A total of three (13%) studies, across three SOLO levels, suggested that gamified learning strategies can improve students' soft skills in medical education programs.

CONCLUSION

These findings collectively emphasize the need for future research and development to design gamified learning experiences capable of fostering higher SOLO taxonomy attainment in medical education. Moreover, there is potential to extend the SOLO framework to encompass motivational and affective learning outcomes, providing a comprehensive understanding of the impact of gamified learning on medical students.

Assessment of professional identity formation: a transcultural validation of the professional identity essay for brazilian portuguese

INTRODUCTION

Professional identity formation (PIF) is recognized worldwide as an outcome of medical education grounded in the psychology of adult development and the literature on medical professionalism. However, instruments to assess and support PIF are scarce. The Professional Identity Essay (PIE) is an open-ended question assessment of PIF that elicits short narrative responses from learners and that can be analyzed to provide formative feedback and an overall stage of development. In this study, our aim was to translate and adapt the PIE to Brazilian Portuguese.

METHODS

We followed a systematic procedure for the translation and cross-cultural adaptation of the instrument. A pilot study was conducted with medical students from the University of São Paulo. After providing individual formative feedback, we administered an online questionnaire to the Brazilian students to better understand the consequences of using the PIE. Content analyses of qualitative data were performed, we employ manifest content analysis, and the categories of analysis emerged from the participants' speeches.

RESULTS

Students found the instrument's questions easy to interpret and self-reflective. It also gave students the opportunity to consider their PIF. The PIE was perceived as reliable and brought more awareness of the students' own processes in addition to a sense of capability to foster their own development. In the same way, the students emphasized the importance of being helped in this process.

CONCLUSION

We found sufficient evidence of the validity of the PIE in terms of content, face validity, and consequences of use. The PIE enhances self-assurance in PIF through formative assessment and is sensitive to different cultures, making it a potential tool for educators.

Motivational simulated teaching of clinical skills using formative assessment methods for medical undergraduate students: between-group evaluation of a simulated course in a Chinese medical college

OBJECTIVES

Teaching clinical skills is an important component of educational programmes for medical undergraduates. However, the extension of the interval between the completion of the course and qualification examination affects the performance of students in the skill examination. This study established a multisource evaluation system to determine whether formative assessment can enhance the instruction of clinical skills.

METHODS

Formative assessment was introduced to the entire training course on clinical skills, in which diversified methods were used to observe the performance of students during training. Students in the experimental group received training for clinical skills using formative assessment (class of 2019, n=128), while students in the control group received traditional training without formative assessment (class of 2018, n=123). Both groups participated in the Objective Structured Clinical Examination (OSCE) conducted by Tongji Medical College, and the exam scores were taken as the objective measure of course outcome. After completing the course, all students in the experimental group were instructed to fill in a questionnaire to evaluate their experience in the training programme, as a subjective measure of course outcome.

RESULTS

Compared with the control group, students in the experimental group received significantly better practical scores in the four clinical skills tested by the OSCE. The questionnaire results revealed that the majority of students who were trained using formative assessment methods considered the course helpful for learning, and appreciated the course for the clinical skills they had gained, and the opportunity to receive and give feedback to the instructors.

CONCLUSIONS

The findings of this study suggest that formative assessment methods are beneficial for learning clinical skills through simulated teaching, as shown by the improved objective clinical skills evaluated by the structured clinical examination, and the self-reported satisfaction with the learning process.

Protocol for development of an assessment tool for competency of ECG interpretation: expert consensus by the RAND/UCLA appropriateness method and cross-sectional testing using multidimensional item response theory

INTRODUCTION

Although the ECG is an important diagnostic tool in medical practice, the competency of ECG interpretation is considered to be poor. Diagnostic inaccuracy involving the misinterpretation of ECG can lead to inappropriate medical judgements and cause negative clinical outcomes, unnecessary medical testing and even fatalities. Despite the importance of assessing ECG interpretation skills, there is currently no established universal, standardised assessment tool for ECG interpretation. The current study seeks to (1) develop a set of items (ECG questions) for estimating competency of ECG interpretation by medical personnel by consensus among expert panels following a process based on the RAND/UCLA Appropriateness Method (RAM) and (2) analyse item parameters and multidimensional latent factors of the test set to develop an assessment tool.

METHODS AND ANALYSIS

This study will be conducted in two steps: (1) selection of question items for ECG interpretation assessment by expert panels via a consensus process following RAM and (2) cross-sectional, web-based testing using a set of ECG questions. A multidisciplinary panel of experts will evaluate the answers and appropriateness and select 50 questions as the next step. Based on data collected from a predicted sample size of 438 test participants recruited from physicians, nurses, medical and nursing students, and other healthcare professionals, we plan to statistically analyse item parameters and participant performance using multidimensional item response theory. Additionally, we will attempt to detect possible latent factors in the competency of ECG interpretation. A test set of question items for ECG interpretation will be proposed on the basis of the extracted parameters.

ETHICS AND DISSEMINATION

The protocol of this study was approved by the Institutional Review Board of Ehime University Graduate School of Medicine (IRB number: 2209008). We will obtain informed consent from all participants. The findings will be submitted for publication in peer-reviewed journals.

Surfing the clinical trials of ECG teaching to medical students: A systematic review

Interpreting an electrocardiogram (ECG) is crucial for every physician. The physician's competency in ECG interpretation needs to be improved at any stage of medical education. The aim of the present study was to review the currently published clinical trials of ECG teaching to medical students and provide suggestions for future works. On May 1, 2022, PubMed, Scopus, Web of Science, Google Scholar, and ERIC were searched to retrieve relevant articles on clinical trials of ECG teaching to medical students. The quality of the included studies was assessed utilizing the Buckley et al. criteria. The screening, data extraction, and quality appraisal processes were duplicated independently. In case of disagreements, consultation with a third author was put forth. In total, 861 citations were found in the databases. After screening abstracts and full texts, 23 studies were deemed eligible. The majority of the studies were of good quality. Peer teaching (7 studies), self-directed learning (6 studies), web-based learning (10 studies), and various assessment modalities (3 studies) comprised the key themes of the studies. Various methods of ECG teaching were encountered in the reviewed studies. Future studies in ECG training should focus on novel and creative teaching methods, the extent to which self-directed learning can be effective, the utility of peer teaching, and the implications of computer-assisted ECG interpretation (e.g., artificial intelligence) for medical students. Long-term knowledge retention assessment studies based on different approaches integrated with clinical outcomes could be beneficial in determining the most efficient modalities.

Examination performance of dentistry students during the COVID-19 pandemic

CONCLUSIONS

Our results imply that online teaching did not systematically improve or worsen the examination performance of undergraduate students of dentistry. Our findings have important policy implications for educational practices in the future. The observed annual grade variation might reflect the effectiveness of remote teaching, changes in students' performance, or non-systematic grading.

MATERIALS AND METHODS

Using administrative data from the University of Helsinki, the examination performance during the years 2018 and 2019 was compared with that of 2020.

OBJECTIVE

The global COVID-19 pandemic has led to an increase in remote teaching and online assessment in higher education. The examination performance of undergraduate students of dentistry was evaluated to assess any possible association between the altered learning environment and learners' performance.

RESULTS

In 16 out of 22 courses (73%) taught remotely during the pandemic, a statistically significant difference (p < .05) between the awarded grades was observed, as compared to the two previous years. Annual variation in examination grades was common even before the onset of remote teaching, but more so after it. The grade variation during the pandemic was nearly as prevalent towards falling grades (26%) as towards rising grades (36%), implying a multifactorial cause possibly unrelated to remote teaching.

Assessment of the competence in electrocardiographic interpretation among Arabic resident doctors at the emergency medicine and internal medicine departments: A multi-center online cross-sectional study

Background

This study aims to assess the electrocardiographic interpretation abilities of resident doctors at internal medicine and emergency medicine departments in eight Arabic countries.

Methods

An online cross-sectional study was conducted between October 7, 2022 and October 21, 2022 in eight Arabic countries. The questionnaire consisted of two main sections: the first section included sociodemographic information, while the second section contained 12 clinical case questions of the most severe cardiac abnormalities with their electrocardiography (ECG) recordings.

Results

Out of 2,509 responses, 630 were eligible for the data analysis. More than half of the participants were males (52.4%). Internal medicine residents were (n = 530, 84.1%), whereas emergency medicine residents were (n = 100, 15.9%). Almost participants were in their first or second years of residency (79.8%). Only 36.2% of the inquired resident doctors had attended an ECG course. Most participants, 85.6%, recognized the ECG wave order correctly, and 50.5% of the participants scored above 7.5/10 on the ECG interpretation scale. The proportions of participants who were properly diagnosed with atrial fibrillation, third-degree heart block, and atrial tachycardia were 71.1, 76.7, and 56.6%, respectively. No statistically significant difference was defined between the internal and emergency medicine residents regarding their knowledge of ECG interpretation (p value = 0.42). However, there was a significant correlation between ECG interpretation and medical residency year (p value < 0.001); the fourth-year resident doctors had the highest scores (mean = 9.24, SD = 1.6). As well, participants in the third and second years of postgraduate medical residency have a probability of adequate knowledge of ECG interpretation more than participants in the first year of residency (OR = 2.1, p value = 0.001) and (OR = 1.88, p value = 0.002), respectively.

Conclusion

According to our research findings, resident doctors in departments of internal medicine and emergency medicine in Arabic nations have adequate ECG interpretation abilities; nevertheless, additional development is required to avoid misconceptions about critical cardiac conditions.

Influence of using simulated or real patients on undergraduate medical students acquiring competencies in medical conversations in surgery: A prospective, controlled study

Background

Communication with patients and their relatives as well as with colleagues and students is an essential part of every physician's daily work. An established method for teaching communication skills is using simulated patients (SPs). However, teaching with SPs is often subjectively perceived by medical students as less instructive than teaching with real patients (RPs). Studies that analyze the influence of SPs compared to RPs for acquiring competencies are lacking. The aim of the present study was therefore to investigate the impact of SPs on long-term learning success for communication skills compared to RPs.

Material and Methods

Study participants were undergraduate third-year medical students who attended a communication unit and were randomized into three groups. The first group trained the role-play part with a SP (SP-group). The second group trained with a SP but thought that the patient was a RP because the students and the tutors were told that they were a RP by the principal investigator (incognito patient group [IP-group]). The third group and their tutors trained with a RP and were told that the patient was a RP (real patient group [RP-group]). Five to 12 weeks after completing the training, the study participants completed a curricular summative objective standardized clinical examination.

Results

There were 146 students who participated in the study. There were no significant differences between the three study groups at the informed consent stations and for those conducting anamnesis interviews.

Conclusion

Communication skills training with SPs appears to be equivalent to training with RPs in terms of competency development in communication-based assessments in surgery. Therefore, SPs should be used in these curricula, especially at an early stage, to enable the students to practice adequate communication skills.

Quality Assurance of Undergraduate Medical Education in Israel by Continuous Monitoring and Prioritization of the Accreditation Standards

External accreditation reviews of undergraduate medical curricula play an important role in their quality assurance. However, these reviews occur only at 4-10-year intervals and are not optimal for the immediate identification of problems related to teaching. Therefore, the Standards of Medical Education in Israel require medical schools to engage in continuous, ongoing monitoring of their teaching programs for compliance with accreditation standards. In this paper, we propose the following: (1) this monitoring be assigned to independent medical education units (MEUs), rather than to an infrastructure of the dean's office, and such MEUs to be part of the school governance and draw their authority from university institutions; and (2) the differences in the importance of the accreditation standards be addressed by discerning between the "most important" standards that have been shown to improve student well-being and/or patient health outcomes; "important" standards associated with student learning and/or performance; "possibly important" standards with face validity or conflicting evidence for validity; and "least important" standards that may lead to undesirable consequences. According to this proposal, MEUs will evolve into entities dedicated to ongoing monitoring of the education program for compliance with accreditation standards, with an authority to implement interventions. Hopefully, this will provide MEUs and faculty with the common purpose of meeting accreditation requirements, and an agreed-upon prioritization of accreditation standards will improve their communication and recommendations to faculty.

Randomised controlled monocentric trial to compare the impact of using professional actors or peers for communication training in a competency-based inverted biochemistry classroom in preclinical medical education

OBJECTIVE

In medical education, biochemistry topics are usually knowledge based, and students often are unaware of their clinical relevance. To improve students' awareness of the relevance, we integrated communication skills training into biochemistry education. No studies before have examined the difference between peer and standardised patient (SP) role plays where students explain the biochemical background of a disease in patient-centred language. Therefore, we evaluated whether students' self-perceived competency in Canadian Medical Education Directives for Specialists (CanMEDS) roles and their opinion of the quality of role play differ if the layperson is played by peers or SPs.

METHODS

We randomly assigned medical students in a preclinical semester to one of the two groups. The groups used predefined scripts to role play a physician-parent consultation with either a peer (peer group) or an SP (SP group) in the parent role. Students then assessed the activity's effects on their competency in CanMEDS roles and motivation and the relevance of the role play. To determine whether students achieved biochemistry learning goals, we evaluated results of a biochemistry exam.

RESULTS

Students' self-perceived competency improved in both groups. The SP group rated their competency in the roles 'Scholar' and 'Professional' significantly higher than the peer group did. The peer group rated their competency in the role of 'Medical Expert' significantly higher if they played the role of the parent rather than physician or observer. The SP group agreed more that they were motivated by the role play and wanted to receive more role play-based teaching. The SP group perceived the role play as being realistic and rated the feedback discussion as more beneficial. The examination results were the same in both groups.

CONCLUSION

We showed that role plays in a biochemistry seminar improve students' self-perceived competency. The use of SPs has some advantages, such as being more realistic.

Use of a Serious Game to Teach Infectious Disease Management in Medical School: Effectiveness and Transfer to a Clinical Examination

Purpose

Physicians of all specialties must be familiar with important principles of infectious diseases, but curricular time for this content is limited and clinical teaching requires considerable resources in terms of available patients and teachers. Serious games are scalable interventions that can help standardize teaching. This study assessed whether knowledge and skills acquired in a serious game translate to better performance in a clinical examination.

Methods

Fifth-year undergraduate medical students (n = 100) at Goettingen Medical School were randomized to three groups receiving different levels of exposure to virtual patients presenting with signs and symptoms of either infective endocarditis or community-acquired pneumonia in a serious game simulating an accident and emergency department. Student performance was assessed based on game logfiles and an objective standardized clinical examination (OSCE).

Results

Higher exposure to virtual patients in the serious game did not result in superior OSCE scores. However, there was good agreement between student performance in the OSCE and in game logfiles (r = 0.477, p = 0.005). An Item Response Theory analysis suggested that items from the serious game covered a wider range of ability, thus better differentiating between students within a given cohort.

Conclusion

Repeated exposure to virtual patients with infectious diseases in a serious game did not directly impact on exam performance but game logfiles might be good and resource-sparing indicators of student ability. One advantage of using serious games in medical education is standardized content, a lower inhibition threshold to learn, and a need of less staff time compared to small-group clinical teaching.

Physician Training for Electrocardiogram Interpretation: A Systematic Review and Meta-Analysis

PURPOSE

Using electrocardiogram (ECG) interpretation as an example of a widely taught diagnostic skill, the authors conducted a systematic review and meta-analysis to demonstrate how research evidence on instruction in diagnosis can be synthesized to facilitate improvement of educational activities (instructional modalities, instructional methods, and interpretation approaches), guide the content and specificity of such activities, and provide direction for research.

METHOD

The authors searched PubMed/MEDLINE, Embase, Cochrane CENTRAL, PsycInfo, CINAHL, ERIC, and Web of Science databases through February 21, 2020, for empirical investigations of ECG interpretation training enrolling medical students, residents, or practicing physicians. They appraised study quality with the Medical Education Research Study Quality Instrument and pooled standardized mean differences (SMDs) using random effects meta-analysis.

RESULTS

Of 1,002 articles identified, 59 were included (enrolling 17,251 participants). Among 10 studies comparing instructional modalities, 8 compared computer-assisted and face-to-face instruction, with pooled SMD 0.23 (95% CI, 0.09, 0.36) indicating a small, statistically significant difference favoring computer-assisted instruction. Among 19 studies comparing instructional methods, 5 evaluated individual versus group training (pooled SMD -0.35 favoring group study [95% CI, -0.06, -0.63]), 4 evaluated peer-led versus faculty-led instruction (pooled SMD 0.38 favoring peer instruction [95% CI, 0.01, 0.74]), and 4 evaluated contrasting ECG features (e.g., QRS width) from 2 or more diagnostic categories versus routine examination of features within a single ECG or diagnosis (pooled SMD 0.23 not significantly favoring contrasting features [95% CI, -0.30, 0.76]). Eight studies compared ECG interpretation approaches, with pooled SMD 0.92 (95% CI, 0.48, 1.37) indicating a large, statistically significant effect favoring more systematic interpretation approaches.

CONCLUSIONS

Some instructional interventions appear to improve learning in ECG interpretation; however, many evidence-based instructional strategies are insufficiently investigated. The findings may have implications for future research and design of training to improve skills in ECG interpretation and other types of visual diagnosis.

Assessments of Physicians' Electrocardiogram Interpretation Skill: A Systematic Review

PURPOSE

To identify features of instruments, test procedures, study design, and validity evidence in published studies of electrocardiogram (ECG) skill assessments.

METHOD

The authors conducted a systematic review, searching MEDLINE, Embase, Cochrane CENTRAL, PsycINFO, CINAHL, ERIC, and Web of Science databases in February 2020 for studies that assessed the ECG interpretation skill of physicians or medical students. Two authors independently screened articles for inclusion and extracted information on test features, study design, risk of bias, and validity evidence.

RESULTS

The authors found 85 eligible studies. Participants included medical students (42 studies), postgraduate physicians (48 studies), and practicing physicians (13 studies). ECG selection criteria were infrequently reported: 25 studies (29%) selected single-diagnosis or straightforward ECGs; 5 (6%) selected complex cases. ECGs were selected by generalists (15 studies [18%]), cardiologists (10 studies [12%]), or unspecified experts (4 studies [5%]). The median number of ECGs per test was 10. The scoring rubric was defined by 2 or more experts in 32 studies (38%), by 1 expert in 5 (6%), and using clinical data in 5 (6%). Scoring was performed by a human rater in 34 studies (40%) and by computer in 7 (8%). Study methods were appraised as low risk of selection bias in 16 studies (19%), participant flow bias in 59 (69%), instrument conduct and scoring bias in 20 (24%), and applicability problems in 56 (66%). Evidence of test score validity was reported infrequently, namely evidence of content (39 studies [46%]), internal structure (11 [13%]), relations with other variables (10 [12%]), response process (2 [2%]), and consequences (3 [4%]).

CONCLUSIONS

ECG interpretation skill assessments consist of idiosyncratic instruments that are too short, composed of items of obscure provenance, with incompletely specified answers, graded by individuals with underreported credentials, yielding scores with limited interpretability. The authors suggest several best practices.

Residency program directors' perceptions about the impact of the American Board of Anesthesiology's Objective Structured Clinical Examination

STUDY OBJECTIVE

To describe how the introduction of an Objective Structured Clinical Examination (OSCE) by the American Board of Anesthesiology (ABA) to its initial certification impacted anesthesiology residencies in the United States.

DESIGN AND SETTING

A sequential mixed-methods design with focus groups and online survey among program directors of Accreditation Council for Graduate Medical Education-accredited anesthesiology residencies.

PATIENTS

No patients were included.

INTERVENTION

None.

MEASUREMENTS

A convenience sample of 34 program directors were interviewed to understand their perceptions of the ABA OSCE. Subsequently, an online survey, based on major themes identified from the focus groups, was sent to all 156 program directors.

MAIN RESULTS

Several themes emerged from the focus group discussions: (1) a mock OSCE was most common for preparing residents for the ABA OSCE; 2) the ABA OSCE led to changes in residency curriculum; 3) the ABA OSCE assessed communication and professionalism skills well, and how well it assessed technical skills was less agreed on. Survey results from 87 program directors (response rate = 56%) were mostly consistent with the themes generated by the focus groups. Eight-one out of 87 programs (93%) specifically prepared their residents for the ABA OSCE. Fifty-two out of 81 program directors (64%) reported the introduction of the ABA OSCE led to curricular changes. Out of 79 program directors, 45 (57%) agreed the ABA OSCE assesses skills essential to anesthesiology practice, and 40 (51%) considered it added value to board certification.

CONCLUSIONS

The introduction of the OSCE by the ABA for board certification has affected the curriculum of many residencies. Approximately 3 in 5 program directors perceived the ABA OSCE measures skills essential to anesthesiologists' practice. Future studies should assess residency graduates' perspective on the usefulness of both mock OSCE preparation and the ABA OSCE, and whether the ABA OSCE performance predicts future clinical practice.

Effectiveness of a serious game addressing guideline adherence: cohort study with 1.5-year follow-up

BACKGROUND

Patients presenting with acute shortness of breath and chest pain should be managed according to guideline recommendations. Serious games can be used to train clinical reasoning. However, only few studies have used outcomes beyond student satisfaction, and most of the published evidence is based on short-term follow-up. This study investigated the effectiveness of a digital simulation of an emergency ward regarding appropriate clinical decision-making.

METHODS

In this prospective trial that ran from summer 2017 to winter 2018/19 at Göttingen Medical University Centre, a total of 178 students enrolled in either the fourth or the fifth year of undergraduate medical education took six 90-min sessions of playing a serious game ('training phase') in which they managed virtual patients presenting with various conditions. Learning outcome was assessed by analysing log-files of in-game activity (including choice of diagnostic methods, differential diagnosis and treatment initiation) with regard to history taking and patient management in three virtual patient cases: Non-ST segment elevation myocardial infarction (NSTEMI), pulmonary embolism (PE) and hypertensive crisis. Fourth-year students were followed up for 1.5 years, and their final performance was compared to the performance of students who had never been exposed to the game but had otherwise taken the same five-year undergraduate course.

RESULTS

During the training phase, overall performance scores increased from 57.6 ± 1.1% to 65.5 ± 1.2% (p < 0.001; effect size 0.656). Performance remained stable over 1.5 years, and the final assessment revealed a strong impact of ever-exposure to the game on management scores (72.6 ± 1.2% vs. 63.5 ± 2.1%, p < 0.001; effect size 0.811). Pre-exposed students were more than twice as likely to correctly diagnose NSTEMI and PE and showed significantly greater adherence to guideline recommendations (e.g., troponin measurement and D-dimer testing in suspected PE).

CONCLUSIONS

The considerable difference observed between previously exposed and unexposed students suggests a long-term effect of using the game although retention of specific virtual patient cases rather than general principles might partially account for this effect. Thus, the game may foster the implementation of guideline recommendations.

Comparison of 'Mental training' and physical practice in the mediation of a structured facial examination: a quasi randomized, blinded and controlled study

BACKGROUND

The correct performance of a structured facial examination presents a fundamental clinical skill to detect facial pathologies. However, many students are not adequately prepared in this basic clinical skill. Many argue that the traditional 'See One, Do One' approach is not sufficient to fully master a clinical skill. 'Mental Training' has successfully been used to train psychomotor and technical skills in sports and other surgical fields, but its use in Oral and Maxillofacial Surgery is not described. We conducted a quasi-experimental to determine if 'Mental Training' was effective in teaching a structured facial examination.

METHODS

Sixty-seven students were randomly assigned to a 'Mental Training' and 'See One, Do One' group. Both groups received standardized video instruction on how to perform a structured facial examination. The 'See One, Do One' group then received 60 min of guided physical practice while the 'Mental Training' group actively developed a detailed, stepwise sequence of the performance of a structured facial examination and visualized this sequence subvocally before practicing the skill. Student performance was measured shortly after (T1) and five to 10 weeks (T2) after the training by two blinded examiners (E1 and E2) using a validated checklist.

RESULTS

Groups did not differ in gender, age or in experience. The 'Mental Training' group averaged significantly more points in T1 (pE1 = 0.00012; pE2 = 0.004; dE1 = 0.86; dE2 = 0.66) and T2 (pE1 = 0.04; pE2 = 0.008, dE1 = 0.37; dE2 = 0.64) than the 'See One, Do One' group. The intragroup comparison showed a significant (pE1 = 0.0002; pE2 = 0.06, dE1 = 1.07; dE2 = 0.50) increase in clinical examination skills in the 'See One, Do One' group, while the 'Mental Training' group maintained an already high level of clinical examination skills between T1 and T2.

DISCUSSION

'Mental Training' is an efficient tool to teach and maintain basic clinical skills. In this study 'Mental Training' was shown to be superior to the commonly used 'See One, Do One' approach in learning how to perform a structured facial examination and should therefore be considered more often to teach physical examination skills.

[Rheumatology in German MD curricula]

Transmitting a substantial amount of basic knowledge in Rheumatology to all medical students is essential for the future medical care of patients with rheumatic diseases for two reasons: on the one hand, future general practitioners will need to master the patterns of rheumatic diseases to recognize them fast enough in new-onset patients and to refer them in time and directly to rheumatologists. On the other hand, the shortage of rheumatologists can only then be relieved in the future when we are able to inspire enthusiasm for our specialty. Adequate rheumatological structures are established only in some of the German faculties of medicine. Structural improvements happen in small steps only but were achieved at several sites. The better the local structures, the higher the chances of committed university teachers in rheumatology to reach all medical students. Probably from 2026 onwards, the learning objectives relevant for examinations will be defined by the national competence-based catalogue of learning objectives in medicine (NKLM), which is currently in the final stages of completion together with the German Federal Institute for Medical and Pharmaceutical Examinations (IMPP). It now appears that systemic autoimmune diseases and inflammatory rheumatic diseases are adequately depicted in this catalogue. If this is achieved, students will know more about these diseases in the future and will diagnose them faster in patients. Work on the NKLM is therefore of highest importance. In addition to the work on the learning objectives, up to date learning materials are required, which have to be available throughout Germany. A Rheumatology script just finished by the committee for medical student education of the German Society of Rheumatology (DGRh) and now available on the DGRh homepage should close this gap.

Accuracy of Physicians' Electrocardiogram Interpretations: A Systematic Review and Meta-analysis

IMPORTANCE

The electrocardiogram (ECG) is the most common cardiovascular diagnostic test. Physicians' skill in ECG interpretation is incompletely understood.

OBJECTIVES

To identify and summarize published research on the accuracy of physicians' ECG interpretations.

DATA SOURCES

A search of PubMed/MEDLINE, Embase, Cochrane CENTRAL (Central Register of Controlled Trials), PsycINFO, CINAHL (Cumulative Index to Nursing and Allied Health), ERIC (Education Resources Information Center), and Web of Science was conducted for articles published from database inception to February 21, 2020.

STUDY SELECTION

Of 1138 articles initially identified, 78 studies that assessed the accuracy of physicians' or medical students' ECG interpretations in a test setting were selected.

DATA EXTRACTION AND SYNTHESIS

Data on study purpose, participants, assessment features, and outcomes were abstracted, and methodological quality was appraised with the Medical Education Research Study Quality Instrument. Results were pooled using random-effects meta-analysis.

MAIN OUTCOMES AND MEASURES

Accuracy of ECG interpretation.

RESULTS

Of 1138 studies initially identified, 78 assessed the accuracy of ECG interpretation. Across all training levels, the median accuracy was 54% (interquartile range [IQR], 40%-66%; n = 62 studies) on pretraining assessments and 67% (IQR, 55%-77%; n = 47 studies) on posttraining assessments. Accuracy varied widely across studies. The pooled accuracy for pretraining assessments was 42.0% (95% CI, 34.3%-49.6%; n = 24 studies; I2 = 99%) for medical students, 55.8% (95% CI, 48.1%-63.6%; n = 37 studies; I2 = 96%) for residents, 68.5% (95% CI, 57.6%-79.5%; n = 10 studies; I2 = 86%) for practicing physicians, and 74.9% (95% CI, 63.2%-86.7%; n = 8 studies; I2 = 22%) for cardiologists.

CONCLUSIONS AND RELEVANCE

Physicians at all training levels had deficiencies in ECG interpretation, even after educational interventions. Improved education across the practice continuum appears warranted. Wide variation in outcomes could reflect real differences in training or skill or differences in assessment design.

[Video presentations for examination questions in psychiatry : A pilot study]

BACKGROUND

The master plan for medical studies 2020 (MM2020) aims at a restructuring of medical education. The examinations should represent the key element of the evaluation and electronically supported examination methods will be expanded.

OBJECTIVE

Videos of actors displaying psychopathological phenomena were implemented into examination questions for the first time through the use of an electronically supported examination (e-examination) and therefore in accordance with the aims of MM2020.

METHODS

A total of four multiple choice (MC) questions referencing short video representations of different psychopathological phenomena were formulated and shown to medical students within an e‑examination. The MC questions were evaluated with respect to scientific quality criteria and the medical students were asked to fill in a questionnaire regarding their evaluation of the content and format of the MC questions after participating in the examination.

RESULTS

The MC video questions were presented to the students directly following the regular e‑examination of the teaching module in psychiatry. Out of 69 students 60 participated in the study. The MC video questions had a high level of acceptance and positive evaluation by the students. With respect to the scientific quality criteria, the MC questions showed heterogeneous results.

CONCLUSION

Due to the versatile applicability and high efficiency, MC questions will continue to play an important role in medical examinations. In accordance with the aims of MM2020, it could be shown that psychiatric knowledge can be examined in a competence-oriented manner and with a practical orientation by means of video presentations in an e‑examination. A broader application of video presentations could complement electronically supported examinations in the discipline of psychiatry.

Comparison of different feedback modalities for the training of procedural skills in Oral and maxillofacial surgery: a blinded, randomized and controlled study

BACKGROUND

The feedback given to students plays an important role in their efficiency related to learning practical skills. In the present study, diverse feedback modalities have been investigated. Our hypothesis is that individualized and unsupervised video feedback can produce a similar learning experience as performing practical skills in an oral and maxillofacial surgery setting with conventional direct expert feedback (control group).

METHODS

This prospective, randomized, controlled, and blinded study compared direct expert feedback (DEF), individualized video feedback (IVF) and unsupervised video feedback (UVF). The participants were fourth-year dental students from University Goethe in Frankfurt. The students were assigned to one of the three feedback methods (n = 20 per group) using simple randomization. All participants watched an instruction video for an interdental ('Ernst') ligature and periphery venous catheterization. Next, the students were video recorded performing the tasks by themselves (pre-test). Following this, every student received feedback using one of the above-mentioned feedback modalities. The participants then performed the same task again while being video recorded (post-test) to measure the acquired competence. Six weeks later, the students participated in an objective structured clinical examination (OSCE) to evaluate their long-term knowledge retention. All examiners were blinded regarding the students' instructional approach and their affiliation in terms of the learning group.

RESULTS

For the interdental ligature, we found significant improvements in performance in each feedback modality group between the pre-test and post-test (p < 0.001). UVF had the strongest effect on performance time. The comparison between each group in the post-test showed no significant differences between the three groups.

CONCLUSION

This study showed that IVF and UVF can be considered an alternative or adjunct to conventional methods (i.e. DEF) when learning procedural skills in oral and maxillofacial surgery. However, DEF showed to be the most effective method of feedback and therefore preferable in teaching.

Comparing video-based versions of Halsted's 'see one, do one' and Peyton's '4-step approach' for teaching surgical skills: a randomized controlled trial

BACKGROUND

Teaching complex motor skills at a high level remains a challenge in medical education. Established methods often involve large amounts of teaching time and material. The implementation of standardized videos in those methods might help save resources. In this study, video-based versions of Peyton's '4-step Approach' and Halsted's 'See One, Do One' are compared. We hypothesized that the video-based '4-step Approach' would be more effective in learning procedural skills than the 'See One, Do One Approach'.

METHODS

One-hundred-two naïve students were trained to perform a structured facial examination and a Bellocq's tamponade with either Halsted's (n = 57) or Peyton's (n = 45) method within a curricular course. Steps 1 (Halsted) and 1-3 (Peyton) were replaced by standardized teaching videos. The performance was measured directly (T1) and 8 weeks (T2) after the intervention by blinded examiners using structured checklists. An item-analysis was also carried out.

RESULTS

At T1, performance scores significantly differed in favor of the video-based '4-step Approach' (p < 0.01) for both skills. No differences were found at T2 (p < 0.362). The item-analysis revealed that Peyton's method was significantly more effective in the complex subparts of both skills.

CONCLUSIONS

The modified video-based version of Peyton's '4-step Approach' is the preferred method for teaching especially complex motor skills in a large curricular scale. Furthermore, an effective way to utilize Peyton's method in a group setting could be demonstrated. Further studies have to investigate the long-term learning retention of this method in a formative setting.

Introducing Summative Progress Testing in Radiology Residency: Little Change in Residents' Test Results After Transitioning from Formative Progress Testing

INTRODUCTION

Educational effects of transitioning from formative to summative progress testing are unclear. Our purpose was to investigate whether such transitioning in radiology residency is associated with a change in progress test results.

METHODS

We investigated a national cohort of radiology residents (N > 300) who were semi-annually assessed through a mandatory progress test. Until 2014, this test was purely formative for all residents, but in 2014/2015, it was transitioned (as part of a national radiology residency program revision) to include a summative pass requirement for new residents. In 7 posttransitioning tests in 2015-2019, including summatively and formatively tested residents who followed the revised and pre-transitioning residency program, respectively, we assessed residents' relative test scores and percentage of residents that reached pass standards.

RESULTS

Due to our educational setting, most posttransitioning tests had no residents in the summative condition in postgraduate year 4-5, nor residents in the formative condition in year 0.5-2. Across the 7 tests, relative test scores in postgraduate year 1-3 of the summative resident group and year 3.5-4.5 of the formative group differed significantly (p < 0.01 and p < 0.05, respectively, Kruskal-Wallis test). However, scores fluctuated without consistent time trends and without consistent differences between both resident groups. Percentage of residents reaching the pass standard did not differ significantly across tests or between groups.

DISCUSSION

Transitioning from formative to summative progress testing was associated with overall steady test results of the whole resident group in 4 post-transitioning years. We do not exclude that transitioning may have positive educational effects for resident subgroups.

Effectiveness of an innovative and interactive smoking cessation training module for dental students: A prospective study

BACKGROUND

Smoking is one of the world's major health problems and dental professionals are in a unique position to promote smoking cessation. However, according to the current literature, neither dental students nor dentists feel adequately prepared to counsel smokers. The purpose of this study was to develop and implement a teaching intervention on smoking cessation for fourth-year dental students and assess its effectiveness in terms of learning outcome on knowledge, communication skills and attitudes.

MATERIALS AND METHODS

In this prospective intervention study, students in the intervention group (n = 28) participated in a teaching module consisting of a podcast, an interactive lecture, a seminar, and small-group sessions with role-play interactions. Knowledge, communication skills and attitudes were measured using written examinations and an objective structured clinical examination (OSCE) at the end of the module and 6 months later. Results were compared with data from a historical control group (n = 27) receiving standard teaching.

RESULTS

Compared with the control group, students in the intervention group had higher scores in the knowledge test (67.1% vs 41.8%; P < .001; d = 2.8) as well as in the OSCE (74.9% vs 44.7%; P < .001; d = 2.3) and also retained more knowledge (52.7% vs 36.5%; P < .001; d = 2.0) and skills (71.8% vs 47.6%; P < .001; d = 2.5) over a period of 6 months. Attitudes were similar across groups and time-points.

CONCLUSION

The teaching intervention equipped dental students with specific knowledge and skills required to effectively counsel smoking patients. Further research is required to assess the transfer of these skills to the clinical setting.

Virtual patients versus small-group teaching in the training of oral and maxillofacial surgery: a randomized controlled trial

BACKGROUND

Computerized virtual patients (VP) have spread into many areas of healthcare delivery and medical education. They provide various advantages like flexibility in pace and space of learning, a high degree of teaching reproducibility and a cost effectiveness. However, the educational benefit of VP as an additive or also as an alternative to traditional teaching formats remains unclear. Moreover, there are no randomized-controlled studies that investigated the use of VP in a dental curriculum. Therefore, this study investigates VP as an alternative to lecturer-led small-group teaching in a curricular, randomized and controlled setting.

METHODS

Randomized and controlled cohort study. Four VP cases were created according to previously published design principles and compared with lecturer-led small group teaching (SGT) within the Oral and Maxillofacial Surgery clerkship for dental students at the Department for Cranio-, Oral and Maxillofacial Plastic Surgery, Goethe University, Frankfurt, Germany. Clinical competence was measured prior (T0), directly (T1) and 6 weeks (T2) after the intervention using theoretical tests and a self-assessment questionnaire. Furthermore, VP design was evaluated using a validated toolkit.

RESULTS

Fifty-seven students (VP = 32; SGT = 25) agreed to participate in the study. No competence differences were found at T0 (p = 0.56). The VP group outperformed (p < .0001) the SGT group at T1. At T2 there was no difference between both groups (p = 0.55). Both interventions led to a significant growth in self-assessed competence. The VP group felt better prepared to diagnose and treat real patients and regarded VP cases as a rewarding learning experience.

CONCLUSIONS

VP cases are an effective alternative to lecture-led SGT in terms of learning efficacy in the short and long-term as well as self-assessed competence growth and student satisfaction. Furthermore, integrating VP cases within a curricular Oral and Maxillofacial Surgery Clerkship is feasible and leads to substantial growth of clinical competence in undergraduate dental students.

Can General Practitioners manage mental disorders in primary care? A partially randomised, pragmatic, cluster trial

BACKGROUND

For a decade, experts have suggested integrating mental health care into primary care to help bridge mental health Treatment Gap. General Practitioners (GPs) are the first port-of-call for many patients with mental ill-health. In Indonesia, the WHO mhGAP is being systematically introduced to its network of 10,000 primary care clinics as an add-on mental health training for pairs of GPs and Nurses, since the end of 2015. In one of 34 provinces, there exists an integrated care model: the co-location of clinical psychologists in primary care clinics. This trial evaluates patient outcomes among those provided mental health care by GPs with those treated by clinical psychologists in primary care.

METHODS

In this partially-randomised, pragmatic, two-arm cluster non-inferiority trial, 14 primary care clinics were assigned to receive the WHO mhGAP training and 14 clinics with the co-location framework were assigned to the Specialist arm. Participants (patients) were blinded to the existence of the other pathway, and outcome assessors were blinded to group assignment. All adult primary care patients who screened positive for psychiatric morbidity were eligible. GPs offered psychosocial and/or pharmacological interventions and Clinical Psychologists offered psychosocial interventions. The primary outcome was health and social functioning as measured by the HoNOS and secondary outcomes include disability measured by WHODAS 2.0, health-related quality of life measured by EQ-5D-3L, and resource use and costs evaluated from a health services perspective, at six months.

RESULTS

153 patients completed the outcome assessment following GP care alongside 141 patients following Clinical Psychologists care. Outcomes of GP care were proven to be statistically not inferior to Clinical Psychologists in reducing symptoms of social and physical impairment, reducing disability, and improving health-related quality of life at six months. Economic analyses indicate lower costs and better outcomes in the Specialist arm and suggest a 50% probability of WHO mhGAP framework being cost-effective at the Indonesian willingness to pay threshold per QALY.

CONCLUSION

General Practitioners supported by nurses in primary care clinics could effectively manage mild to moderate mental health issues commonly found among primary care patients. They provide non-stigmatising mental health care within community context, helping to reduce the mental health Treatment Gap.

TRIAL REGISTRATION

ClinicalTrials.gov NCT02700490.

Quantifying the medical student learning curve for ECG rhythm strip interpretation using deliberate practice

Objectives: Obtaining competency in medical skills such as interpretation of electrocardiograms (ECGs) requires repeated practice and feedback. Structured repeated practice and feedback for ECGs is likely not provided to most medical students, so skill development is dependent on opportunistic training during clinical rotations. Our aim was to describe: the amount of deliberate practice completed for learning ECG rhythm strip diagnoses in first year medical students, the learning curve for rhythm strip diagnosis, and student experiences with deliberate practice. Methods: First year medical students from two medical schools were provided with online rhythm strip practice cases. Diagnostic accuracy was measured throughout practice, and students were provided feedback for every case they completed. Total cases practiced and time spent practicing were correlated with their performance during practice and on an exam. Results: 314 of 384 (82%) students consented. The mean number of ECGs each student practiced was 59 (range 0-280), representing 18,466 total instances of deliberate practice. We generated mathematical models that accurately correlated both the number of cases practiced and time spent practicing, with diagnostic accuracy on an exam (p<0.001). For example, students would need to spend on average of 112 minutes and complete 34 practice cases to obtain 75% on an ECG rhythm strip exam. Student satisfaction was high using the online cases. Conclusions: We succeeded in delivering deliberate practice for ECG rhythm strip interpretation to a large cohort of students at 2 medical schools. We quantified a learning curve that estimates the number of cases and practice time required to achieve pre-determined levels of diagnostic accuracy. This data can help inform a competency-based approach to curriculum development.

Educational impact of assessment on medical students' learning at Tehran University of Medical Sciences: a qualitative study

OBJECTIVES

It has been shown that assessment strongly affects students' performance. A deeper insight needs to be gained into the interplay of assessment and learning. The aim of the current study was to develop a model to explain the educational impact of assessments on students' learning, before, during and after the test.

DESIGN

This study used semistructured interviews, focus group discussions and observation and collection of field notes. A qualitative methodology using the grounded theory data analysis approach was then used to generate an explanation of the process of how assessment impacts students' learning.

SETTING

School of Medicine, Tehran University of Medical Sciences.

PARTICIPANTS

Participants were medical students and teachers with first-hand experience or expertise in assessment as well as their willingness to participate in the study. Fifteen people (eight medical students, seven faculty members) were interviewed. One focus group discussion (with five students) was held.

RESULTS

The extracted concepts from our study were classified into four main categories. These categories include elements of the assessment programme which affect learning, the mechanism through which they exert their effects, contextual factors and the impact they have on learning. These elements and their interplay occur within an environment with its antecedent characteristics.

CONCLUSIONS

This study suggested a model for understanding the elements of the assessment which, within the context, affect learning, the mechanisms through which they impart their effects and the final outcomes obtained.

[Competence-based assessment in the national licensing examination in Germany]

In Germany, future physicians have to pass a national licensing examination at the end of their medical studies. Passing this examination is the requirement for the license to practice medicine. The Masterplan Medizinstudium 2020 with its 41 measures aims to shift the paradigm in medical education and medical licensing examinations.The main goals of the Masterplan include the development towards competency-based and practical medical education and examination as well as the strengthening of general medicine. The healthcare policy takes into account social developments, which are very important for the medical education and licensing examination.Seven measures of the Masterplan relate to the realignment of the licensing examinations. Their function to drive learning should better support students in achieving the study goal defined in the German Medical Licensure Act: to educate a medical doctor scientifically and practically who is qualified for autonomous and independent professional practice, postgraduate education and continuous training.

Effects and Consequences of Being an OSCE Examiner in Surgery-A Qualitative Study

OBJECTIVE

Even though objective structured clinical examination (OSCE) is a well-investigated format for competency-based practical examination, only a few studies have explored the motivations of OSCE examiners and their opinions, both positive and negative, toward being an examiner. The aim of this study was to gain insights into the views of OSCE examiners using semi-structured interviews.

DESIGN

Surgical OSCE examiners were queried at two medical faculties in Germany via semi-structured interviews. The interviews were transcribed verbatim and analyzed using the techniques of structured qualitative content analysis.

SETTING

This study was conducted at the medical faculties of the Goethe University, Frankfurt, Germany and of the Otto-von-Guericke University, Magdeburg, Germany.

PARTICIPANTS

All of the study participants were surgeons working at the university hospital of one of the faculties.

RESULTS

A total of 29 examiners were queried until a saturation of content was achieved. A critical reflection of one's own teaching was described as a major benefit by most participants. Furthermore, they noted that the standards and competences examined during the OSCE boosted the detail of their teaching sessions in the wards. However, the examiners criticized missed operations due the examination and were not appreciated by superiors for being an examiner. Most of the examiners (22/29) preferred to be an examiner themselves rather than appointing student peer examiners. If they had appointed someone else, that would mean they would miss valuable experiences useful for their own teaching.

CONCLUSIONS

Being an OSCE examiner confers several advantages, notably the reflection of one's own teaching, which was described as highly valuable by the examiners.

[Analysis of the German state medical examination regarding psychiatrically relevant topics]

BACKGROUND

The second part of the state medical examination is an essential part of medical education in Germany. From 2006 until 2013, the second and third parts of the licensing examination were combined, thus earning this examination the reputation of being notoriously difficult to pass. The aim of this exploratory study was to examine psychiatric and psychotherapeutic questions in the written part of the second examination.

OBJECTIVES

This study analyzed the proportion of psychiatric and psychotherapeutic questions in the written part of the second leg of the German state medical examination. In a second step, a more detailed analysis aimed to identify more specific subject areas and to allocate diagnoses according to ICD-10, allowing a detailed overview of the presence of psychiatric topics.

MATERIAL AND METHODS

A total of 8 examinations from fall 2010 up to spring 2014 a were included in the analysis. Every examination comprised 320 questions, thus a total of 2560 questions were analyzed. All questions pertaining to psychiatric or psychotherapeutic topics were included. Questions were categorized according to a predefined set of subject areas and an ICD-10 diagnosis.

RESULTS

Out of the total of 2560 questions analyzed, 166 were categorized as being in psychiatry or psychotherapy, with each examination comprising an average of 20.75 (SD = 5.80) relevant questions. Most questions were allocated to the subject area of pharmacology (n = 53), followed by psychopathologic assessment (n = 27) and psychotherapy (n = 12). Sorted according to ICD-10 criteria, most questions could be assigned to the F3 category, the affective disorders group (21.9 %, n = 35).

CONCLUSION

In comparison to the actual strain placed on the healthcare system, the results indicate an underrepresentation of psychiatric disorders in the second part of the German state medical examination. The distribution of specific disorders mirrors the actual distribution at least in part; however, certain practically relevant disorders are severely underrepresented.

[Urology onLINE-webinar for assistants : Implementation and evaluation of a voluntary, web-based e‑learning training series for urology assistants in continuing education (Urology onLINE)]

BACKGROUND AND OBJECTIVES

The rapid development of new knowledge resources is essential for continuous training and continuing education. Since the training period often coincides with the starting of a family, time resources are often scarce. For this reason, a new, voluntary, web-based e‑learning training series was designed for urological assistants (urology onLINE). We investigated to what extent the offer of a web-based training series is used by urological training assistants and how it is evaluated by participants and speakers.

MATERIALS AND METHODS

The training series includes a one-month training on a topic from the urological continuing education, which is presented online and whose contents are checked by means of interspersed CME questions. During the investigation period November 2016 to October 2017, participants in the Urology onLINE training series were evaluated. In addition, an evaluation of individual events and an evaluation of the work load of the speakers took place.

RESULTS

On average, 60 participants participated in the individual events. These were rated very well with an average grade of 1.43 ± 0.21. Two thirds of the participants experienced an active and inquisitive experience during the event. The workload for the speakers was less than that of a comparable classroom event.

CONCLUSIONS

Overall, the new Urology onLINE training series aims to contribute to the increase in spatial and temporal flexibility, and complements existing training formats, especially in times of scarce time resources.

Workplace-based assessment of family medicine competencies using "field note tool" - A pilot study

Background

Department of Family Medicine in a medical college in South India introduced "field note" (FN) as a tool for Work-Place Based Assessment in postgraduate training. FN collects "open-ended" feedback from both resident and faculty and helps them to arrive at an action plan. This study describes our experience of implementing FN and perceptions of learners and faculty.

Methods

While precepting the residents in Family Medicine service areas, faculty documented their observations of the resident's clinical work using FN and provided an action plan. Faculty and residents described their experience and provided feedback. Focus group discussions were conducted for faculty and residents. Data were coded and grouped into themes.

Results

Four residents and seven faculties participated in the study during 12 weeks period using 17 consultations. Clinical expert (13/17) and communicator (6/17) are the most commonly assessed competencies followed by professionalism (2/17) and collaborator (2/17). Faculty and residents agreed that "FN" was a useful tool and it helped the faculty to give feedback and guide the learner. Residents and faculty arrived at an action plan in 70% of the consultations. Three of four residents perceived the change in their behaviour positively after the use of FN. Both resident and faculty found the rating of the learner using Dreyfus scale as a barrier.

Conclusion

FN could be one of the important tools in our "Toolbox of Assessment Methods" for family medicine specialty. There is a need for sensitizing the learners to feedback process and training the faculty in assessment and feedback.

Comparison of level of cognitive process between case-based items and non-case-based items of the interuniversity progress test of medicine in the Netherlands

PURPOSE

It is assumed that case-based questions require higher order cognitive processing, whereas questions that are not case-based require lower order cognitive processing. In this study, we investigated to what extent case-based questions and questions that are not case-based, relate to Bloom's taxonomy.

METHODS

In this article, 4800 questions of the Progress Test were classified whether it was a case-based question and the level of Bloom's taxonomy. Lower-order questions require students to remember or/and basically understand the knowledge. Higher-order questions require students to apply, analyze, or/and evaluate. A phi-coefficient was calculated to investigate the relations between the presence of case-based questions and the required level of cognitive processing.

RESULTS

Our results demonstrated that case-based questions were measuring higher levels of cognitive processing in 98.1% of the questions. Of the non-case-based questions, 33.7% required a higher level of cognitive processing. The phi-coefficient demonstrated a significant moderate correlation between the presence of a patient case in a question and its required level of cognitive processing (phi-coefficient = 0.55, p<0.001).

CONCLUSION

Medical teachers should be aware of the association between item formats (case-based versus non-case-based) and the cognitive processes they elicit in order to meet a certain balance in a test, taking the learning objectives as well as the test difficulty into account.

E-learning for chest x-ray interpretation improves medical student skills and confidence levels

BACKGROUND

Radiology is an important aspect of medicine to which medical students often do not receive sufficient exposure. The aim of this project was to determine whether the integration of an innovative e-learning module on chest x-ray interpretation of the heart would enhance the radiological interpretive skills, and improve the confidence, of first year graduate entry medical students.

METHODS

All first-year graduate entry (all students had a prior university degree) medical students at the University of Limerick (n = 152) during academic year 2015-16 were invited to participate in this study. An assessment instrument was developed which consisted of 5 radiological cases to be interpreted over a designated and supervised 15-min time period. Students underwent a pre-, mid- and post-intervention assessment of their radiology interpretative skills. An online e-module was provided following the pre-test and additional practice cases were provided following the mid-intervention test. Assessment scores and confidence levels were compared pre-, mid- and post-intervention.

RESULTS

The overall performance (out of a total score of 25) for the 87 students who completed all three assessments increased from 13.2 (SD 3.36) pre-intervention to 14.3 (SD 2.97) mid-intervention to 15.8 (SD 3.40) post-intervention. This change over time was statistically significant (p < 0.001) with a medium effect size (eta-squared = 0.35). Increases from pre- to post-intervention were observed in each of the five areas assessed, although performance remained poor in diagnosis post-intervention. Of the 118 students who provided feedback after the intervention, 102 (86.4%) stated that they would recommend the resource to a colleague to improve their interpretative skills.

CONCLUSIONS

This study suggests that early exposure to e-learning radiology modules is beneficial in undergraduate medical school curricula. Further studies are encouraged to assess how long the improvement may last before attrition.

Pretest Scores Uniquely Predict 1-Year-Delayed Performance in a Simulation-Based Mastery Course for Central Line Insertion

INTRODUCTION

Within simulation-based mastery learning (SBML) courses, there is inconsistent inclusion of learner pretesting, which requires considerable resources and is contrary to popular instructional frameworks. However, it may have several benefits, including its direct benefit as a form of deliberate practice and its facilitation of more learner-specific subsequent deliberate practice. We consider an unexplored potential benefit of pretesting: its ability to predict variable long-term learner performance.

METHODS

Twenty-seven residents completed an SBML course in central line insertion. Residents were tested on simulated central line insertion precourse, immediately postcourse, and after between 64 and 82 weeks. We analyzed pretest scores' prediction of delayed test scores, above and beyond prediction by program year, line insertion experiences in the interim, and immediate posttest scores.

RESULTS

Pretest scores related strongly to delayed test scores (r = 0.59, P = 0.01; disattenuated ρ = 0.75). The number of independent central lines inserted also related to year-delayed test scores (r = 0.44, P = 0.02); other predictors did not discernibly relate. In a regression model jointly predicting delayed test scores, pretest was a significant predictor (β = 0.487, P = 0.011); number of independent insertions was not (β = 0.234, P = 0.198).

CONCLUSIONS

This study suggests that pretests can play a major role in predicting learner variance in learning gains from SBML courses, thus facilitating more targeted refresher training. It also exposes a risk in SBML courses that learners who meet immediate mastery standards may be incorrectly assumed to have equal long-term learning gains.

[The examination mix at the Medical University Vienna]

The present work describes a curriculum design with the instrument of an examination mix for the study of medicine at the medical University of Vienna. The fields of application of special examination formats as well as the advantages and disadvantages of the individual examination forms are presented. Types of summative written examinations, assessment of practical skills and abilities (OSCE), oral-practical examinations, as well as formative examinations are illustrated. Studies show that repeated testing leads to higher learning gains. Therefore, the challenge is to develop suitable methods for the continuous assessment of the learning progress in the study and to apply accordingly. Thus, in addition to the written summative exams, more and more oral and practical forms of examinations in courses with continuous assessment character should be used.

[Examination facility at Hannover Medical School]

Dissection of the examination system of Hannover Medical School has identified potential for improvement of the complete procudere. Five scopes has been identified: 1. advancement of electronic examinations, 2. improvement of quality control, 3. central management of all exams, 4. more transperancy, 5. establishment of an incentive structure. The strategies for improvement were presented.

Training of clinical reasoning with a Serious Game versus small-group problem-based learning: A prospective study

INTRODUCTION

Serious Games are increasingly being used in undergraduate medical education. They are usually intended to enhance learning with a focus on knowledge acquisition and skills development. According to the current literature, few studies have assessed their effectiveness regarding clinical reasoning (CR). The aim of this prospective study was to compare a Serious Game, the virtual Accident & Emergency department 'EMERGE' to small-group problem-based learning (PBL) regarding student learning outcome on clinical reasoning in the short term.

METHODS

A total of 112 final-year medical students self-selected to participate in ten 90-minute sessions of either small-group PBL or playing EMERGE. CR was assessed in a formative examination consisting of six key feature cases and a final 45-minute EMERGE session.

RESULTS

Overall, the EMERGE group (n = 78) scored significantly higher than the PBL group (n = 34) in the key feature examination (62.5 (IQR: 17.7)% vs. 54.2 (IQR: 21.9)%; p = 0.015). There was no significant difference in performance levels between groups regarding those cases which had been discussed in both instructional formats during the training phase. In the final EMERGE session, the EMERGE group achieved significantly better results than the PBL group in all four cases regarding the total score as well as in three of four cases regarding the final diagnosis and the correct therapeutic interventions.

CONCLUSION

EMERGE can be used effectively for CR training in undergraduate medical education. The difference in key feature exam scores was driven by additional exposure to more cases in EMERGE compared to PBL despite identical learning time in both instructional formats. EMERGE is a potential alternative to intensive small-group teaching. Further work is needed to establish how Serious Games enhance CR most effectively.

Step Up-Not On-The Step 2 Clinical Skills Exam: Directors of Clinical Skills Courses (DOCS) Oppose Ending Step 2 CS

Recently, a student-initiated movement to end the United States Medical Licensing Examination Step 2 Clinical Skills and the Comprehensive Osteopathic Medical Licensing Examination Level 2-Performance Evaluation has gained momentum. These are the only national licensing examinations designed to assess clinical skills competence in the stepwise process through which physicians gain licensure and certification. Therefore, the movement to end these examinations and the ensuing debate merit careful consideration. The authors, elected representatives of the Directors of Clinical Skills Courses, an organization comprising clinical skills educators in the United States and beyond, believe abolishing the national clinical skills examinations would have a major negative impact on the clinical skills training of medical students, and that forfeiting a national clinical skills competency standard has the potential to diminish the quality of care provided to patients. In this Perspective, the authors offer important additional background information, outline key concerns regarding the consequences of ending these national clinical skills examinations, and provide recommendations for moving forward: reducing the costs for students, exploring alternatives, increasing the value and transparency of the current examinations, recognizing and enhancing the strengths of the current examinations, and engaging in a national dialogue about the issue.

The effect of repeated testing vs repeated practice on skills learning in undergraduate dental education

OBJECTIVES

Recent studies in undergraduate medical education have demonstrated the advantage of repeated testing over repeated practice with regard to knowledge and skills retention. The aim of this study was to investigate whether this "testing effect" also applies to skills retention in undergraduate dental education.

METHODS

In this prospective, randomised controlled trial, fourth-year dental students at Göttingen University Medical Centre participated in a training session on surgical suturing in winter term 2014/2015. Following this, they were either assigned to two sessions of additional skills training (group A) or two sessions of skills assessment with feedback (group B). These sessions were spaced over a period of 4 weeks. Skills retention was assessed in a summative objective structured clinical examination (OSCE) at the end of term, that is 6 months after the initial teaching session.

RESULTS

A total of 32 students completed the study. With regard to suturing, OSCE performance was significantly better in group B than group A (81.9±13.1% vs 63.0±15.4%; P=0.001; Cohen's d=1.33). There was no significant OSCE performance difference in the two groups with regard to other learning objectives that were addressed in the end-of-term examination. Thus, the group difference was specific to suturing skills.

CONCLUSIONS

This is the first study to demonstrate that in dental education, repeated testing produces more favourable skills retention than repeated practice. Test-enhanced learning might be a viable concept for skills retention in undergraduate dentistry education.

Is computer-assisted instruction more effective than other educational methods in achieving ECG competence among medical students and residents? Protocol for a systematic review and meta-analysis

INTRODUCTION

Although ECG interpretation is an essential skill in clinical medicine, medical students and residents often lack ECG competence. Novel teaching methods are increasingly being implemented and investigated to improve ECG training. Computer-assisted instruction is one such method under investigation; however, its efficacy in achieving better ECG competence among medical students and residents remains uncertain.

METHODS AND ANALYSIS

This article describes the protocol for a systematic review and meta-analysis that will compare the effectiveness of computer-assisted instruction with other teaching methods used for the ECG training of medical students and residents. Only studies with a comparative research design will be considered. Articles will be searched for in electronic databases (PubMed, Scopus, Web of Science, Academic Search Premier, CINAHL, PsycINFO, Education Resources Information Center, Africa-Wide Information and Teacher Reference Center). In addition, we will review citation indexes and conduct a grey literature search. Data extraction will be done on articles that met the predefined eligibility criteria. A descriptive analysis of the different teaching modalities will be provided and their educational impact will be assessed in terms of effect size and the modified version of Kirkpatrick framework for the evaluation of educational interventions. This systematic review aims to provide evidence as to whether computer-assisted instruction is an effective teaching modality for ECG training. It is hoped that the information garnered from this systematic review will assist in future curricular development and improve ECG training.

ETHICS AND DISSEMINATION

As this research is a systematic review of published literature, ethical approval is not required. The results will be reported according to the Preferred Reporting Items for Systematic Review and Meta-Analysis statement and will be submitted to a peer-reviewed journal. The protocol and systematic review will be included in a PhD dissertation.

PROSPERO REGISTRATION NUMBER

CRD42017067054; Pre-results.

[Choosing Wisely in medical education]

Choosing Wisely recommendations address situations where physicians will have to make decisions about further diagnostic and therapeutic steps. Undergraduate medical education needs to equip students with the foundations on which clinical reasoning skills can be acquired and fostered throughout their clinical career. Teaching these skills usually involves patients (e.g., bedside teaching, electives, clinical attachments) but it can also be delivered in the format of formalised small-group, case-based learning. Case-based key feature tests have been developed to facilitate the assessment of learning outcomes related to clinical reasoning. Repeated testing with key feature cases yields better medium-term retention than repeatedly studying the same material (without questions). The project 'Choosing Wisely in medical education', which was funded by the German Association for Internal Medicine, involves the creation of key feature cases with reference to the German set of Choosing Wisely recommendations. This article presents the results of the first pilot study using these new cases.

Computer model for the cardiovascular system: development of an e-learning tool for teaching of medical students

BACKGROUND

This study combined themes in cardiovascular modelling, clinical cardiology and e-learning to create an on-line environment that would assist undergraduate medical students in understanding key physiological and pathophysiological processes in the cardiovascular system.

METHODS

An interactive on-line environment was developed incorporating a lumped-parameter mathematical model of the human cardiovascular system. The model outputs were used to characterise the progression of key disease processes and allowed students to classify disease severity with the aim of improving their understanding of abnormal physiology in a clinical context. Access to the on-line environment was offered to students at all stages of undergraduate training as an adjunct to routine lectures and tutorials in cardiac pathophysiology. Student feedback was collected on this novel on-line material in the course of routine audits of teaching delivery.

RESULTS

Medical students, irrespective of their stage of undergraduate training, reported that they found the models and the environment interesting and a positive experience. After exposure to the environment, there was a statistically significant improvement in student performance on a series of 6 questions based on cardiovascular medicine, with a 33% and 22% increase in the number of questions answered correctly, p < 0.0001 and p < 0.001 respectively.

CONCLUSIONS

Considerable improvement was found in students' knowledge and understanding during assessment after exposure to the e-learning environment. Opportunities exist for development of similar environments in other fields of medicine, refinement of the existing environment and further engagement with student cohorts. This work combines some exciting and developing fields in medical education, but routine adoption of these types of tool will be possible only with the engagement of all stake-holders, from educationalists, clinicians, modellers to, most importantly, medical students.