Visions and reality: the idea of competence-oriented assessment for German medical students is not yet realised in licensing examinations

Exploring the situational motivation of medical students through clinical medicine level test: a cross-sectional study

China's Clinical Medicine Level Test (CMLT) was designed to evaluate the key factors that affect the learning outcome of medical students prior to entering clinical practice. In this study, we systemically analyzed the performance of a cohort of medical students at Jinan University School of Medicine participating in the recent CMLT. The analytical results of the medical students' written and objective structured clinical examination (OSCE) scores showed that their academic performance was predominantly associated with students' internship allocations, although other demographic characteristics such as sex, age, geographical origin of students, and grade point average (GPA) might be sporadically related to the students' OSCE performance at different OSCE stations. To explore the inherent reasons behind this, a survey was implemented among the medical students who participated in the examination to further interpret the factors influencing the students' learning outcome. The findings of this questionnaire manifested that intrinsic motivation and identified regulation acted as the major motivational profiles for the medical students from three different internship sites, whereas external regulation emerged as the crucial factor to make the students perform well academically in the CMLT. The result of this study suggested that strengthening the quality control of the clinical learning environment and improving medical administration may still be the most effective approaches to ensure the quality of clinical medical education.

Implementation of a uniform nationwide medical licensing examination in general practice. A feasibility study

Objective: A competency-based training of medical students that is adapted to the realities of care is required internationally and is being intended in Germany with the Master Plan for Medical Studies 2020. In order to test these competencies, the German National Institute for state examinations in Medicine, Pharmacy and Psychotherapy (IMPP) has developed a concept for the redesign of the final part of the medical licensing examination in Germany. It focuses on general and interprofessional healthcare in the examination with outpatients. The aim of this work is to assess the feasibility of the new final examination on the basis of pilot examinations in family practices and to derive further steps for the national implementation. Methods: Fourteen medical students in their internship year completed a full examination with patients aged 42 to 84 years. Examiners evaluated the examination performance using standardised evaluation forms. Feasibility was qualitatively assessed in terms of compliance with content and time limits, examination results, patient reflections, and implementation in the practice. Results: Students were able to complete all tasks within the given time frame. Based on the evaluation forms, the examiners assessed the performance of the students. Patients appreciated the structured course of the examination in the familiar location of their family practice. For the nationwide implementation of the examination, 2,500 examination practices are required for about 10,000 examinees per year. Four students can then be examined on two days per year in each practice. Conclusions: Oral-practical examinations with outpatients in general medical practices can be carried out successfully throughout the nation. An implementation of the examinations throughout Germany requires that medical studies are restructured and that this new curriculum is implemented as intended by the Master Plan for Medical Studies 2020. Furthermore, training and remuneration of examiners together with a legal framework for the new examination must be established.

Interprofessional assessment of medical students' competences with an instrument suitable for physicians and nurses

BACKGROUND

Physicians need a set of specific competences to perform well in interprofessional teams in their first year of residency. These competences should be achieved with graduation from medical school. Assessments during undergraduate medical studies are mostly rated by supervisors only. The aim of our study was to compare the rating of core facets of competence of medical students late in their undergraduate training as well as the rating confidence between three different groups of assessors (supervisors, residents, and nurses) in an assessment simulating the first day of residency.

METHODS

Sixty-seven advanced medical students from three different medical schools (Hamburg, Oldenburg and Munich) participated in a 360-degree assessment simulating the first working day of a resident. Each participant was rated by three assessors - a supervisor, a resident and a nurse - in seven facets of competence relevant for the first year of residency: (1) responsibility, (2) teamwork and collegiality, (3) knowing and maintaining own personal bounds and possibilities, (4) structure, work planning and priorities, (5) coping with mistakes, (6) scientifically and empirically grounded method of working, and (7) verbal communication with colleagues and supervisors. Means of all assessed competences and confidences of judgement of the three rating groups were compared. Additionally, correlations between assessed competences and confidence of judgement within each group of raters were computed.

RESULTS

All rating groups showed consistent assessment decisions (Cronbach's α: supervisors = .90, residents = .80, nurses = .78). Nurses assessed the participants significantly higher in all competences compared to supervisors and residents (all p ≤ .05) with moderate and high effect sizes (d = .667-1.068). While supervisors' and residents' ratings were highest for "teamwork and collegiality", participants received the highest rating by nurses for "responsibility". Competences assessed by nurses were strongly positively correlated with their confidence of judgment while supervisors' assessments correlated only moderately with their confidence of judgment in two competences.

CONCLUSIONS

Different professional perspectives provide differentiated competence ratings for medical students in the role of a beginning resident. Rating confidence should be enhanced by empirically derived behavior checklists with anchors, which need to be included in rater training to decrease raters' subjectivity.

A pilot survey of student perceptions on the benefit of the OSCE and MCQ modalities

Objective: The objective structured clinical examination (OSCE) has become widely accepted as a form of assessment in medical education. At the same time, the more traditional multiple choice question (MCQ) examinations remain a central modality of student assessment. This pilot survey aimed to investigate students' perceptions about the benefits of the OSCE and MCQs to yield data supporting the implementation of this assessment strategy into the national medical licensing examination in Germany. Methods: A questionnaire was delivered electronically to 34 German medical schools. Students in years 3-6 were invited to rate 11 items about objectives of good medical assessment. All items were presented for both the OSCE and MCQs using a 5-point Likert Scale (1=strongly disagree to 5=strongly agree). Factor analysis was used to identify underlying components in the ratings. Average scores of items that belonged to a component were computed. Results: Data analysis included 1,082 students from 32 medical schools. For the OSCE, factor analysis revealed two components, which were labelled "educational impact" and "development of clinical competence". The average scores of items were 3.37 and 3.55, respectively. For the MCQ modality, also two components emerged. These were labelled "perceived weaknesses of MCQs" and "perceived strengths of MCQs" (consisting of items such as "promotes my theoretical knowledge"). The average scores for these components were 1.85 and 3.62. Conclusion: The results of this pilot survey indicate that students consider both OSCE and MCQs as useful assessments for the purposes for which they were designed. The assessment strategy thus appears appropriate and it should be used in the national licensing examination.