Benchmarking for research-related competencies - a curricular mapping approach at medical faculties in Germany

Can positive expectations help to improve the learning of risk literacy? A cluster-randomized study in undergraduate medical students

BACKGROUND

Risk literacy, i.e., the ability to calculate and apply risk parameters, represents a key competence for risk communication and medical decision making. However, risk literacy is reportedly low in medical students. The successful acquisition of statistical competencies is often difficult, and can be hampered by emotional learning obstacles, calling for interventions to support learning. In this cluster-randomized study, we aimed to translate findings from placebo research to medical education. Specifically, we tested if the acquisition of risk literacy during a seminar unit can be facilitated by positive expectations, induced by a positive and non-threatening framing of the content and learning goals.

METHODS

The study took place during a mandatory 2.5-h seminar on "risk literacy" for 2^nd year medical students. The seminar teaches both statistical knowledge and its application in patient communication. To test the effects of expectations on risk literacy acquisition, the (otherwise identical) seminar was framed either as "communication training" (positive framing condition) or "statistics seminar" (negative framing condition). All N = 200 students of the semester were invited to participate, and cluster-randomized to the positive or negative framing condition (4 seminar groups each condition). Risk literacy was assessed with the "Quick Risk Test" (QRT) at the beginning and end of the seminar, along with statistics anxiety and subjective learning success using questionnaires.

RESULTS

Data from N = 192 students were included. At the end of the seminar, risk literacy was increased in both framing conditions, with a significantly greater increase in QRT scores in the positive framing condition. Statistics anxiety was significantly decreased in both framing conditions, with no evidence of group differences. Subjective learning success was overall high and comparable between groups.

CONCLUSIONS

Supporting our hypothesis, positive framing led to a significantly greater increase in risk literacy (i.e., in QRT scores). Our data offer first support that positive framing of learning goals may help to facilitate the acquisition of statistical knowledge. Expectation-orientated interventions may thus offer a feasible tool to optimize learning settings and framing of learning objectives in medical statistics courses.

[Project report on fostering scientific competencies in pain medicine in the context of student education]

AIM

The training of scientific skills and competencies is an essential part of academic medical studies. As part of the MaReCuM model study program at Heidelberg University's Mannheim Medical School, a fifth-year rotation on scientific skills in the field of pain medicine was implemented. This paper describes this competence-oriented rotation as well as the investigation of the educational effect.

METHOD

A total of 114 fifth-year medical students participated in the survey (response rate: 83%). The control group completed the fifth year prior to the implementation of the rotation. The experimental group was required to participate in the rotation and the real healthcare research study "Case management program: low back pain". A survey of both groups was conducted on the first day of the rotation and at the end of the module.

RESULTS

The innovative and competency-based learning unit was successfully implemented as part of the MaReCuM model study program and carried out with partners in general practice as well as the Mannheim Institute of Public Health. The participating students accepted the rotation well. There was no measurable effect on the subjective learning success of the rotation in the evaluation.

DISCUSSION

To the authors' knowledge, this educational approach has never been tested before in a German study program. The presented rotation offers an additional option for the training of scientific competencies as part of medical studies. The missing of a measurable effect could be due to the extensive experience of the medical students as well as the limitations on participation in a real healthcare study. An additional learning opportunity could be created by connecting the preexisting lectures to a longitudinal module on scholarly competencies. The implementation of the program also offers a unique opportunity for educational research on the acquisition of scientific competencies in medical students.

Patterns of Domain-Specific Learning Among Medical Undergraduate Students in Relation to Confidence in Their Physiology Knowledge: Insights From a Pre-post Study

RESEARCH FOCUS

The promotion of domain-specific knowledge is a central goal of higher education and, in the field of medicine, it is particularly essential to promote global health. Domain-specific knowledge on its own is not exhaustive; confidence regarding the factual truth of this knowledge content is also required. An increase in both knowledge and confidence is considered a necessary prerequisite for making professional decisions in the clinical context. Especially the knowledge of human physiology is fundamental and simultaneously critical to medical decision-making. However, numerous studies have shown difficulties in understanding and misconceptions in this area of knowledge. Therefore, we investigate (i) how preclinical medical students acquire knowledge in physiology over the course of their studies and simultaneously gain confidence in the correctness of this knowledge as well as (ii) the interrelations between these variables, and (iii) how they affect the development of domain-specific knowledge.

METHOD

In a pre-post study, 169 medical students' development of physiology knowledge and their confidence related to this knowledge were assessed via paper-pencil questionnaires before and after attending physiology seminars for one semester. Data from a longitudinal sample of n = 97 students were analyzed using mean comparisons, regression analyses, and latent class analyses (LCAs). In addition, four types of item responses were formed based on confidence and correctness in the knowledge test.

RESULTS

We found a significant and large increase in the students' physiology knowledge, with task-related confidence being the strongest predictor (apart from learning motivation). Moreover, a significantly higher level of confidence at t2 was confirmed, with the level of prior confidence being a strong predictor (apart from knowledge at t2). Furthermore, based on the students' development of knowledge and confidence levels between measurement points, three empirically distinct groups were distinguished: knowledge gainers, confidence gainers, and overall gainers. The students whose confidence in incorrect knowledge increased constituted one particularly striking group. Therefore, the training of both knowledge and the ability to critically reflect on one's knowledge and skills as well as an assessment of their development in education is required, especially in professions such as medicine, where knowledge-based decisions made with confidence are of vital importance.

Approaches to mapping an undergraduate medical curriculum to a national competency-based catalogue of learning outcomes

BACKGROUND

Curriculum mapping plays an increasing role in the design and implementation of competency-based medical education. We present an exemplary mapping of an institutional undergraduate medical curriculum to a national competency-based outcome framework to compare approaches to mapping procedures in their capacity to depict the relative coverage.

METHODS

The mapping process was performed by a small working group that continuously reconciled its findings. In step 1, we mapped the course objectives of our programme (Charité Berlin, Germany) to the National Competency-Based Catalogue of Learning Outcomes Medicine (NKLM). In step 2, we employed three primarily quantitative approaches (single, multiple, and subordinate match) and one primarily qualitative approach (content comparison) to derive the degree of NKLM coverage.

RESULTS

In step 1, we mapped a total of 4400 programme objectives to 2105 NKLM objectives. In step 2, the quantitative approaches provided a general overview of the pattern of coverage, while the qualitative approach required more effort but provided a better representation of the scope and depth of coverage.

DISCUSSION

The mapping approach chosen markedly impacts on the results how of well an institutional curriculum covers a national standard. This study highlights the need for more rigour in the methodology and reporting of curriculum mapping.

Defining Learning Outcomes as a Prerequisite of Implementing a Longitudinal and Transdisciplinary Curriculum with Regard to Digital Competencies at Hannover Medical School

BACKGROUND

Worldwide educational programs face the challenge how to define and integrate digital competencies in medical education. This article describes the implementation of learning outcomes with respect to digital competencies in the compulsory curriculum at Hannover Medical School (MHH).

METHODS

An interdisciplinary MHH project group was constituted consisting of physicians and experts in medical informatics and in curriculum development. Over the course of 7 work sessions the group compared different international and national frameworks dealing with digital competencies for physicians. By a consensus driven approach the working group drew up a collection of learning outcomes which were regarded relevant to be incorporated in the curriculum at MHH.

RESULTS

The analysis of different frameworks indicated that data literacy is a central domain within all viewed preexisting catalogs. During the course of the project group analysis, 57 learning outcomes with respect to digital competencies were identified as necessary to be integrated in the compulsory curriculum. They were divided in 5 main categories: "handling of medical data," "the digital infrastructure of the health system," "scope of application: usage in patient care and in the field of preventive medicine," "medico-legal and ethical basics," and "transformation processes in medicine due to digitalization."

CONCLUSIONS

The MHH project group concluded that medical students should be taught digital competencies that enable an understanding of underlying functional principles of digital systems rather than their correct utilization. The presented project indicates that a close interdisciplinary collaboration of physicians and medical informaticians can be a promising approach to incorporate digital competencies in the undergraduate medical curriculum.

Integration of Scientific Competence into Gross Anatomy Teaching Using Poster Presentations: Feasibility and Perception among Medical Students

Scientific competences as defined in the German competency framework describes the ability to think independently and act scientifically, and forms a central component of medical education. This report describes its integration into anatomical teaching. On the basis of the findings in dissection courses from two consecutive years, students worked on either a case report (n = 70) or an original work (n=6) in the format of a scientific poster while learning to use primary literature. Posters were evaluated by juror teams using standardized evaluation criteria. Student perception of the project was estimated by quantitative and qualitative data obtained from the faculty´s course evaluation and an online-survey. Overall, students worked collaboratively and invested extra-time (median [MD] 3.0 hours) in poster creation. Primary literature was integrated in 90.8% of the posters. Overall poster quality was satisfactory (46.3 ±8.5 [mean ±standard deviation] out of 72 points), but several insufficiencies were identified. Students integrated information gained from the donor´s death certificate, post-mortem full-body computer tomography (CT) scan (22.4%) and histopathological workup (31.6%) in their case reports. Students were positive about the experience of learning new scientific skills (MD 4 on a six-point Likert scale), but free text answers revealed that some students experienced the project as an extra burden in a demanding course. In summary, it was feasible to introduce students to science during the dissection course and to increase interest in science in approximately a third of the survey respondents. Further adjustments to ensure the posters´ scientific quality might be necessary in the future.

Surgeons' participation in the development of collaboration and management competencies in undergraduate medical education

The teaching of professional roles in medical education is an interdisciplinary concern. However, surgeons require specific standards of professionalism for certain context-based situations. In addition to communication, studies require collaboration, leadership, error-/conflict-management, patient-safety and decision-making as essential competencies for surgeons. Standards for corresponding competencies are defined in special chapters of the German National Competency-based Learning Objectives for Undergraduate Medical Education (NKLM; chapter 8, 10). The current study asks whether these chapters are adequately taught in surgical curricula. Eight German faculties contributed to analysing mapping data considering surgical courses of undergraduate programs. All faculties used the MERlin mapping platform and agreed on procedures for data collection and processing. Sub-competency and objective coverage, as well as the achievement of the competency level were mapped. Overall counts of explicit citations were used for analysis. Collaboration within the medical team is a strongly represented topic. In contrast, interprofessional cooperation, particularly in healthcare sector issues is less represented. Patient safety and dealing with errors and complications is most emphasized for the Manager/Leader, while time management, career planning and leadership are not addressed. Overall, the involvement of surgery in teaching the competencies of the Collaborator and Manager/Leader is currently low. However, there are indications of a curricular development towards explicit teaching of these roles in surgery. Moreover, implicitly taught roles are numerous, which indicates a beginning awareness of professional roles.

Development, implementation, and evaluation of a curriculum for medical students on conflicts of interest and communicating risk

Background: Insufficient risk competence of physicians, conflicts of interests from interactions with pharmaceutical companies, and the often distorted presentation of benefits and risks of therapies compromise the advising of patients by physicians in the framework of shared decision-making. An important cause of this is that teaching on this subject is mostly lacking, or fragmented when it does take place [1], [2], [3], [4]. Even though the German National Competence-Based Catalog of Learning Goals in Medicine defines learning goals on the topics of conflicts of interest and communication of risk, there are no classes that integrate both topics. Our goal was to develop a model curriculum to teach conflicts of interest and communication of risk that would integrate statistical know-how, communicational competency on the presentation of benefits and risks, and the meaning and management of conflicts of interest. Project Description: The development of the curriculum took place according to the six-step cycle of Kern et al [5]. An integrated curriculum was conceptualized, piloted, and adapted with the support of experts for the topics of shared decision-making, conflicts of interest, and communication of risk. The final version of the curriculum was implemented at the medical schools of Mainz and Heidelberg and evaluated by the students. Results: The final curriculum consists of 19 lesson units. The contents are the fundamentals of statistics, theory of risk communication, practical exercises on communication of risk, and the fundamentals of the mechanisms of effect of conflicts of interest, recognition of distortions in data, and introductions to professional management of conflicts of interest. The course was implemented three times at two different medical schools with a total of 32 students, and it was positively rated by most of the 27 participating students who evaluated it on the 1-6 German school grading scale (mean: 1.4; SD: 0.49; range: 1-2). Discussion: The curriculum we developed fills a gap in the current medical education. The innovative concept, which sensibly connects the transmission of theory and practice, was positively received by the students. The next steps are an evaluation of the curriculum by means of a two-center randomized study and the implementation at German and international medical schools. The process should be accompanied by continuous evaluation and further improvement.

Are we preparing for collaboration, advocacy and leadership? Targeted multi-site analysis of collaborative intrinsic roles implementation in medical undergraduate curricula

BACKGROUND

The Collaborator, Health Advocate and Leader/Manager roles are highly relevant for safe patient management and optimization of healthcare system in rehabilitation and prevention. They are defined in competency-based frameworks and incorporate competencies empowering physicians to master typical daily tasks in interdisciplinary, interprofessional and institutional collaboration. However, appropriate implementation of roles remains difficult in undergraduate medical education (UME) and needs to be closely monitored. The aim of this cross-institutional mapping study was to examine for the roles of Collaborator, Health Advocate and Leader/Manager: (1) To what extent do German UME programs explicitly meet the given standards after 5 years of study? (2) Which information may be obtained from multi-site mapping data for evidence-based reflection on curricula and framework?

METHODS

In a joint project of eight German UME programs, 80 to 100% of courses were mapped from teachers' perspective against given national standards: (sub-)competency coverage, competency level attainment and assessment. All faculties used a common tool and consented procedures for data collection and processing. The roles' representation was characterized by the curricular weighting of each role content expressed by the percentage of courses referring to it (citations). Data were visualized in a benchmarking approach related to a general mean of the intrinsic roles as reference line.

RESULTS

(Sub-)competencies of the Health Advocate are consistently well-integrated in curricula with a wide range of generally high curricular weightings. The Collaborator reveals average curricular representation, but also signs of ongoing curricular development in relevant parts and clear weaknesses regarding assessment and achieved outcomes. The Leader/Manager displays consistently lowest curricular weightings with several substantial deficiencies in curricular representation, constructive alignment and/or outcome level. Our data allow identifying challenges to be considered by local curriculum developers or framework reviewers (e.g. non-achievement of competency levels, potential underrepresentation, lacking constructive alignment).

CONCLUSION

Our non-normative, process-related benchmarking approach provides a differentiated crosscut snapshot to compare programs in the field of others, thus revealing shortcomings in role implementation, especially for Leader/Manager and Collaborator. The synopsis of multi-site data may serve as an external reference for program self-assessment and goal-oriented curriculum development. It may also provide practical data for framework review.

Teaching Conflicts of Interest and Shared Decision-Making to Improve Risk Communication: a Randomized Controlled Trial

BACKGROUND

Risk communication is a core aspect of a physician's work and a fundamental prerequisite for successful shared decision-making. However, many physicians are not able to adequately communicate risks to patients due to a lack of understanding of statistics as well as inadequate management of conflicts of interest (COI).

OBJECTIVE

To evaluate the effects of an integrated curriculum encompassing COI and shared decision-making on the participants' risk communication competence, that is, their competence to advise patients on the benefits and harms of diagnostic or therapeutic interventions.

DESIGN

A rater-blind randomized controlled trial with a 30 (± 1)-week follow-up conducted from October 2016 to June 2017 at two German academic medical centers.

PARTICIPANTS

Sixty-three medical students in their fourth or fifth year.

INTERVENTIONS

Participants received either a newly developed 15-h curriculum or a course manual adapted from teaching as usual.

MAIN MEASURES

Primary outcome: change in risk communication performance in a video-observed structured clinical examination (VOSCE).

KEY RESULTS

Participants were 25.7 years old on average (SD 3.6); 73% (46/63) were female. Increase in risk communication performance was significantly higher in the intervention group with post-intervention Cohen's d of 2.35 (95% confidence interval (CI) 1.62 to 3.01, p < 0.01) and of 1.83 (CI 1.13 to 2.47, p < 0.01) 30 (± 1) weeks later. Secondary outcomes with the exception of frequency of interactions with the pharmaceutical industry also showed relevant improvements in the intervention as compared with the control group (d between 0.91 and 2.04 (p < 0.001)).

CONCLUSIONS

Our results show that an integrated curriculum encompassing COI and risk communication leads to a large and sustainable increase in risk communication performance. We interpret the large effect sizes to be a result of the integration of topics that are usually taught separately, leading to a more effective organization of knowledge.

TRIAL REGISTRATION

The trial is registered in the International Clinical Trials Registry with the trial number DRKS00010890.

Developing a preliminary questionnaire for the faculty development programme needs of medical teachers using Delphi technique

Objective

The study aimed to develop a preliminary medical teachers' faculty development programme (FDP) needs questionnaire through two rounds of Delphi technique.

Methods

This study utilised the Delphi study between April to June 2019. Face-to-face interviews and a literature review were conducted to propose a set of domains and items for the FDP needs of medical teachers. Two rounds of the Delphi technique were incorporated to obtain a consensus for the proposed questionnaire by 10 expert panels from their respective fields. The consensus was pre-defined as a mean score of four or above and with a percent agreement of 75%.

Results

Initially, four domains and 26 items were proposed. Finally, a total of six domains and 38 items were endorsed by the expert panels. The selected domains included six competencies, including teaching, assessment, research, curriculum, publication, and public service. These domains consisted of seven, nine, six, seven, four, and five items, respectively.

Conclusion

This study developed the first preliminary FDPs needs questionnaire specifically designed for medical teachers. It would be an effective instrument to measure the needs of the FDPs in medical education.

Intrinsic roles in the crosshair - strategic analysis of multi-site role implementation with an adapted matrix map approach

BACKGROUND

The implementation of competency-based intrinsic roles in undergraduate medical education remains a challenge. Faculties in transition need to be provided with generalizable curricular data in order to facilitate orientation on curricular roles' representation and to decide on steps of curriculum development. Explicit and implicit representation of objectives and multi-site agreement can be viewed as status indicators for the adoption of roles. Our aim was to develop a pragmatic cross-locational approach to capture roles' developmental status in an overview and prioritize strategic recommendations.

METHODS

Based on the mapping data from six German medical faculties, the relationship between explicit and implicit curricular representation of role' objectives (weighting) and extent of programs' consent (agreement) was calculated. Data was visualized in a role-specific Matrix Map to analyse roles' implicit-explicit relation and risk-value potential. The matrix was combined with Roger's stages of innovation diffusion for differentiated interpretation of the developmental role status.

RESULTS

Entangling multi-site agreement and curricular weighting, the 4-Field-Matrix allows to assess objectives based on their current localization in a quadrant: "Disregard" (lower left) and "Progress" quadrant (upper left) reveal the diffusion period; "Potential" (lower right) and "Emphasis" quadrant (upper right) indicate the adoption period. The role patterns differ in curricular representation, progression and clarity: (1) Scholar: explicit/implicit - scattered across the matrix; most explicit objectives in "Progress". (2) Health Advocate: explicit - primarily in "Emphasis"; only role in which the explicit representation significantly exceeds the implicit. (3) Collaborator: explicit - mainly "Potential"; implicit - "Progress" or "Emphasis". (4) Professional: explicit - primarily "Potential" but also "Emphasis"; implicit - "Progress" and "Emphasis"; appears better adopted but scattered in weighting; high hidden curricula. (5) Manager: explicit and implicit - exclusively in "Potential", without signs of development. Role patterns correspond to evidences from literature. Exemplified with roles, quadrant-specific strategies and measures are suggested. Framework reviewers may gain information for discussion of critical content.

CONCLUSION

The Matrix Map enables to catch intuitively the status of intrinsic roles' profiles regarding role pattern, implicit-explicit relation and programs agreement. Thus, interpretation and informed discussions are fostered. Further target-oriented analyses and strategic developments can be conducted to enhance transparency and resource-efficiency.

Promoting medical competencies through a didactic tutor qualification programme - a qualitative study based on the CanMEDS Physician Competency Framework

BACKGROUND

In peer-led tutorial courses, qualified medical students ("tutors") provide their peers with opportunities to deepen their theoretical knowledge effectively and to practice clinical skills already in preclinical semesters. At the Medical Faculty of Heidelberg University, a structured medical didactic qualification programme prepares and trains future tutors for their responsibilities. This programme consists of four modules: 1. medical didactics and group leadership, 2. subject-specific training, 3. performance of tutorial courses as well as 4. collegial advice and reflection on the tutors' activities. The aim of this study is to systematically analyse and present the development of role competencies for medical tutors based on the CanMEDS Physician Competency Framework through the didactic qualification programme.

METHODS

We applied a qualitative research approach to detect CanMEDS role competencies acquisition within the tutor qualification programme. The CanMEDS framework describes key competencies, grouped thematically under seven professional roles. Two tutors and three training coordinators independently assigned the individual modules of the tutor qualification programme to the key competencies of the CanMEDS framework. Tutors and training coordinators compared and discussed the allocations within the groups in a consensus finding process. All authors analysed the findings in order to find out the so-called "hidden curriculum". The views of both groups are presented separately.

RESULTS

The training programme promotes the acquisition of competencies in all seven CanMEDS roles. The roles of the scholar and the leader are promoted in all modules. In addition, the first and fourth module focus predominately on the role of the collaborator, the second on the role of the medical expert and communicator, and the fourth on the role of the professional.

CONCLUSIONS

The systematic analysis through assignment of the CanMEDS roles to the individual modules of the tutor qualification programme documents the comprehensive acquisition of competencies, not only with regard to the tutor activity, but generally with regard to the later role of the physician. The reflection on one's own competency acquisition can support the promotion of corresponding competencies in the qualification programme and their transfer into the professional practice later.

Boosting competence-orientation in undergraduate medical education - A web-based tool linking curricular mapping and visual analytics

Objectives: Transition to competency-based medical education is a highly challenging endeavor. Students, teachers and institutions need curricular transparency for understanding the build-up of competencies in terms of coverage, sequence and consistence of learning objectives and assessment. The project aim was to develop and implement a web-based interactive platform for curriculum mapping, diagnostics, and development. The tool should be transferable to other faculties and allow description and visualization of medical curricula in comparison to given national competency-based standards. Methods: In a design-based multi-center approach, four German medical faculties cooperated and developed a standardized, common mapping tool (MERlin database). Implemented are techniques for big data handling and visual analytics. Results: The platform profile is adapted closely to user needs. Intuitive data entry and comfortable quality maintenance support teacher engagement. Individual navigation for curricular diagnostics is guided by practice-oriented questions. Sophisticated, easy-understandable visualizations show curricular strengths and weaknesses. Transparency in contributing departments facilitates goal-oriented dialogs. Currently, 14 of 38 German faculties use the platform. Conclusions: In view of huge amounts of data and complex curricular structures, the MERlin database facilitates effective curriculum mapping, goal-oriented curriculum development, comparison to national competency-based standards, effective data sharing and benchmarking across faculties with different curriculum management systems.