Quality indicators for learner-centered postgraduate medical e-learning

The added value of a face-to-face pan-European course-what makes it worth it?

Introduction

Over the past decade, digital education has seen widespread adoption, particularly accentuated during the COVID-19 pandemic. The post-COVID era has further emphasized the advantages of digital education in terms of cost, availability, and sustainability. However, concerns regarding the efficacy of digital education, particularly in skills-based learning and the absence of social interaction, have been raised. This paper will look at the added value of international, face-to-face, skills-based courses.

Method

This study evaluates the potential added value of face-to-face international skills courses using the European "Gynecology Experts Training for Upcoming Professionals" (GET-UP) course. Focus group discussions were conducted with participants and faculty members to explore beliefs, attitudes, and perceptions regarding face-to-face learning. Qualitative analysis was performed using thematic analysis to identify domains of added value.

Results

The GET-UP course, conducted over 4 days with a diverse European faculty and participants, highlighted several added-value domains. Themes including diversity, role models, preparation, live interaction, and community emerged from the analysis, emphasizing the significance of face-to-face interaction in enriching the learning experience beyond attaining learning goals.

Discussion

The study underscores the importance of face-to-face interaction in educational settings, offering insights into diverse teaching methods, role modeling opportunities, enhanced preparation, live interactions, and fostering a sense of community. While digital education continues to evolve with interactive features, this study suggests that the inherent pressure and dynamics of face-to-face learning provide unique benefits that may not be easily replicated in digital environments. Future research should investigate and validate these findings further to inform educational practices effectively.

Fluid and electrolyte management: increasing the knowledge of House Officers using an interactive eLearning tool

BACKGROUND

Interactive case-based tutorials represent a well-established method of improving House Officer learning. There has been little research on how tutorials of this kind can be improved, and whether their use changes practice.

AIM

Our study aims to assess whether our eLearning tutorial on IV fluid and electrolyte prescribing improves the underlying knowledge base and confidence of participating House Officers, with regards to fluid and electrolyte balance physiology and prescribing.

METHOD

An interactive eLearning module with core information on fluid and electrolyte prescribing and associated cases with questions and answers was created and distributed to participating House Officers in the 2019-2020 cohort nationwide. Participants were asked to complete pre-eLearning and post-eLearning questionnaires as well as a feedback survey to assess the efficacy of the module.

RESULTS

Forty-nine House Officers completed the eLearning module and associated questionnaires. A majority of participants (69.3%) reported their previous teaching on fluid and electrolyte management as "very poor", "poor" or "mediocre". The average score for the pre-eLearning knowledge test was 75%, compared to a score of 97% for the post-eLearning knowledge test, resulting in a 22% increase in correct answers (p < 0.001). We found an increase of 53% in feeling "confident" or "very confident" in assessing and managing fluid requirements, and an increase of 57.1% in feeling "confident" or "very confident" in managing electrolyte requirements after undertaking the eLearning module.

CONCLUSION

An interactive eLearning tutorial with real-world applications provides an effective, low-cost intervention that can improve confidence and skill in prescribing IV fluids.

The baroreceptor reflex brought to life outside the classroom - an e-learning based asynchronous laboratory class using a non-supervised modified Active Standing Test

BACKGROUND

E-learning based laboratory classes can replace or enhance in-classroom laboratories. They typically offer temporal flexibility, self-determined learning speed, repeatability and do not require supervision or face-to-face contact. The aim of this feasibility study was to investigate whether the established in-classroom laboratory class on the baroreceptor reflex (BRR) can be transformed into a new e-learning based asynchronous laboratory class for untrained, non-supervised students without medical equipment. The BRR is a fundamental cardiovascular process which is regularly visualized in physiology during in-classroom laboratories by a student-performed Active Standing Test (AST). During this voluntary provocation of orthostatic stress, the BRR reliably causes a solid rise in heart rate (HR) and a stabilization or even increase in blood pressure (BP).

METHODS

The conventional AST was modified by omission of BP measurements which would require medical devices and was embedded into a framework of interactive digital material allowing independent student performance. With specific adaptions, this instrument was implemented to 1st and 2nd year curricula of human medicine, dental medicine, midwifery and pharmacy. An audience response system was used to collect the students' data on HR, epidemiology, technical problems, satisfaction and orthostatic symptoms. As primary outcome, we investigated the students' correct performance of the modified AST regarding textbook conformity of the HR data. Secondary outcomes included technical feasibility, the students' satisfaction and consistency of HR data within predefined subgroups (e.g., gender, curricula). Descriptive statistics are reported.

RESULTS

The class was completed by 217 students (mean age: 23 ± 8 [SD], 81% female, 19% male). Mean reported rise of HR during standing was ~ 20 bpm (~ 30%) which is highly concordant to textbooks. Reported feasibility (~ 80% negated any technical issues) and students' satisfaction (4.4 on 5-point Likert-scale) were high. The HR data were consistent within the subgroups.

CONCLUSION

This study demonstrates that the highly relevant BRR can be successfully addressed in an e-learning based asynchronous laboratory class implementing a non-supervised AST restricted to HR measurements embedded in digital material. The robust HR response and the adjustable complexity allow an application to different healthcare-related curricula. This class, therefore, provides a broad audience access to a fundamental concept of cardiovascular physiology.

Learning Theory-Driven Tips for Designing Effective Learning Solutions for the Continuous Education of Community Pharmacists to Enhance Patient-Centered Care—A Qualitative Study

The constant development of medical and pharmaceutical sciences and the changing roles of pharmacists highlight the importance of lifelong learning in their profession. Given the identified knowledge gaps in the literature in terms of pharmacists’ preferences for lifelong learning, the study aimed to evaluate the opinions and attitudes of community pharmacists towards lifelong learning, including their previous experiences and educational needs, in order to propose evidence-based tips for designing such solutions and interventions intended for them both in face-to-face and online forms. For this purpose, ten semi-structured in-depth interviews were conducted with Polish community pharmacists on the topic using a thematic guide. Subsequently, they were subjected to literal transcription and interpretative phenomenological analysis by two independent researchers using phenomenology as the qualitative approach. The identified themes covered the topic’s relevance for pharmacists’ work, practice-oriented form and content, previous learners’ experiences as a foundation for further learning, commercial initiatives’ risks, motivation sources, and barriers for participation in lifelong learning solutions so far. Based on the insights provided by the respondents, as well as scientifically proven learning theories and educational principles, ten tips were formulated for designing recipient-friendly learning solutions and interventions within the framework of postgraduate lifelong learning of pharmacists.

E-learning in clinical microbiology and infectious diseases

BACKGROUND

Becoming and staying competent is a challenge in clinical microbiology and infectious diseases because of dramatic increases in medical knowledge, discovery of new pathogens, emerging infections, new resistance mechanisms and laboratory techniques. E-learning is an effective way of meeting educational needs by providing more efficient and flexible training. E-learning resources have become more important to acquire new knowledge and skills, especially at a time of physical distancing.

OBJECTIVES

This review aims to summarize the implementation of e-learning in clinical microbiology and infectious diseases with references to existing examples and resources.

SOURCES

Literature and online resources for e-learning, online teaching/education in medical education, clinical microbiology and infectious diseases.

CONTENT

The principles and common methods of e-learning and frequently used digital tools are described. For all aspects of e-learning/distance learning, available resources and examples of applications in clinical microbiology and infectious diseases are presented.

IMPLICATIONS

The techniques, tools and resources described in this article should be considered for the development and implementation of e-learning programmes in clinical microbiology and infectious disease training.

Development of an Instructional Design Evaluation Survey for Postgraduate Medical E-Learning: Content Validation Study

Background

E-Learning has taken a firm place in postgraduate medical education. Whereas 10 years ago it was promising, it now has a definite niche and is clearly here to stay. However, evaluating the effect of postgraduate medical e-learning (PGMeL) and improving upon it can be complicated. While the learning aims of e-learning are evaluated, there are no instruments to evaluate the instructional design of PGMeL. Such an evaluation instrument may be developed by following the Association for Medical Education in Europe (AMEE) 7-step process. The first 5 steps of this process were previously performed by literature reviews, focus group discussion, and an international Delphi study.

Objective

This study will continue with steps 6 and 7 and answer the research question: Is a content-validated PGMeL evaluation survey useful, understandable, and of added value for creators of e-learning?

Methods

There are five phases in this study: creating a survey from 37 items (phase A); testing readability and question interpretation (phase B); adjusting, rewriting, and translating surveys (phase C); gathering completed surveys from three PGMeL modules (phase D); and holding focus group discussions with the e-learning authors (phase E). Phase E was carried out by presenting the results of the evaluations from phase D, followed by a group discussion. There are four groups of participants in this study. Groups A and B are experienced end users of PGMeL and participated in phase B. Group C are users who undertook e-learning and were asked to complete the survey in phase D. Group D are the authors of the e-learning modules described above.

Results

From a list of 36 items, we developed a postgraduate Medical E-Learning Evaluation Survey (MEES). Seven residents participated in the phase B group discussion: 4 items were interpreted differently, 3 were not readable, and 2 items were double. The items from phase B were rewritten and, after adjustment, understood correctly. The MEES was translated into Dutch and again pilot-tested. All items were clear and were understood correctly. The MEES version used for the evaluation contained 3 positive domains (motivation, learning enhancers, and real-world translation) and 2 negative domains (barriers and learning discouragers), with 36 items in those domains, 5 Likert scale questions of 1 to 10, and 5 open questions asking participants to give their own comments in each domain. Three e-learning modules were evaluated from July to November 2018. There were a total of 158 responses from a Dutch module, a European OB/GYN (obstetrics and gynecology) module, and a surgical module offered worldwide. Finally, 3 focus group discussions took place with a total of 10 participants. Usefulness was much appreciated, understandability was good, and added value was high. Four items needed additional explanation by the authors, and a Creators’ Manual was written at their request.

Conclusions

The MEES is the first survey to evaluate the instructional design of PGMeL and was constructed following all 7 steps of the AMEE. This study completes the design of the survey and shows its usefulness and added value to the authors. It finishes with a final, publicly available survey that includes a Creators’ Manual. We briefly discuss the number of responses needed and conclude that more is better; in the end, however, one has to work with what is available. The next steps would be to see whether improvement can be measured by using the MEES and continue to work on the end understandability in different languages and cultural groups.

Influencing factors in the implementation of postgraduate medical e-learning: a thematic analysis

BACKGROUND

Postgraduate medical e-learning (PGMeL) is being progressively used and evaluated. Its impact continues to grow, yet there are barriers to its implementation. Although more attention is now being paid to quality evaluation models, little has been written about the successful implementation of PGMeL. This study aims to determine factors and define themes influencing the successful implementation of PGMeL.

METHODS

We performed 10 semi-structured interviews with experienced e-learning creators, after which we carried out a thematic analysis to name and describe factors and themes.

RESULTS

Although this was not the objective of the study, the participants stressed the importance of a definition of success. Associated with this definition were: reaching your target audience, achieving learning aims, satisfying your audience and maintaining continuity. Three themes were identified containing eleven factors that influence successful implementation. The themes were named and defined after the group that had the most influence on the factors. We named them creator-, organization- and learner-dependent factors. The creator dependent factors are: the learning aim, pedagogical strategies, content expertise, evaluation and the creators motivational path. The organization dependent factors are management support, recourse and culture. Finally, the learner dependent factors are technology, motivators/barriers and value.

CONCLUSIONS

This study shows that implementing PGMeL has creator-, organization- and learner-dependent factors which should be taken into account during the creating of the PGMeL. Although creator- and learner-dependent factors are mentioned in other studies, the present study also stresses the importance of organization-dependent factors. Innovation implementation theories such as Rogers' diffusion of innovation or Kotter's eight steps of change management show a great overlap with these factors. Future studies can both evaluate the use of these innovation models in creating PGMeL and assess the effect of the organizational factors in greater depth.

Digital Learning to Improve Safe and Effective Prescribing: A Systematic Review

With the aim to modernize and harmonize prescribing education, the European Association for Clinical Pharmacology and Therapeutics (EACPT) Working Group on education recommended the extensive use and distribution of digital learning resources (DLRs). However, it is unclear whether the complex task of prescribing medicine can be taught digitally. Therefore, the aim of this review was to investigate the effect of diverse DLRs in clinical pharmacology and therapeutics education. Databases PubMed, EMBASE, CINAHL, ERIC, and CENTRAL were systematically searched. Sixty-five articles were included in the analyses. Direct effects on patients were studied, but not detected, in six articles. Skills and behavior were studied in 11 articles, 8 of which reported positive effects. Knowledge acquisition was investigated in 19 articles, all with positive effects. Qualitative analyses yielded 10 recommendations for the future development of DLRs. Digital learning is effective in teaching knowledge, attitudes, and skills associated with safe and effective prescribing.

A 9-Step Theory- and Evidence-Based Postgraduate Medical Digital Education Development Model: Empirical Development and Validation

BACKGROUND

Digital education tools (e-learning, technology-enhanced learning) can be defined as any educational intervention that is electronically mediated. Decveloping and applying such tools and interventions for postgraduate medical professionals who work and learn after graduation can be called postgraduate medical digital education (PGMDE), which is increasingly being used and evaluated. However, evaluation has focused mainly on reaching the learning goals and little on the design. Design models for digital education (instructional design models) help educators create a digital education curriculum, but none have been aimed at PGMDE. Studies show the need for efficient, motivating, useful, and satisfactory digital education.

OBJECTIVE

Our objective was (1) to create an empirical instructional design model for PGMDE founded in evidence and theory, with postgraduate medical professionals who work and learn after graduation as the target audience, and (2) to compare our model with existing models used to evaluate and create PGMDE.

METHODS

Previously we performed an integrative literature review, focus group discussions, and a Delphi procedure to determine which building blocks for such a model would be relevant according to experts and users. This resulted in 37 relevant items. We then used those 37 items and arranged them into chronological steps. After we created the initial 9-step plan, we compared these steps with other models reported in the literature.

RESULTS

The final 9 steps were (1) describe who, why, what, (2) select educational strategies, (3) translate to the real world, (4) choose the technology, (5) complete the team, (6) plan the budget, (7) plan the timing and timeline, (8) implement the project, and (9) evaluate continuously. On comparing this 9-step model with other models, we found that no other was as complete, nor were any of the other models aimed at PGMDE.

CONCLUSIONS

Our 9-step model is the first, to our knowledge, to be based on evidence and theory building blocks aimed at PGMDE. We have described a complete set of evidence-based steps, expanding a 3-domain model (motivate, learn, and apply) to an instructional design model that can help every educator in creating efficient, motivating, useful, and satisfactory PGMDE. Although certain steps are more robust and have a deeper theoretical background in current research (such as education), others (such as budget) have been barely touched upon and should be investigated more thoroughly in order that proper guidelines may also be provided for them.

How We Evaluate Postgraduate Medical E-Learning: Systematic Review

BACKGROUND

Electronic learning (e-learning) in postgraduate medical education has seen a rapid evolution; however, we tend to evaluate it only on its primary outcome or learning aim, whereas its effectiveness also depends on its instructional design. We believe it is important to have an overview of all the methods currently used to evaluate e-learning design so that the preferred method may be identified and the next steps needed to continue to evaluate postgraduate medical e-learning may be outlined.

OBJECTIVE

This study aimed to identify and compare the outcomes and methods used to evaluate postgraduate medical e-learning.

METHODS

We performed a systematic literature review using the Web of Science, PubMed, Education Resources Information Center, and Cumulative Index of Nursing and Allied Health Literature databases. Studies that used postgraduates as participants and evaluated any form of e-learning were included. Studies without any evaluation outcome (eg, just a description of e-learning) were excluded.

RESULTS

The initial search identified 5973 articles, of which we used 418 for our analysis. The types of studies were trials, prospective cohorts, case reports, and reviews. The primary outcomes of the included studies were knowledge, skills, and attitude. A total of 12 instruments were used to evaluate a specific primary outcome, such as laparoscopic skills or stress related to training. The secondary outcomes mainly evaluated satisfaction, motivation, efficiency, and usefulness. We found 13 e-learning design methods across 19 studies (4% 19/418). The methods evaluated usability, motivational characteristics, and the use of learning styles or were based on instructional design theories, such as Gagne's instructional design, the Heidelberg inventory, Kern's curriculum development steps, and a scale based on the cognitive load theory. Finally, 2 instruments attempted to evaluate several aspects of a design, based on the experience of creating e-learning.

CONCLUSIONS

Evaluating the effect of e-learning design is complicated. Given the diversity of e-learning methods, there are many ways to carry out such an evaluation, and probably, many ways to do so correctly. However, the current literature shows us that we have yet to reach any form of consensus about which indicators to evaluate. There is a great need for an evaluation tool that is properly constructed, validated, and tested. This could be a more homogeneous way to compare the effects of e-learning and for the authors of e-learning to continue to improve their product.

Consensus on Quality Indicators of Postgraduate Medical E-Learning: Delphi Study

BACKGROUND

The progressive use of e-learning in postgraduate medical education calls for useful quality indicators. Many evaluation tools exist. However, these are diversely used and their empirical foundation is often lacking.

OBJECTIVE

We aimed to identify an empirically founded set of quality indicators to set the bar for “good enough” e-learning.

METHODS

We performed a Delphi procedure with a group of 13 international education experts and 10 experienced users of e-learning. The questionnaire started with 57 items. These items were the result of a previous literature review and focus group study performed with experts and users. Consensus was met when a rate of agreement of more than two-thirds was achieved.

RESULTS

In the first round, the participants accepted 37 items of the 57 as important, reached no consensus on 20, and added 15 new items. In the second round, we added the comments from the first round to the items on which there was no consensus and added the 15 new items. After this round, a total of 72 items were addressed and, of these, 37 items were accepted and 34 were rejected due to lack of consensus.

CONCLUSIONS

This study produced a list of 37 items that can form the basis of an evaluation tool to evaluate postgraduate medical e-learning. This is, to our knowledge, the first time that quality indicators for postgraduate medical e-learning have been defined and validated. The next step is to create and validate an e-learning evaluation tool from these items.