Context matters: emergent variability in an effectiveness trial of online teaching modules

Rethinking Informed Consent as an EPA in Surgery

ABSTRACT

Over the past decade, entrustable professional activities (EPAs) have become an important element in the competency-based medical education movement. In this Commentary, the authors explore informed consent as an EPA within resident surgical training. In doing so, they foreground the concept of culture and reexamine the nature of trust and entrustment decisions from within a cultural framework. The authors identify role modeling and professional identity formation as core elements in the training process and suggest that faculty are sometimes better off using these tools than uncritically adopting a formal EPA framework for what is, in essence, a professionally oriented and values-based moral enterprise. They conclude that EPAs work best when they are developed at a local level, stressing the unique culture of specialty and program as well as the care that must be taken when attempting to transfer notions of entrustment from the undergraduate medical education level to graduate medical education settings.

Online learning in Health Professions Education. Part 1: Teaching and learning in online environments: AMEE Guide No. 161

Online learning in Health Professions Education (HPE) has been evolving over decades, but COVID-19 changed its use abruptly. Technology allowed necessary HPE during COVID-19, but also demonstrated that many HP educators and learners had little knowledge and experience of these complex sociotechnical environments. Due to the educational benefits and flexibility that technology can afford, many higher education experts agree that online learning will continue and evolve long after COVID-19. As HP educators stand at the crossroads of technology integration, it is important that we examine the evidence, theories, advantages/disadvantages, and pedagogically informed design of online learning. This Guide will provide foundational concepts and practical strategies to support HPE educators and institutions toward advancing pedagogically informed use of online HPE. This Guide consists of two parts. The first part will provide an overview of evidence, theories, formats, and educational design in online learning, including contemporary issues and considerations such as learner engagement, faculty development, inclusivity, accessibility, copyright, and privacy. The second part (to be published as a separate Guide) focuses on specific technology tool types with practical examples for implementation and integration of the concepts discussed in Guide 1, and will include digital scholarship, learning analytics, and emerging technologies. In sum, both guides should be read together, as Guide 1 provides the foundation required for the practical application of technology showcased in Guide 2.Please refer to the video abstract for Part 1 of this Guide at https://bit.ly/AMEEGuideOnlineLearning.

A GP's clinical reasoning in the context of multimorbidity: beyond the perception of an intuitive approach

INTRODUCTION

GP's clinical reasoning processes in the context of patients suffering from multimorbidity are often a process which remains implicit. Therefore, the goal of this case study analysis is to gain a better understanding of the processes at play in the management of patients suffering from multimorbidity.

METHODS

A case study analysis, using a qualitative thematic analysis was conducted. This case follows a 54-year-old woman who has been under the care of her GP for almost 10 years and suffers from a number of chronic conditions. The clinical reasoning of an experienced GP who can explicitly unfold his processes was chosen for this case analysis.

RESULTS

Four main themes emerged from this case analysis: The different roles that GPs have to manage; the GP's cognitive flexibility and continual adaptation of their clinical reasoning processes, the patient's empowerment, and the challenges related to the collaboration with specialists and healthcare professionals.

CONCLUSION

This could help GPs gain a clearer understanding of their clinical reasoning processes and motivate them to communicate their findings with others during clinical supervision or teaching. Furthermore, this may emphasize the importance of valuing the role of the primary care physician in the management of multimorbid patients.

Rethinking Context in Continuing Professional Development: From Identifying Barriers to Understanding Social Dynamics

INTRODUCTION

For continuing professional development (CPD) to reach its potential to improve outcomes requires an understanding of the role of context and the influencing conditions that enable interventions to succeed. We argue that the heuristic use of frameworks to design and implement interventions tends to conceptualize context as defined lists of barriers, which may obscure consideration of how different contextual factors interact with and intersect with each other.

METHODS

We suggest a framework approach that would benefit from postmodernist theory that explores how ideologies, meanings, and social structures in health care settings shape social practices. As an illustrative example, we conducted a Foucauldian discourse analysis of diabetes care to make visible how the social, historical, and political conditions in which clinicians experience, practice, and shape possibilities for behavior change.

RESULTS

The discursive construction of continuing education as a knowledge translation mechanism assumes and is contingent on family physicians to implement guidelines. However, they enact other discursively constituted roles that may run in opposition. This paradoxical position creates a tension that must be navigated by family physicians, who may perceive it possible to provide good care without necessarily implementing guidelines.

DISCUSSION

We suggest marrying "framework" thinking with postmodernist theory that explores how ideologies, meanings, and social structures shape practice behavior change. Such a proposed reconceptualization of context in the design of continuing professional development interventions could provide a more robust and nuanced understanding of how the dynamic relationships and interactions between clinicians, patients, and their work environments shape educational effectiveness.

"Important but risky": attitudes of global thought leaders towards cost and value research in health professions education

Studies of cost and value can inform educational decision making, yet our understanding of the barriers to such research is incomplete. To address this gap, our aim was to explore the attitudes of global thought leaders in HPE towards cost and value research. This was a qualitative virtual interview study underpinned by social constructionism. In telephone or videoconference interviews in 2018-2019, we asked global healthcare professional thought leaders their views regarding HPE cost and value research, outstanding research questions in this area and why addressing these questions was important. Analysis was inductive and thematic, and incorporated review and comments from the original interviewees (member checking). We interviewed 11 thought leaders, nine of whom gave later feedback on our data interpretation (member checking). We identified four themes: Cost research is really important but potentially risky (quantifying and reporting costs provides evidence for decision-making but could lead to increased accountability and loss of autonomy); I don't have the knowledge and skills (lack of economic literacy); it's not what I went into education research to do (professional identity); and it's difficult to generate generalizable findings (the importance of context). This study contributes to a wider conversation in the literature about cost and value research by bringing in the views of global HPE thought leaders. Our findings provide insight to inform how best to engage and empower educators and researchers in the processes of asking and answering meaningful, acceptable and relevant cost and value questions in HPE.

Live in Front of Students Teaching Sessions (LISTS): a Novel Learning Experience from Jordan During the COVID-19 Pandemic

Objective

The COVID-19 pandemic uncovered innovative approaches in medical education. Modifications are needed to overcome the drawbacks of pure online teaching. Our study aimed at testing a hybrid method of live online practical anatomy sessions in which an element of face-to-face teacher-student interaction is maintained.

Methods

We performed an experiment with a one-group design in which medical and medical laboratory sciences students were taught different practical anatomy topics using either purely online or live in front of students teaching sessions (LISTS). Students' performance and perceptions were quantitatively assessed.

Results

For 108 medical laboratory sciences students, the mean quiz scores were significantly higher for the topics taught by the LISTS approach (p = 0.025). For two groups of 13 and 17 medical students, the performance in exams was significantly higher for the topics taught using the LISTS method (p = 0.000 and 0.011, respectively) with large effect sizes. Students' perceptions of preference, enjoyment, and satisfaction were all in favor of LISTS.

Conclusions

Our results confirmed that keeping at least a minimum of interaction between the teacher and students can have a significant improvement in the performance and engagement in practical anatomy sessions for health professionals. The results indicate that the extra effort of LISTS was worth it.

Two sides of the same coin: Quality improvement and program evaluation in health professions education

Health professions education is in constant pursuit of new ways of teaching and assessment in order to improve the training of healthcare professionals. Educators are often challenged with designing, implementing, and evaluating programs in the context of their professional practice, particularly those in response to dynamic and emerging social needs. This article explores the synergies and intersections of two approaches-quality improvement and program evaluation-and the potential utility of their combinations within our field to design, evaluate, and most importantly, improve educational programming. We argue that the inclusion of established quality improvement frameworks within program evaluation provides a proven mechanism for driving change, can optimize programming within the multi-contextual education systems, and, ultimately, that these two approaches are complementary to one another. These combinations hold great promise for optimizing programming in alignment with social missions, where it has been difficult for institutions worldwide to generate and capture evidence of social accountability.

The curious case of case study research

The conceptualisation of 'good' medical education research as hypothesis testing to identify universal truths that are generalisable across contexts has been challenged. Joining this conversation, the field of health professions education research is complex and contextual and there are ways of examining and reporting locally based activities and innovations, which can be of general value. This position leads to a focus on case study research (CSR), inquiry bound in time and place that generates thick descriptions and close interpretations to reach explanations. CSR has grown in sophistication in recent years and can inform practice and advance the science of medical and health professions education. The authors evaluated the current state of the science of CSR in the medical education literature by identifying and reviewing 160 papers. Most articles presented as 'case studies' were not in fact CSR. Moreover, most articles failed to go beyond a 'we did this' account. The authors explore definitions of CSR, and they examine dominant CSR methodologists, Yin, Stake and Merriam, and their respective approaches to CSR. They then set out some of the basic tenets of CSR (case definition, methods of data collection and analysis) and consider the logics of CSR (its structures, purposes, assumptions and symbols). CSR challenges are considered next (such as emic and etic perspectives; ethical complexities; generalisability; quality; and reporting and reflexivity). The authors conclude that context is a mechanism, which needs to be understood, and rigorous CSR provides the structures and criticality to do so, opening up new areas of understanding and inquiry.

Is Digital Animation Superior to Text Resources for Facial Transplantation Education? A Randomized Controlled Trial

BACKGROUND

Facial transplantation is a complex surgical procedure that requires education of medical professionals, patients, and the general public. The objective of this randomized controlled trial was to compare the educational efficacy of facial transplantation digital animation versus text resources for teaching medical students a complex surgical procedure and its applicable anatomical and surgical principles.

METHODS

Medical student volunteers were recruited and prospectively randomized to an animation or a text group. Students completed preintervention and postintervention assessments of confidence and knowledge of head and neck anatomy and facial transplantation. Student Evaluations of Educational Quality survey assessed student satisfaction with both tools following study crossover. Knowledge assessment was developed using National Board of Medical Examiners guidelines and content validated by four experts. The Cohen d statistic was used to measure the effect size of each intervention.

RESULTS

Thirty-six students were assigned randomly to the animation group (n = 19) or the text (n = 17) group. Postintervention analysis demonstrated significantly higher performance scores in the animation group compared to the text group (p = 0.029). The animation group had a larger effect size (Cohen d = 1.96 versus 1.27). Only students in the animation group reported significantly improved confidence in head and neck anatomy after the intervention (p = 0.002). Both groups demonstrated significantly increased postintervention confidence in knowledge of facial transplantation (p < 0.001). Satisfaction scores revealed that students significantly favored the animation resource (p < 0.001).

CONCLUSIONS

Animation can be an effective tool for facial transplantation education and results in improved student knowledge, confidence, and satisfaction compared to text resources. Facial transplantation can also be used as an educational framework for teaching medical students anatomical and surgical principles.

Implementing Systematic Faculty Development to Support an EPA-Based Program of Assessment: Strategies, Outcomes, and Lessons Learned

Problem: Development of a novel, competency-based program of assessment requires creation of a plan to measure the processes that enable successful implementation. The principles of implementation science outline the importance of considering key drivers that support and sustain transformative change within an educational program. The introduction of Entrustable Professional Activities (EPAs) as a framework for assessment has underscored the need to create a structured plan to prepare assessors to engage in a new paradigm of assessment. Although approaches to rater training for workplace-based assessments have been described, specific strategies to prepare assessors to apply standards related to the level of supervision a student needs have not been documented. Intervention: We describe our systematic approach to prepare assessors, faculty and postgraduate trainees, to complete EPA assessments for medical students during the clerkship phase of our curriculum. This institution-wide program is designed to build assessors' skills in direct observation of learners during authentic patient encounters. Assessors apply new knowledge and practice skills in using established performance expectations to determine the level of supervision a learner needs to perform clinical tasks. Assessors also learn to provide feedback and narrative comments to coach students and promote their ongoing clinical development. Data visualizations for assessors facilitate reinforcement of the tenets learned during training. Collaborative learning and peer feedback during faculty development sessions promote the formation of a community of practice among assessors. Context: Faculty development for assessors was implemented in advance of implementation of the EPA program. Assessors in the program include residents/fellows who work closely with students, faculty with discipline-specific expertise and a group of experienced clinicians who were selected to serve as experts in competency-based EPA assessments, the Master Assessors. Training focused on creating a shared understanding about the application of criteria used to evaluate student performance. EPA assessments based on the AAMC's Core Entrustable Professional Activities for Entering Residency, were completed in nine core clerkships. EPA assessments included a supervision rating based on a modified scale for use in undergraduate medical education. Impact: Data from EPA assessments completed during the first year of the program were analyzed to evaluate the effectiveness of the faculty development activities implemented to prepare assessors to consistently apply standards for assessment. A systematic approach to training and attention to critical drivers that enabled institution-wide implementation, led to consistency in the supervision rating for students' first EPA assessment completed by any type of assessor, ratings by assessors done within a specific clinical context, and ratings assigned by a group of specific assessors across clinical settings. Lessons learned: A systematic approach to faculty development with a willingness to be flexible and reach potential participants using existing infrastructure, can facilitate assessors' engagement in a new culture of assessment. Interaction among participants during training sessions not only promotes learning but also contributes to community building. A leadership group responsible to oversee faculty development can ensure that the needs of stakeholders are addressed and that a change in assessment culture is sustained.

Publishing during COVID-19: Lessons for health professions education research

Want to know how COVID‐19 changes health professional education research priorities? Our Editor‐in‐chief argues that in the most important ways it shouldn't.

Critical Realism and Realist Inquiry in Medical Education

Understanding complex interventions, such as in medical education, requires a philosophy of science that can explain how and why things work, or fail to work, in different contexts. Critical realism and its operationalization in the form of realist inquiry provides this explanatory power. Ontologically, critical realism posits that the social world is real, that it exists independent of our knowledge of it, and that it is driven by causal mechanisms. However, unlike postpositivism, a realist epistemological position is that our understanding of the mechanisms that underlay social reality is limited and subjective. Critical realism is focused on understanding the mechanisms that drive social reality even when they are not directly observable. One of the most commonly used methodologies in the critical realist paradigm is realist inquiry, which focuses on the relationships between context, mechanisms, and outcomes. At its core, realist inquiry is concerned with "What works for whom, under what circumstances, how, and why?" To that end, realist inquiry explores the mechanisms that drive social systems and the ways in which these mechanisms work to develop explanatory theories of the phenomena under consideration. Although, compared with other approaches, realist inquiry is relatively new in medical education, the value of realist inquiry is in its ability to model how complex interventions function differently across multiple contexts, explaining what works, how it works, for whom, and in what contexts.

Embracing standardisation and contextualisation in medical education

CONTEXT

The tensions that emerge between the universal and the local in a global world require continuous negotiation. However, in medical education, standardization and contextual diversity tend to operate as separate philosophies, with little attention to the interplay between them.

METHODS

The authors synthesise the literature related to the intersections and resulting tensions between standardization and contextual diversity in medical education. In doing so, the authors analyze the interplay between these competing concepts in two domains of medical education (admissions and competency-based medical education), and provide concrete examples drawn from the literature.

RESULTS

Standardization offers many rewards: its common articulations and assumptions promote patient safety, foster continuous quality improvement, and enable the spread of best practices. Standardization may also contribute to greater fairness, equity, reliability and validity in high stakes processes, and can provide stakeholders, including the public, with tangible reassurance and a sense of the stable and timeless. At the same time, contextual variation in medical education can afford myriad learning opportunities, and it can improve alignment between training and local workforce needs. The inevitable diversity of contexts for learning and practice renders any absolute standardization of programs, experiences, or outcomes an impossibility.

CONCLUSIONS

The authors propose a number of ways to examine the interplay of contextual diversity and standardization and suggest three ways to move beyond an either/or stance. In reconciling the laudable goals of standardization and the realities of the innumerable contexts in which we train and deliver care, we are better positioned to design and deliver a medical education system that is globally responsible and locally engaged.

Surgery Website as a 24/7 Adjunct to a Surgical Curriculum

OBJECTIVE

Successfully teaching duty hour restricted trainees demands engaging learning opportunities. Our surgical educational website and its associated assets were assessed to understand how such a resource was being used.

DESIGN

Our website was accessible to all Mayo Clinic employees via the internal web network. Website access data from April 2015 through October 2016 were retrospectively collected using Piwik.

SETTING

Academic, tertiary care referral center with a large general surgery training program. Mayo Clinic, Rochester, MN.

PARTICIPANTS

A total of 257 Mayo Clinic employees used the website.

RESULTS

The website had 48,794 views from 6313 visits by 257 users who spent an average of 14 ± 11 minutes on the website. Our website houses 295 videos, 51 interactive modules, 14 educational documents, and 7 flashcard tutorials. The most popular content type was videos, with a total of 30,864 views. The most popular visiting time of the day was between 8 pm and 9 pm with 6358 views (13%), and Thursday was the most popular day with 17,907 views (37%).  A total of 78% of users accessed content beyond the homepage. Average visits peaked in relation to 2 components of our curriculum: a 240% increase one day before our biannual intern simulation assessments, and a 61% increase one day before our weekly conducted Friday simulation sessions. Interns who rotated on the service of the staff surgeon who actively endorses the website had 93% more actions per visit as compared to other users. The highest clicks were on the home banner for our weekly simulation session pre-emptive videos, followed by "groin anatomy," and "TEP hernia repair" videos.

CONCLUSIONS

Our website acted as a "just-in-time" accessible portal to reliable surgical information. It supplemented the time sensitive educational needs of our learners by serving as a heavily used adjunct to 3 components of our surgical education curriculum: weekly simulation sessions, biannual assessments, and clinical rotations.

"Yes, and …" Exploring the Future of Learning Analytics in Medical Education

This Conversations Starter article presents a selected research abstract from the 2017 Association of American Medical Colleges Northeastern Region Group on Educational Affairs annual spring meeting. The abstract is paired with the integrative commentary of three experts who shared their thoughts stimulated by the study. Commentators brainstormed "what's next" with learning analytics in medical education, including advancements in interaction metrics and the use of interactivity analysis to deepen understanding of perceptual, cognitive, and social learning and transfer processes.

Learning in professionally 'distant' contexts: opportunities and challenges

The changing nature of healthcare education and delivery is such that clinicians will increasingly find themselves practicing in contexts that are physically and/or conceptually different from the settings in which they were trained, a practice that conflicts on some level with socio-cultural theories of learning that emphasize learning in context. Our objective was therefore to explore learning in 'professionally distant' contexts. Using paramedic education, where portions of training occur in hospital settings despite preparing students for out-of-hospital work, fifty-three informants (11 current students, 13 recent graduates, 16 paramedic program faculty and 13 program coordinators/directors) took part in five semi-structured focus groups. Participants reflected on the value and role of hospital placements in paramedic student development. All sessions were audio recorded, transcribed verbatim and analyzed using inductive thematic analysis. In this context six educational advantages and two challenges were identified when using professionally distant learning environments. Learning could still be associated with features such as (a) engagement through "authenticity", (b) technical skill development, (c) interpersonal skill development, (d) psychological resilience, (e) healthcare system knowledge and (f) scaffolding. Variability in learning and misalignment with learning goals were identified as potential threats. Learning environments that are professionally distant from eventual practice settings may prove meaningful by providing learners with foundational and preparatory learning experiences for competencies that may be transferrable. This suggests that where learning occurs may be less important than how the experience contributes to the learner's development and the meaning or value he/she derives from it.

Faculty Perceptions of Online Teaching Effectiveness and Indicators of Quality

Online education programs in nursing are increasing rapidly. Faculty need to be competent in their role and possess the skills necessary to positively impact student outcomes. Existing research offers effective teaching strategies for online education; however, there may be some disconnect in the application of these strategies and faculty perceptions of associated outcomes. Focus groups were formed to uncover how nursing faculty in an online program define and describe teaching effectiveness and quality indicators in an asynchronous online environment. A semistructured interview format guided group discussion. Participants (n = 11) included nurse educators from an online university with an average of 15 years of experience teaching in nursing academia and 6 years in an online environment. Teaching effectiveness, indicators of quality, and student success were three categories that emerged from the analysis of data. What materialized from the analysis was an overarching concept of a “dance” that occurs in the online environment. Effective online teachers facilitate, connect, lead, and work in synchrony with students to obtain indicators of quality such as student success, student improvement over time, and student application of knowledge to the professional role.

A Big Data and Learning Analytics Approach to Process-Level Feedback in Cognitive Simulations

Collecting and analyzing large amounts of process data for the purposes of education can be considered a big data/learning analytics (BD/LA) approach to improving learning. However, in the education of health care professionals, the application of BD/LA is limited to date. The authors discuss the potential advantages of the BD/LA approach for the process of learning via cognitive simulations. Using the lens of a cognitive model of radiograph interpretation with four phases (orientation, searching/scanning, feature detection, and decision making), they reanalyzed process data from a cognitive simulation of pediatric ankle radiography where 46 practitioners from three expertise levels classified 234 cases online. To illustrate the big data component, they highlight the data available in a digital environment (time-stamped, click-level process data). Learning analytics were illustrated using algorithmic computer-enabled approaches to process-level feedback.For each phase, the authors were able to identify examples of potentially useful BD/LA measures. For orientation, the trackable behavior of re-reviewing the clinical history was associated with increased diagnostic accuracy. For searching/scanning, evidence of skipping views was associated with an increased false-negative rate. For feature detection, heat maps overlaid on the radiograph can provide a metacognitive visualization of common novice errors. For decision making, the measured influence of sequence effects can reflect susceptibility to bias, whereas computer-generated path maps can provide insights into learners' diagnostic strategies.In conclusion, the augmented collection and dynamic analysis of learning process data within a cognitive simulation can improve feedback and prompt more precise reflection on a novice clinician's skill development.

Teaching Child and Adolescent Psychiatry in the Twenty-First Century: A Reflection on the Role of Technology in Education

Technology has become an integral part of everyday life and is starting to shape the landscape of graduate medical education. This article reviews the use of technology in teaching child and adolescent psychiatry (CAP) fellows, and 3 main aspects are considered. The first aspect is use of technology to enhance active learning. The second aspect covers technology and administrative tasks, and the third aspect is the development of a technology curriculum for CAP trainees. The article concludes with a brief review of some of the challenges and pitfalls that have to be considered and recommendations for future research.

Mapping the dark matter of context: a conceptual scoping review

CONTEXT

Like dark matter, the contexts for medical education are largely invisible to those within them, although context can have profound influences on teaching, learning and practice. For something that is so intrinsic to the field of medical education, the concept of context remains troubling to scholars and those running medical education programmes. This paper reports on a critical and conceptual review of the concept of context within the medical education literature and beyond.

METHODS

A review was undertaken drawing on two sources: concepts of context in the medical education literature, and concepts of context across multiple academic disciplines. This body of material was iteratively, discursively and inductively synthesised.

RESULTS

Few of the articles from the medical education literature described or defined context directly, tending instead to focus on describing specific elements of context, such as clinical disciplines, physical settings and political pressures, that could or did influence learning outcomes. The results were framed in terms of what context 'is', how context works (in terms of context-mechanism-outcome), and how context can be represented using patterns. The authors propose a definition of context in medical education, along with the means to model, contrast and compare different contexts based on recurring patterns.

CONCLUSIONS

Context matters in medical education and it can, despite many challenges, be considered systematically and objectively. The findings from this study both represent a catalyst and challenge medical education researchers to look at context afresh.

Observational analysis of near-peer and faculty tutoring in problem-based learning groups

CONTEXT

Near-peer and faculty staff tutors may facilitate problem-based learning (PBL) through different means. Near-peer tutors are thought to compensate for their lack of subject matter expertise with greater adeptness at group facilitation and a better understanding of their learners. However, theoretical explanations of tutor effectiveness have been developed largely from recollections of tutor practices gathered through student evaluation surveys, focus groups and interviews. A closer look at what happens during PBL sessions tutored by near-peers and faculty members seems warranted to augment theory from a grounded perspective.

METHODS

We conducted an observational study to explore interactional practices during PBL tutorials at our medical school, at which near-peer tutoring of Year 2 students is an established practice. Between October 2014 and May 2015, video-recordings were made of nine purposively sampled tutor groups using three tutor types (near-peer, clinical faculty and basic science faculty staff) across three systems-based units. An investigator team comprising a Year 2 student, a Year 4 student and a behavioural scientist independently analysed the videos until their observations reached saturation and then met face to face to discuss their detailed field notes.

RESULTS

Through constant comparison, narratives of tutor practices and group dynamics were generated for each of the nine tutor groups, representing the collective impressions of the members of the investigator team.

CONCLUSIONS

Variation was greater within than across tutor types. Tutors' practices idiosyncratically and sometimes substantially diverged from PBL principles, yet all tutors attempted to convey authority or 'insider' status with respect to the short- and long-term goals of medical education. Students prompted these status demonstrations by expressing gratitude, asking questions and exhibiting analogous status demonstrations themselves. Understanding the socio-cognitive nature of tutoring from a grounded perspective may provide a means to develop faculty staff of all types to better meet learner needs in a principled fashion.

Evaluating structured assessment of anaesthesiologists' non-technical skills

BACKGROUND

Non-technical skills (NTS) are essential for safe and efficient anaesthesia. Assessment instruments with appropriate validity evidence can be used to ensure that anaesthesiologists possess the NTS necessary to deliver high-standard patient care. The aims were to collect validity evidence using a contemporary validity framework for the assessment instrument Anaesthesiologists' Non-Technical Skills in Denmark (ANTSdk) regarding response process and internal structure (including reliability), and to investigate the effect of rater training on these properties.

METHODS

An explorative study was undertaken at the Danish Institute for Medical Simulation, Copenhagen, Denmark. In a 1-day session, using ANTSdk, a convenience sample of 19 anaesthesiologists rated trainee anaesthesiologists' NTS in nine video-recorded simulation scenarios before and after a 3-h training session.

RESULTS

Response process evidence: participants considered ANTSdk useful and feasible for NTS assessment. Internal structure evidence: inter-rater reliability (single measures) largely expressed substantial agreement (ICC ≥ 0.55 and ICC ≥ 0.60 for pre- and post-training ratings respectively). Strong internal consistency of ratings was found (Spearman's correlation coefficient ≥ 0.82). Accuracy of participants' ratings compared with reference ratings (± 1 scale point) was notable (76% and 78% for pre- and post-training ratings, respectively). The results indicate that the elements 'Demonstrating self-awareness', 'Reassessing decisions', 'Assessing competencies', and 'Supporting others' need more attention in future rater training.

CONCLUSION

The validity evidence collected on content, response process, and internal structure, suggests that ANTSdk is easy to use on video-recorded simulation scenarios, indicating that ANTSdk is a feasible instrument for NTS assessment during anaesthesia training.

Exploring Medical Student Learning Needs in the Pediatric Emergency Department: "What Do You Want to Learn Right Now?"

BACKGROUND

When precepting medical students in the emergency department, faculty physicians often have only minutes per patient encounter to devote to direct teaching. Instructional strategies that promote independent student learning after each case may have merit. It is not known, however, to what degree patient-triggered learning needs are amenable to independent study.

OBJECTIVE

The aims of this study were to determine self- and faculty-reported learner needs at the time of patient encounters and to assess the degree to which these perceived needs may be satisfied by independent study.

METHODS

We interviewed medical students and faculty in our pediatric emergency department. Immediately before or after they saw a patient, we asked "What do you want to learn right now?". For half of the student interviews, we separately asked the same questions of their preceptors. Interviews were taped and transcribed. Responses were coded by 3 investigators who did content analysis to identify dominant themes and the extent to which the learning need could be addressed independently. Investigators agreed that Accreditation Council for Graduate Medical Education competency domains could be used to classify the responses.

RESULTS

We interviewed 82 students and 44 preceptors yielding 126 patient-triggered learning needs. Competency area(s) were medical knowledge (70), patient care (1), interpersonal skills (27), systems-based practice (2), practice-based learning (3), and professionalism (4). Two raters independently assigned the same competency in 89%. Medical knowledge competency learning needs were almost all at least moderately amenable to independent learning (68/70, 98%), but the other competencies were not (22/57, 39%) according to the raters (interrater reliability, 0.7). Preceptor responses were congruent in competency type with students' responses in 29 (67%) of 43. Students listed interpersonal skills deficits far more often than did faculty (24% vs 5%, P < 0.05).

CONCLUSIONS

Most student learning needs in the pediatric emergency department focus on medical knowledge. These deficits could be amenable to structured independent study at the point of care.

Exploring patterns and pattern languages of medical education

CONTEXT

The practices and concepts of medical education are often treated as global constants even though they can take many forms depending on the contexts in which they are realised. This represents challenges in presenting and appraising medical education research, as well as in translating practices and concepts between different contexts. This paper explores the problem and seeks to respond to its challenges.

METHODS

This paper explores the application of architectural theorist Christopher Alexander's work on patterns and pattern languages to medical education. The authors review the underlying concepts of patterns and pattern language, they consider the development of pattern languages in medical education, they suggest possible applications of pattern languages for medical education and they discuss the implications of such use. Examples are drawn from across the field of medical education.

RESULTS

The authors argue that the deliberate and systematic use of patterns and pattern languages in describing medical educational activities, systems and contexts can help us to make sense of the world, and the pattern languages of medical education have the potential to advance understanding and scholarship in medical education, to drive innovation and to enable critical engagement with many of the underlying issues in this field.

Getting Real: Embracing the Conditions of the Third-Year Clerkship and Reimagining the Curriculum to Enable Deliberate Practice

There are many calls in the literature for changes in how medical students are educated. Although many curricular innovations have been attempted, a look at the theory behind how complex skills are learned provides useful information to guide new curriculum developments. The requirement of deliberate practice as the road map for success in the learning of clinical skills suggests that perhaps the current clinical milieu is not an optimal place for medical students to learn. The idiosyncrasy inherent in the dramatically changed medical landscape of the last 20 years makes it difficult for such practice to occur; the apprentice model of legitimate peripheral participation in a community of practice as it used to exist does no longer. Indeed, current workplace environments are at odds with the needs of medical students. Overwhelming numbers of goals and objectives in existing third-year clerkships serve as wish lists of what students should learn. They should be replaced by a systematic, longitudinal curriculum in which all students can be guaranteed to have encountered the core clinical competencies as defined. Moving the goals and objectives of the current clerkships to a longitudinal, spiral curricular format frees up clinical time in the third year to be used for students to find their future specialty and socialize into medicine. Doing so allows for an opportunity for students to spend extended time in areas of their interest. Moving to such a new curriculum format maximizes and optimizes learning while embracing the reality of current clinical workplace environments.

Learning curves in health professions education

Learning curves, which graphically show the relationship between learning effort and achievement, are common in published education research but are not often used in day-to-day educational activities. The purpose of this article is to describe the generation and analysis of learning curves and their applicability to health professions education. The authors argue that the time is right for a closer look at using learning curves-given their desirable properties-to inform both self-directed instruction by individuals and education management by instructors.A typical learning curve is made up of a measure of learning (y-axis), a measure of effort (x-axis), and a mathematical linking function. At the individual level, learning curves make manifest a single person's progress towards competence including his/her rate of learning, the inflection point where learning becomes more effortful, and the remaining distance to mastery attainment. At the group level, overlaid learning curves show the full variation of a group of learners' paths through a given learning domain. Specifically, they make overt the difference between time-based and competency-based approaches to instruction. Additionally, instructors can use learning curve information to more accurately target educational resources to those who most require them.The learning curve approach requires a fine-grained collection of data that will not be possible in all educational settings; however, the increased use of an assessment paradigm that explicitly includes effort and its link to individual achievement could result in increased learner engagement and more effective instructional design.

Medical Student and Tutor Perceptions of Video Versus Text in an Interactive Online Virtual Patient for Problem-Based Learning: A Pilot Study

Background

The impact of the use of video resources in primarily paper-based problem-based learning (PBL) settings has been widely explored. Although it can provide many benefits, the use of video can also hamper the critical thinking of learners in contexts where learners are developing clinical reasoning. However, the use of video has not been explored in the context of interactive virtual patients for PBL.

Objective

A pilot study was conducted to explore how undergraduate medical students interpreted and evaluated information from video- and text-based materials presented in the context of a branched interactive online virtual patient designed for PBL. The goal was to inform the development and use of virtual patients for PBL and to inform future research in this area.

Methods

An existing virtual patient for PBL was adapted for use in video and provided as an intervention to students in the transition year of the undergraduate medicine course at St George’s, University of London. Survey instruments were used to capture student and PBL tutor experiences and perceptions of the intervention, and a formative review meeting was run with PBL tutors. Descriptive statistics were generated for the structured responses and a thematic analysis was used to identify emergent themes in the unstructured responses.

Results

Analysis of student responses (n=119) and tutor comments (n=18) yielded 8 distinct themes relating to the perceived educational efficacy of information presented in video and text formats in a PBL context. Although some students found some characteristics of the videos beneficial, when asked to express a preference for video or text the majority of those that responded to the question (65%, 65/100) expressed a preference for text. Student responses indicated that the use of video slowed the pace of PBL and impeded students’ ability to review and critically appraise the presented information.

Conclusions

Our findings suggest that text was perceived to be a better source of information than video in virtual patients for PBL. More specifically, the use of video was perceived as beneficial for providing details, visual information, and context where text was unable to do so. However, learner acceptance of text was higher in the context of PBL, particularly when targeting clinical reasoning skills. This pilot study has provided the foundation for further research into the effectiveness of different virtual patient designs for PBL.

Virtual patient activity patterns for clinical learning

BACKGROUND

Virtual patients are software tools that present learners with patient case situations and tasks. Some virtual patients take the learner through a guided case scenario, whereas others require learners to make diagnostic and therapeutic decisions. Much attention has been paid to the design of virtual patients and their use as standalone activities, but rather less attention has been paid to their use in broader educational activities. This article describes a series of activity patterns that make use of virtual patients.

CONTEXT

The article describes five patterns of clinical teaching activities that make use of virtual patients: independent study activities; collaborative group activities; blended activities; bridging activities; and reference activities. These patterns were developed inductively from the authors' teaching practices over a number of years. These are not the only activity patterns and designs that can make use of virtual patients but they are ones that have been found to be particularly useful over time and in many different contexts.

INNOVATION

Although the design of educational artifacts such as virtual patients is important, clinical teachers also need to consider the ways in which they are used. Different kinds of activity can employ different kinds of virtual patients of varying levels of complexity. An activity focus can allow clinical teachers to make more effective and broader use of virtual patients.

IMPLICATIONS

Virtual patients can be used for more than independent study. Clinical teachers are encouraged to explore the multitude of uses that virtual patients can be put to, and the ways in which activities can be constructed around them. Different kinds of activity can employ different kinds of virtual patients, of varying levels of complexity.

Conceptualization and Reporting of Context in the North American Continuing Medical Education Literature: A Scoping Review Protocol

Within continuing medication education (CME), it has been argued that an "authentic" clinical context should be built into CME activities for knowledge to be effectively translated into clinical practice. However, although context is considered significant in the success (or lack thereof) of an intervention, there is a lack of consensus on what exactly context is. This scoping review arises from concerns surrounding the opaque, complex, and potentially problematic relationship between context and the effective design and implementation of CME interventions. In this article, we present a protocol for examining how context is discussed within the CME literature. The specific purpose of this scoping review is to summarize the breadth of existing evidence on context within the North American CME literature. The scoping review methodology will also highlight gaps in the current literature, which can inform future research endeavors.

Evaluating technology-enhanced learning: A comprehensive framework

BACKGROUND

The absence of a standard, comprehensive approach to evaluating technology-enhanced learning (TEL) limits the utility of individual evaluations, and impedes the integration and synthesis of results across studies.

PURPOSE

To outline a comprehensive framework for approaching TEL evaluation in medical education, and to develop instruments for measuring the perceptions of TEL learners and instructors.

METHODS AND RESULTS

Using both theoretical constructs of inquiry in education and a synthesis of existing models and instruments, we outlined a general model for evaluation that links utility, principles, and practices. From this we derived a framework for TEL evaluation that identifies seven data collection activities: needs analysis; documentation of processes, decisions, and final product; usability testing; observation of implementation; assessment of participant experience; assessment of learning outcomes; and evaluation of cost, reusability, and sustainability. We then used existing quality standards and approaches to develop instruments for assessing the experiences of learners and instructors using TEL.

CONCLUSIONS

No single evaluation is likely to collect all of this information, nor would any single audience likely find all information elements equally useful. However, consistent use of a common evaluation framework across different courses and institutions would avoid duplication of effort and allow cross-course comparisons.

Deciding 'what to teach' health professionals: a human-centred systems engineering perspective

CONTEXT

Recent calls propose the conceptualisation of medical education research as 'an improvement science for complex social interventions'. This involves developing principled, yet contextually grounded, descriptions of health care practice that increase the likelihood of successful intervention. Defining what health professionals should be taught using theoretical perspectives and analytical techniques borrowed from human-centred systems engineering (HCSE) may acknowledge this call by allowing learning objectives and performance assessment criteria to be aligned with the demands of actual work.

DISCUSSION

Human-centred systems engineering is a multidisciplinary endeavour that seeks to promote the safe, efficient and productive performance of socio-technological systems. Systems theories in HCSE explain how environmental conditions constrain and afford human goal-directed behaviour and are modified by such. Many of the techniques used in HCSE research that are applicable to examining health care practice should be familiar to medical education researchers. This method differs from other empirical approaches that have been applied to the study of health care practice in its emphasis on practical problem solving via intervention design.

CONCLUSIONS

Learning objectives and performance assessment criteria derived from an HCSE perspective target people's attunement to environmental conditions as they strive to enact goal-directed behaviour. Implementing educational interventions from an HCSE perspective should facilitate a sustained positive impact across contexts because theories of person-environment interaction enable principled adaptations of interventions to local circumstances.

Virtual patients as activities: exploring the research implications of an activity theoretical stance

Virtual patients are computer-based simulators of patient encounters for the purposes of instruction, practice, and assessment. Although virtual patients have been around for some time they have yet to become part of mainstream medical education. A major reason for this would seem to be a lack of clarity as to what educational value virtual patients actually have. This paper argues that virtual patients should be seen as activities rather than artifacts and that activity theory can be used to generate different ways to frame scholarship in and around virtual patients. Drawing on the work of Leont'ev and Engeström this paper describes a range of perspectives based on the operations, actions, and objectives in and around virtual patients; the use of virtual patients to mediate activities; and the sociocultural context and the participants in virtual patient activities. This approach allows us to move beyond the 'does or does not work' discourse of much of the existing scholarship around virtual patients and, to an extent, around educational technologies as a whole. Activity perspectives, and activity theory in particular, offer new horizons for research and evaluation that address many of the limitations of intervention-based paradigms of inquiry.