Improving Graduate Medical Education by Aggregating Data Across the Medical Education Continuum

ABSTRACT

Meaningful improvements to graduate medical education (GME) have been achieved in recent decades, yet many GME improvement pilots have been small trials without rigorous outcome measures and with limited generalizability. Thus, lack of access to large-scale data is a key barrier to generating empiric evidence to improve GME. In this article, the authors examine the potential of a national GME data infrastructure to improve GME, review the output of 2 national workshops on this topic, and propose a path toward achieving this goal.The authors envision a future where medical education is shaped by evidence from rigorous research powered by comprehensive, multi-institutional data. To achieve this goal, premedical education, undergraduate medical education, GME, and practicing physician data must be collected using a common data dictionary and standards and longitudinally linked using unique individual identifiers. The envisioned data infrastructure could provide a foundation for evidence-based decisions across all aspects of GME and help optimize the education of individual residents.Two workshops hosted by the National Academies of Sciences, Engineering, and Medicine Board on Health Care Services explored the prospect of better using GME data to improve education and its outcomes. There was broad consensus about the potential value of a longitudinal data infrastructure to improve GME. Significant obstacles were also noted.Suggested next steps outlined by the authors include producing a more complete inventory of data already being collected and managed by key medical education leadership organizations, pursuing a grass-roots data sharing pilot among GME-sponsoring institutions, and formulating the technical and governance frameworks needed to aggregate data across organizations.The power and potential of big data is evident across many disciplines, and the authors believe that harnessing the power of big data in GME is the best next step toward advancing evidence-based physician education.

Remote learning developments in postgraduate medical education in response to the COVID-19 pandemic - A BEME systematic review: BEME Guide No. 71

BACKGROUND

Prior reviews investigated medical education developments in response to COVID-19, identifying the pivot to remote learning as a key area for future investigation. This review synthesized online learning developments aimed at replacing previously face-to-face 'classroom' activities for postgraduate learners.

METHODS

Four online databases (CINAHL, Embase, PsychINFO, and PubMed) and MedEdPublish were searched through 21 December 2020. Two authors independently screened titles, abstracts and full texts, performed data extraction, and assessed risk of bias. The PICRAT technology integration framework was applied to examine how teachers integrated and learners engaged with technology. A descriptive synthesis and outcomes were reported. A thematic analysis explored limitations and lessons learned.

RESULTS

Fifty-one publications were included. Fifteen collaborations were featured, including international partnerships and national networks of program directors. Thirty-nine developments described pivots of existing educational offerings online and twelve described new developments. Most interventions included synchronous activities (n Fif5). Virtual engagement was promoted through chat, virtual whiteboards, polling, and breakouts. Teacher's use of technology largely replaced traditional practice. Student engagement was largely interactive. Underpinning theories were uncommon. Quality assessments revealed moderate to high risk of bias in study reporting and methodology. Forty-five developments assessed reaction; twenty-five attitudes, knowledge or skills; and two behavior. Outcomes were markedly positive. Eighteen publications reported social media or other outcomes, including reach, engagement, and participation. Limitations included loss of social interactions, lack of hands-on experiences, challenges with technology and issues with study design. Lessons learned highlighted the flexibility of online learning, as well as practical advice to optimize the online environment.

CONCLUSIONS

This review offers guidance to educators attempting to optimize learning in a post-pandemic world. Future developments would benefit from leveraging collaborations, considering technology integration frameworks, underpinning developments with theory, exploring additional outcomes, and designing and reporting developments in a manner that supports replication.

Pivot to online learning for adapting or continuing workplace-based clinical learning in medical education following the COVID-19 pandemic: A BEME systematic review: BEME Guide No. 70

BACKGROUND

The novel coronavirus disease was declared a pandemic in March 2020, which necessitated adaptations to medical education. This systematic review synthesises published reports of medical educational developments and innovations that pivot to online learning from workplace-based clinical learning in response to the pandemic. The objectives were to synthesise what adaptations/innovation were implemented (description), their impact (justification), and 'how' and 'why' these were selected (explanation and rationale).

METHODS

The authors systematically searched four online databases up to December 21, 2020. Two authors independently screened titles, abstracts and full-texts, performed data extraction, and assessed the risk of bias. Our findings are reported in alignment with the STORIES (STructured apprOach to the Reporting in healthcare education of Evidence Synthesis) statement and BEME guidance.

RESULTS

Fifty-five articles were included. Most were from North America (n = 40), and nearly 70% focused on undergraduate medical education (UGME). Key developments were rapid shifts from workplace-based learning to virtual spaces, including online electives, telesimulation, telehealth, radiology, and pathology image repositories, live-streaming or pre-recorded videos of surgical procedures, stepping up of medical students to support clinical services, remote adaptations for clinical visits, multidisciplinary team meetings and ward rounds. Challenges included lack of personal interactions, lack of standardised telemedicine curricula and need for faculty time, technical resources, and devices. Assessment of risk of bias revealed poor reporting of underpinning theory, resources, setting, educational methods, and content.

CONCLUSIONS

This review highlights the response of medical educators in deploying adaptations and innovations. Whilst few are new, the complexity, concomitant use of multiple methods and the specific pragmatic choices of educators offers useful insight to clinical teachers who wish to deploy such methods within their own practice. Future works that offer more specific details to allow replication and understanding of conceptual underpinnings are likely to justify an update to this review.

Online learning developments in undergraduate medical education in response to the COVID-19 pandemic: A BEME systematic review: BEME Guide No. 69

BACKGROUND

The COVID-19 pandemic spurred an abrupt transition away from in-person educational activities. This systematic review investigated the pivot to online learning for nonclinical undergraduate medical education (UGME) activities and explored descriptions of educational offerings deployed, their impact, and lessons learned.

METHODS

The authors systematically searched four online databases and conducted a manual electronic search of MedEdPublish up to December 21, 2020. Two authors independently screened titles, abstracts and full texts, performed data extraction and assessed risk of bias. A third author resolved discrepancies. Findings were reported in accordance with the STORIES (STructured apprOach to the Reporting in healthcare education of Evidence Synthesis) statement and BEME guidance.

RESULTS

Fifty-six articles were included. The majority (n = 41) described the rapid transition of existing offerings to online formats, whereas fewer (n = 15) described novel activities. The majority (n = 27) included a combination of synchronous and asynchronous components. Didactics (n = 40) and small groups (n = 26) were the most common instructional methods. Teachers largely integrated technology to replace and amplify rather than transform learning, though learner engagement was often interactive. Thematic analysis revealed unique challenges of online learning, as well as exemplary practices. The quality of study designs and reporting was modest, with underpinning theory at highest risk of bias. Virtually all studies (n = 54) assessed reaction/satisfaction, fewer than half (n = 23) assessed changes in attitudes, knowledge or skills, and none assessed behavioral, organizational or patient outcomes.

CONCLUSIONS

UGME educators successfully transitioned face-to-face instructional methods online and implemented novel solutions during the COVID-19 pandemic. Although technology's potential to transform teaching is not yet fully realized, the use of synchronous and asynchronous formats encouraged virtual engagement, while offering flexible, self-directed learning. As we transition from emergency remote learning to a post-pandemic world, educators must underpin new developments with theory, report additional outcomes and provide details that support replication.

Emergency Medicine Virtual Conference Participants’ Engagement with Competing Activities

Introduction: Residency didactic conferences transitioned to a virtual format during the COVID-19 pandemic. This format creates questions about effective educational practices, which depend on learner engagement. In this study we sought to characterize the competitive demands for learner attention during virtual didactics and to pilot methodology for future studies. Methods: This was a prospective, observational, cohort study of attendees at virtual didactics from a single emergency medicine residency, which employed a self-report strategy informed by validated classroom assessments of student engagement. We deployed an online, two-question survey polling across six conference days using random signaled sampling. Participants reported all activities during the preceding five minutes. Results: There were 1303 responses over 40 survey deployments across six nonadjacent days. Respondents were residents (63.4%); faculty (27.5%); fellows (2.3%); students (2%); and others (4.8%). Across all responses, about 85% indicated engagement in the virtual conference within the last five minutes of the polls. The average number of activities engaged in was 2.0 (standard deviation = 1.1). Additional activities included education-related (34.2%), work-related (21.1%), social (18.8%), personal (14.6%), self-care (13.4%), and entertainment (4.4%). Conclusion: Learners engage in a variety of activities during virtual didactics. Engagement appears to fluctuate temporally, which may inform teaching strategies. This information may also provide unique instructor feedback. This pilot study demonstrates methodology for future studies of conference engagement and learning outcomes.

No time, no room: On-shift teaching for any shift

It can be difficult to carve out dedicated time to teach in the busy environment of the Emergency Department. With some intention, you can find educational opportunities in every shift. Here we highlight some of those opportunities, with careful attention to those without special equipment or preparation required. These can be used to teach less traditional practice points such as interprofessional communication as well as clinical pearls. Select the moments that make the most sense for you, your trainees, and your shift.

Coping with COVID-19: medical students as strong and responsible stewards of their education

BACKGROUND

Due to the COVID-19 pandemic, clinical rotations at the University of Michigan Medical School (UMMS) were suspended on March 17, 2020, per the Association of American Medical Colleges' recommendations. No alternative curriculum existed to fill the educational void for clinical students. The traditional approach to curriculum development was not feasible during the pandemic as faculty were redeployed to clinical care, and the immediate need for continued learning necessitated a new model.

APPROACH

One student developed an outline for an online course on pandemics based on peer-to-peer conversations regarding learners' interests and needs, and she proposed that students author the content given the immediate need for a curriculum. Fifteen student volunteers developed content to fill knowledge gaps, and expert faculty reviewers confirmed that the student authors had successfully curated a comprehensive curriculum.

EVALUATION

The crowdsourced student content coalesced into a 40-hour curriculum required for all 371 clinical-level students at UMMS. This student-driven effort took just 17 days from outline to implementation, and the final product is a full course comprising five modules, multiple choice questions, discussion boards, and assignments. Learners were surveyed to gauge success, and 93% rated this content as relevant to all medical students.

REFLECTION

The successful implementation of this model for curriculum development, grounded in the Master Adaptive Learner framework, suggests that medical students can be entrusted as stewards of their own education. As we return to a post-pandemic "normal," this approach could be applied to the maintenance and de novo development of future curricula.

An update on developments in medical education in response to the COVID-19 pandemic: A BEME scoping review: BEME Guide No. 64

BACKGROUND

COVID-19 has fundamentally altered how education is delivered. Gordon et al. previously conducted a review of medical education developments in response to COVID-19; however, the field has rapidly evolved in the ensuing months. This scoping review aims to map the extent, range and nature of subsequent developments, summarizing the expanding evidence base and identifying areas for future research.

METHODS

The authors followed the five stages of a scoping review outlined by Arskey and O'Malley. Four online databases and MedEdPublish were searched. Two authors independently screened titles, abstracts and full texts. Included articles described developments in medical education deployed in response to COVID-19 and reported outcomes. Data extraction was completed by two authors and synthesized into a variety of maps and charts.

RESULTS

One hundred twenty-seven articles were included: 104 were from North America, Asia and Europe; 51 were undergraduate, 41 graduate, 22 continuing medical education, and 13 mixed; 35 were implemented by universities, 75 by academic hospitals, and 17 by organizations or collaborations. The focus of developments included pivoting to online learning (n = 58), simulation (n = 24), assessment (n = 11), well-being (n = 8), telehealth (n = 5), clinical service reconfigurations (n = 4), interviews (n = 4), service provision (n = 2), faculty development (n = 2) and other (n = 9). The most common Kirkpatrick outcome reported was Level 1, however, a number of studies reported 2a or 2b. A few described Levels 3, 4a, 4b or other outcomes (e.g. quality improvement).

CONCLUSIONS

This scoping review mapped the available literature on developments in medical education in response to COVID-19, summarizing developments and outcomes to serve as a guide for future work. The review highlighted areas of relative strength, as well as several gaps. Numerous articles have been written about remote learning and simulation and these areas are ripe for full systematic reviews. Telehealth, interviews and faculty development were lacking and need urgent attention.

Approach to Acute Weakness

Weakness has a broad differential diagnosis. A paradigm for organizing possibilities is to consider what part of the nervous system is involved, ranging from brain, spinal cord, nerve roots, and peripheral nerves to the neuromuscular junction. The clinician can consider internal versus external causes. Some neurologic conditions have subtle presentations yet carry a risk of short-term decompensation if not recognized. It is helpful to consider whether an emergency department presentation of weakness is a new disease process or represents an exacerbation of an established condition. Emergency presentations of weakness are challenging, and one must carefully consider potential serious causes.

Population screening for von Willebrand disease in adolescents with heavy menstrual bleeding

We used Ohio Medicaid data to determine frequency and predictors of laboratory screening for von Willebrand disease and other bleeding disorders in adolescents with heavy menses. Despite expert guidelines, screening was performed in only a minority of subjects (<15% of those with heavy menses and <25% of those with menstrual bleeding defined as severe), with younger adolescents and those from more urban counties more likely to undergo screening.