Effect of Teaching Bayesian Methods Using Learning by Concept vs Learning by Example on Medical Students' Ability to Estimate Probability of a Diagnosis: A Randomized Clinical Trial

Importance

Clinicians use probability estimates to make a diagnosis. Teaching students to make more accurate probability estimates could improve the diagnostic process and, ultimately, the quality of medical care.

Objective

To test whether novice clinicians can be taught to make more accurate bayesian revisions of diagnostic probabilities using teaching methods that apply either explicit conceptual instruction or repeated examples.

Design, Setting, and Participants

A randomized clinical trial of 2 methods for teaching bayesian updating and diagnostic reasoning was performed. A web-based platform was used for consent, randomization, intervention, and testing of the effect of the intervention. Participants included 61 medical students at McMaster University and Eastern Virginia Medical School recruited from May 1 to September 30, 2018.

Interventions

Students were randomized to (1) receive explicit conceptual instruction regarding diagnostic testing and bayesian revision (concept group), (2) exposure to repeated examples of cases with feedback regarding posttest probability (experience group), or (3) a control condition with no conceptual instruction or repeated examples.

Main Outcomes and Measures

Students in all 3 groups were tested on their ability to update the probability of a diagnosis based on either negative or positive test results. Their probability revisions were compared with posttest probability revisions that were calculated using the Bayes rule and known test sensitivity and specificity.

Results

Of the 61 participants, 22 were assigned to the concept group, 20 to the experience group, and 19 to the control group. Approximate age was 25 years. Two participants were first-year; 37, second-year; 12, third-year; and 10, fourth-year students. Mean (SE) probability estimates of students in the concept group were statistically significantly closer to calculated bayesian probability than the other 2 groups (concept, 0.4%; [0.7%]; experience, 3.5% [0.7%]; control, 4.3% [0.7%]; P < .001). Although statistically significant, the differences between groups were relatively modest, and students in all groups performed better than expected, based on prior reports in the literature.

Conclusions and Relevance

The study showed a modest advantage for students who received theoretical instruction on bayesian concepts. All participants' probability estimates were, on average, close to the bayesian calculation. These findings have implications for how to teach diagnostic reasoning to novice clinicians.

Trial Registration

ClinicalTrials.gov identifier: NCT04130607.

The state of health advocacy training in postgraduate medical education: a scoping review

CONTEXT

Health advocacy is an essential component of postgraduate medical education, and is part of many physician competency frameworks such as the Canadian Medical Education Directives for Specialists (CanMEDS) roles. There is little consensus about how advocacy should be taught and assessed in the postgraduate context. There are no consolidated guides to assist in the design and implementation of postgraduate health advocacy curricula.

OBJECTIVES

This scoping review aims to identify and analyse existing literature pertaining to health advocacy education and assessment in postgraduate medicine. We specifically sought to summarise themes from the literature that may be useful to medical educators to inform further health advocacy curriculum interventions.

METHODS

MEDLINE, Embase and ERIC were searched using MeSH (medical student headings) and non-MeSH search terms. Additional articles were found using forward snowballing. The grey literature search included Google and relevant stakeholder websites, regulatory bodies, physician associations, government agencies and academic institutions. We followed a stepwise scoping review methodology, followed by thematic analysis using an inductive approach.

RESULTS

Of the 123 documents reviewed in full, five major themes emerged: (i) conceptions of health advocacy have evolved towards advocating with rather than for patients, communities and populations; (ii) longitudinal curricula were less common but appeared the most promising, often linked to scholarly or policy objectives; (iii) hands-on, immersive opportunities build competence and confidence; (iv) community-identified needs and partnerships are increasingly considered in designing curriculum, and (v) resident-led and motivated programmes appear to engage residents and allow for achievement of stated outcomes. There remain significant challenges to assessment of health advocacy competencies, and assessment tools for macro-level health advocacy were notably absent.

CONCLUSIONS

There is considerable heterogeneity in the way health advocacy is taught, assessed and incorporated into postgraduate curricula across programmes and disciplines. We consolidated recommendations from the literature to inform further health advocacy curriculum design, implementation and assessment.

Clinical chemistry score versus high-sensitivity cardiac troponin I and T tests alone to identify patients at low or high risk for myocardial infarction or death at presentation to the emergency department

BACKGROUND

Testing for high-sensitivity cardiac troponin (hs-cTn) may assist triage and clinical decision-making in patients presenting to the emergency department with symptoms of acute coronary syndrome; however, this could result in the misclassification of risk because of analytical variation or laboratory error. We sought to evaluate a new laboratory-based risk-stratification tool that incorporates tests for hs-cTn, glucose level and estimated glomerular filtration rate to identify patients at risk of myocardial infarction or death when presenting to the emergency department.

METHODS

We constructed the clinical chemistry score (CCS) (range 0-5 points) and validated it as a predictor of 30-day myocardial infarction (MI) or death using data from 4 cohort studies involving patients who presented to the emergency department with symptoms suggestive of acute coronary syndrome. We calculated diagnostic parameters for the CCS score separately using high-sensitivity cardiac troponin I (hs-cTnI) and high-sensitivity cardiac troponin T (hs-cTnT).

RESULTS

For the combined cohorts (n = 4245), 17.1% of participants had an MI or died within 30 days. A CCS score of 0 points best identified low-risk participants: the hs-cTnI CCS had a sensitivity of 100% (95% confidence interval [CI] 99.5%-100%), with 8.9% (95% CI 8.1%-9.8%) of the population classified as being at low risk of MI or death within 30 days; the hs-cTnT CCS had a sensitivity of 99.9% (95% CI 99.2%-100%), with 10.5% (95% CI 9.6%-11.4%) of the population classified as being at low risk. The CCS had better sensitivity than hs-cTn alone (hs-cTnI < 5 ng/L: 96.6%, 95% CI 95.0%-97.8%; hs-cTnT < 6 ng/L: 98.2%, 95% CI 97.0%-99.0%). A CCS score of 5 points best identified patients at high risk (hs-cTnI CCS: specificity 96.6%, 95% CI 96.0%-97.2%; 11.2% [95% CI 10.3%-12.2%] of the population classified as being at high risk; hs-cTnT CCS: specificity 94.0%, 95% CI 93.1%-94.7%; 13.1% [95% CI 12.1%-14.1%] of the population classified as being at high risk) compared with using the overall 99th percentiles for the hs-cTn assays (specificity of hs-cTnI 93.2%, 95% CI 92.3-94.0; specificity of hs-cTnT 73.8%, 95% CI 72.3-75.2).

INTERPRETATION

The CCS score at the chosen cut-offs was more sensitive and specific than hs-cTn alone for risk stratification of patients presenting to the emergency department with suspected acute coronary syndrome. Study registration: ClinicalTrials.gov, nos. NCT01994577; NCT02355457.

Management Reasoning: Implications for Health Professions Educators and a Research Agenda

Substantial research has illuminated the clinical reasoning processes involved in diagnosis (diagnostic reasoning). Far less is known about the processes entailed in patient management (management reasoning), including decisions about treatment, further testing, follow-up visits, and allocation of limited resources. The authors' purpose is to articulate key differences between diagnostic and management reasoning, implications for health professions education, and areas of needed research.Diagnostic reasoning focuses primarily on classification (i.e., assigning meaningful labels to a pattern of symptoms, signs, and test results). Management reasoning involves negotiation of a plan and ongoing monitoring/adjustment of that plan. A diagnosis can usually be established as correct or incorrect, whereas there are typically multiple reasonable management approaches. Patient preferences, clinician attitudes, clinical contexts, and logistical constraints should not influence diagnosis, whereas management nearly always involves prioritization among such factors. Diagnostic classifications do not necessarily require direct patient interaction, whereas management prioritizations require communication and negotiation. Diagnoses can be defined at a single time point (given enough information), whereas management decisions are expected to evolve over time. Finally, management is typically more complex than diagnosis.Management reasoning may require educational approaches distinct from those used for diagnostic reasoning, including teaching distinct skills (e.g., negotiating with patients, tolerating uncertainty, and monitoring treatment) and developing assessments that account for underlying reasoning processes and multiple acceptable solutions.Areas of needed research include if and how cognitive processes differ for management and diagnostic reasoning, how and when management reasoning abilities develop, and how to support management reasoning in clinical practice.

Experienced physician descriptions of intuition in clinical reasoning: a typology

Background

Diagnostic intuition is a rapid, non-analytic, unconscious mode of reasoning. A small body of evidence points to the ubiquity of intuition, and its usefulness in generating diagnostic hypotheses and ascertaining severity of illness. Little is known about how experienced physicians understand this phenomenon, and how they work with it in clinical practice.

Methods

Descriptions of how experienced physicians perceive their use of diagnostic intuition in clinical practice were elicited through interviews conducted with 30 physicians in emergency, internal and family medicine. Each participant was asked to share stories of diagnostic intuition, including times when intuition was both correct and incorrect. Multiple coders conducted descriptive analysis to analyze the salient aspects of these stories.

Results

Physicians provided descriptions of what diagnostic intuition is, when it occurs and what type of activity it prompts. From stories of correct intuition, a typology of four different types of intuition was identified: Sick/Not Sick, Something Not Right, Frame-shifting and Abduction. Most physician accounts of diagnostic intuition linked this phenomenon to non-analytic reasoning and emphasized the importance of experience in developing a trustworthy sense of intuition that can be used to effectively engage analytic reasoning to evaluate clinical evidence.

Conclusions

The participants recounted myriad stories of diagnostic intuition that alerted them to unusual diagnoses, previous diagnostic error or deleterious trajectories. While this qualitative study can offer no conclusions about the representativeness of these stories, it suggests that physicians perceive clinical intuition as beneficial for correcting and advancing diagnoses of both common and rare conditions.

Debiasing versus knowledge retrieval checklists to reduce diagnostic error in ECG interpretation

There is an ongoing debate regarding the cause of diagnostic errors. One view is that errors result from unconscious application of cognitive heuristics; the alternative is that errors are a consequence of knowledge deficits. The objective of this study was to compare the effectiveness of checklists that (a) identify and address cognitive biases or (b) promote knowledge retrieval, as a means to reduce errors in ECG interpretation. Novice postgraduate year (PGY) 1 emergency medicine and internal medicine residents (n = 40) and experienced cardiology fellows (PGY 4-6) (n = 21) were randomly allocated to three conditions: a debiasing checklist, a content (knowledge) checklist, or control (no checklist) to be used while interpreting 20 ECGs. Half of the ECGs were deliberately engineered to predispose to bias. Diagnostic performance under either checklist intervention was not significantly better than the control. As expected, more errors occurred when cases were designed to induce bias (F = 96.9, p < 0.0001). There was no significant interaction between the instructional condition and level of learner. Checklists attempting to help learners identify cognitive bias or mobilize domain-specific knowledge did not have an overall effect in reducing diagnostic errors in ECG interpretation, although they may help novices. Even when cognitive biases are deliberately inserted in cases, cognitive debiasing checklists did not improve participants' performance.

Attending Emergency Physicians' Perceptions of a Programmatic Workplace-Based Assessment System: The McMaster Modular Assessment Program (McMAP)

Construct: The McMaster Modular Assessment Program (McMAP) is a programmatic workplace-based assessment (WBA) system that provides emergency medicine trainees with competency judgments through frequent task-specific and global daily assessments. Background: The longevity of McMAP relative to other programmatic WBA systems affords a unique view that precedes large-scale transitions to competency-based medical education (CBME), particularly in North America. Although prior work has described the perspective of residents using this system, the in-depth experiences of assessors using the system have yet to be explored. This perspective is important for understanding the validity of the competency judgments the system produces. Approach: We conducted a qualitative study that used semi-structured interviews analyzed using interpretive description (Thorne) to explore 16 attending physicians' experiences using McMAP. Data analysis was completed independently by 2 researchers, who met regularly to discuss codes and resolve any disagreements. Results: Having a structured assessment framework for a range of clinical tasks has helped encourage what attendings perceived to be more frequent and better-quality assessments, with the added advantages of being holistic, flexible, and learner-driven. However, attendings also perceived a number of challenges of McMAP and programmatic WBA more broadly. These included a reluctance to give and to document negative feedback, "gaming" of the system by both attendings and residents, and a variety of logistic and technology-related concerns. Conclusions: Based on our findings, we offer several key recommendations that can help programs maximize the benefits of programmatic WBA as they transition to CBME.

Correction to: Debiasing versus knowledge retrieval checklists to reduce diagnostic error in ECG interpretation

Due to an unfortunate turn of events, Fig. 3 was omitted from the original publication.

A Core Components Framework for Evaluating Implementation of Competency-Based Medical Education Programs

PURPOSE

The rapid adoption of competency-based medical education (CBME) provides an unprecedented opportunity to study implementation. Examining "fidelity of implementation"-that is, whether CBME is being implemented as intended-is hampered, however, by the lack of a common framework. This article details the development of such a framework.

METHOD

A two-step method was used. First, a perspective indicating how CBME is intended to bring about change was described. Accordingly, core components were identified. Drawing from the literature, the core components were organized into a draft framework. Using a modified Delphi approach, the second step examined consensus amongst an international group of experts in CBME.

RESULTS

Two different viewpoints describing how a CBME program can bring about change were found: production and reform. Because the reform model was most consistent with the characterization of CBME as a transformative innovation, this perspective was used to create a draft framework. Following the Delphi process, five core components of CBME curricula were identified: outcome competencies, sequenced progression, tailored learning experiences, competency-focused instruction, and programmatic assessment. With some modification in wording, consensus emerged amongst the panel of international experts.

CONCLUSIONS

Typically, implementation evaluation relies on the creation of a specific checklist of practices. Given the ongoing evolution and complexity of CBME, this work, however, focused on identifying core components. Consistent with recent developments in program evaluation, where implementation is described as a developmental trajectory toward fidelity, identifying core components is presented as a fundamental first step toward gaining a more sophisticated understanding of implementation.

Understanding diagnosis through ACTion: evaluation of a point-of-care checklist for junior emergency medical residents

Background Avoiding or correcting a diagnostic error first requires identification of an error and perhaps deciding to revise a diagnosis, but little is known about the factors that lead to revision. Three aspects of reflective practice, seeking Alternative explanations, exploring the Consequences of missing these alternative diagnoses, identifying Traits that may contradict the provisional diagnosis, were incorporated into a three-point diagnostic checklist (abbreviated to ACT). Methods Seventeen first and second year emergency medicine residents from the University of Toronto participated. Participants read up to eight case vignettes and completed the ACT diagnostic checklist. Provisional and final diagnoses and all responses for alternatives, consequences, and traits were individually scored as correct or incorrect. Additionally, each consequence was scored on a severity scale from 0 (not severe) to 3 (very severe). Average scores for alternatives, consequences, and traits and the severity rating for each consequence were entered into a binary logistic regression analysis with the outcome of revised or retained provisional diagnosis. Results Only 13% of diagnoses were revised. The binary logistic regression revealed that three scores derived from the ACT tool responses were associated with the decision to revise: severity rating of the consequence for missing the provisional diagnosis, the percent correct for identifying consequences, and the percent correct for identifying traits (χ2 = 23.5, df = 6, p < 0.001). The other three factors were not significant predictors. Conclusions Decisions to revise diagnoses may be cued by the detection of contradictory evidence. Education interventions may be more effective at reducing diagnostic error by targeting the ability to detect contradictory information within patient cases.

Perfecting practice: a protocol for assessing simulation-based mastery learning and deliberate practice versus self-guided practice for bougie-assisted cricothyroidotomy performance

BACKGROUND

Simulation-based medical education (SBME) is a cornerstone for procedural skill training in residency education. Multiple studies have concluded that SBME is highly effective, superior to traditional clinical education, and translates to improved patient outcomes. Additionally it is widely accepted that mastery learning, which comprises deliberate practice, is essential for expert level performance for routine skills; however, given that highly structured practice is more time and resource-intensive, it is important to assess its value for the acquisition of rarely performed technical skills. The bougie-assisted cricothyroidotomy (BAC), a rarely performed, lifesaving procedure, is an ideal skill for evaluating the utility of highly structured practice as it is relevant across many acute care specialties and rare - making it unlikely for learners to have had significant previous training or clinical experience. The purpose of this study is to compare a modified mastery learning approach with deliberate practice versus self-guided practice on technical skill performance using a bougie-assisted cricothyroidotomy model.

METHODS

A multi-centre, randomized study will be conducted at four Canadian and one American residency programs with 160 residents assigned to either mastery learning and deliberate practice (ML + DP), or self-guided practice for BAC. Skill performance, using a global rating scale, will be assessed before, immediately after practice, and 6 months later. The two groups will be compared to assess whether the type of practice impacts performance and skill retention.

DISCUSSION

Mastery learning coupled with deliberate practice provides systematic and focused feedback during skill acquisition. However, it is resource-intensive and its efficacy is not fully defined. This multi-centre study will provide generalizable data about the utility of highly structured practice for technical skill acquisition of a rare, lifesaving procedure within postgraduate medical education. Study findings will guide educators in the selection of an optimal training strategy, addressing both short and long term performance.

Coaching for Chaos: A Qualitative Study of Instructional Methods for Multipatient Management in the Emergency Department

BACKGROUND

Busy environments, like the emergency department (ED), require teachers to develop instructional strategies for coaching trainees to function within these same environments. Few studies have documented the strategies used by emergency physician (EP)-teachers within these busy, chaotic environments, instead emphasizing teaching in more predictable environments such as the outpatient clinic, hospital wards, or operating room. The authors sought to discover what strategies EP-teachers were using and what trainees recalled experiencing when learning to handle these unpredictable, overcrowded, complex, multipatient environments.

METHOD

An interpretive description study was conducted at multiple teaching hospitals affiliated with McMaster University from July 2014 to May 2015. Participants (10 EP-teachers and 10 junior residents) were asked to recall teaching strategies related to handling ED patient flow. Participants were asked to describe techniques that they used, observed, or experienced as trainees. Two independent coders read through interview transcripts, analyzing these documents inductively and iteratively.

RESULTS

Two main types of strategies to teach ED management were discovered: 1) workplace-based methods, including both observation and in situ instruction; and 2) principle-based advice. The most often described techniques were workplace-based methods, which included a variety of in situ techniques ranging from conversations to managerial coaching (e.g., collaborative problem-solving of real-life administrative dilemmas).

CONCLUSIONS

A mix of strategies are used to teach and coach trainees to handle multipatient environments. Further research is required to determine how to optimize the use of these techniques and innovate new strategies to support the learning of these crucial skills.

Clinical Informatics Competencies in the Emergency Medicine Specialist Training Standards of Five International Jurisdictions

BACKGROUND

The field of clinical informatics (CI), and specifically the electronic health record, has been identified as a key facilitator to achieve a sustainable evidence-based health care system for the future. International graduate medical education (GME) programs have been challenged to ensure that their trainees are provided with appropriate skills to deliver effective and efficient health care in an evolving environment.

OBJECTIVES

This study explored how international emergency medicine (EM) specialist training standards address competencies and training in relevant areas of CI.

METHODS

A list of categories of CI competencies relative to EM was developed following a thematic review of published references documenting CI curriculum and competencies. Publicly available documents outlining core content, curriculum, and competencies from international organizations responsible for specialty GME and/or credentialing in EM for Australasia, Canada, Europe, the United Kingdom, and the United States were identified. These EM training standards were reviewed to identify inclusion of topics related to the relevant categories of CI competencies.

RESULTS

A total of 23 EM curriculum documents were included in the review. Curricula content related to critical appraisal/evidence-based medicine, leadership, quality improvement, and privacy/security were included in all EM curricula. The CI topics related to fundamental computer skills, computerized provider order entry, and patient-centered informatics were only included in the EM curricula documents for the United States and were absent for the other jurisdictions.

CONCLUSION

There is variation in the CI-related content of the international EM specialty training standards reviewed. Given the increasing importance of CI in the future delivery of health care, organizations responsible for training and credentialing specialist emergency physicians must ensure that their training standards incorporate relevant CI content, thus ensuring that their trainees gain competence in essential aspects of CI.

The ALiEM Faculty Incubator: A Novel Online Approach to Faculty Development in Education Scholarship

PROBLEM

Early- and midcareer clinician educators often lack a local discipline-specific community of practice (CoP) that encourages scholarly activity. As a result, these faculty members may feel disconnected from other scholars.

APPROACH

Academic Life in Emergency Medicine (ALiEM) piloted the Faculty Incubator. This longitudinal, asynchronous, online curriculum focused on developing a virtual CoP among 30 early- to midcareer medical educators (the "incubatees"), 8 core faculty mentors, and 10 guest mentors. The yearlong curriculum included 12 monthly modules focusing on core concepts in medical education scholarship. The initiative connected the incubatees with a virtual community of peers and mentors, with whom they completed multiple scholarly projects, sought mentorship, and engaged professionally. The authors used an online, closed, social media platform (Slack) to facilitate the exchange of ideas.

OUTCOMES

In the inaugural year (March 2016-February 2017), the mentorship team facilitated exceptional levels of online engagement among incubatees. All participants (incubatees, core mentors, and guest mentors) shared 1,081 files and exchanged a total of 22,665 messages (approximately 62 per day). Of these, 3,036 (13.4%) were via open channels, 5,483 (24.2%) via small groups, and 14,146 (62.4%) via direct messages.

NEXT STEPS

The ALiEM Faculty Incubator represents a proof of concept, and initial outcomes show that it is possible to engage an international group of early- to midcareer medical educators to create a vibrant online CoP. The Faculty Incubator leaders plan to determine whether this engaged group of health professions educators will increase their scholarly output as a result of this initiative.

Quality Evaluation Scores are no more Reliable than Gestalt in Evaluating the Quality of Emergency Medicine Blogs: A METRIQ Study

Construct: We investigated the quality of emergency medicine (EM) blogs as educational resources.

PURPOSE

Online medical education resources such as blogs are increasingly used by EM trainees and clinicians. However, quality evaluations of these resources using gestalt are unreliable. We investigated the reliability of two previously derived quality evaluation instruments for blogs.

APPROACH

Sixty English-language EM websites that published clinically oriented blog posts between January 1 and February 24, 2016, were identified. A random number generator selected 10 websites, and the 2 most recent clinically oriented blog posts from each site were evaluated using gestalt, the Academic Life in Emergency Medicine (ALiEM) Approved Instructional Resources (AIR) score, and the Medical Education Translational Resources: Impact and Quality (METRIQ-8) score, by a sample of medical students, EM residents, and EM attendings. Each rater evaluated all 20 blog posts with gestalt and 15 of the 20 blog posts with the ALiEM AIR and METRIQ-8 scores. Pearson's correlations were calculated between the average scores for each metric. Single-measure intraclass correlation coefficients (ICCs) evaluated the reliability of each instrument.

RESULTS

Our study included 121 medical students, 88 EM residents, and 100 EM attendings who completed ratings. The average gestalt rating of each blog post correlated strongly with the average scores for ALiEM AIR (r = .94) and METRIQ-8 (r = .91). Single-measure ICCs were fair for gestalt (0.37, IQR 0.25-0.56), ALiEM AIR (0.41, IQR 0.29-0.60) and METRIQ-8 (0.40, IQR 0.28-0.59).

CONCLUSION

The average scores of each blog post correlated strongly with gestalt ratings. However, neither ALiEM AIR nor METRIQ-8 showed higher reliability than gestalt. Improved reliability may be possible through rater training and instrument refinement.

The 3 faces of clinical reasoning: Epistemological explorations of disparate error reduction strategies

There is general consensus that clinical reasoning involves 2 stages: a rapid stage where 1 or more diagnostic hypotheses are advanced and a slower stage where these hypotheses are tested or confirmed. The rapid hypothesis generation stage is considered inaccessible for analysis or observation. Consequently, recent research on clinical reasoning has focused specifically on improving the accuracy of the slower, hypothesis confirmation stage. Three perspectives have developed in this line of research, and each proposes different error reduction strategies for clinical reasoning. This paper considers these 3 perspectives and examines the underlying assumptions. Additionally, this paper reviews the evidence, or lack of, behind each class of error reduction strategies. The first perspective takes an epidemiological stance, appealing to the benefits of incorporating population data and evidence-based medicine in every day clinical reasoning. The second builds on the heuristic and bias research programme, appealing to a special class of dual process reasoning models that theorizes a rapid error prone cognitive process for problem solving with a slower more logical cognitive process capable of correcting those errors. Finally, the third perspective borrows from an exemplar model of categorization that explicitly relates clinical knowledge and experience to diagnostic accuracy.

Managing Multiplicity: Conceptualizing Physician Cognition in Multipatient Environments

PURPOSE

Emergency physicians (EPs) regularly manage multiple patients simultaneously, often making time-sensitive decisions around priorities for multiple patients. Few studies have explored physician cognition in multipatient scenarios. The authors sought to develop a conceptual framework to describe how EPs think in busy, multipatient environments.

METHOD

From July 2014 to May 2015, a qualitative study was conducted at McMaster University, using a think-aloud protocol to examine how 10 attending EPs and 10 junior residents made decisions in multipatient environments. Participants engaged in the think-aloud exercise for five different simulated multipatient scenarios. Transcripts from recorded interviews were analyzed inductively, with an iterative process involving two independent coders, and compared between attendings and residents.

RESULTS

The attending EPs and junior residents used similar processes to prioritize patients in these multipatient scenarios. The think-aloud processes demonstrated a similar process used by almost all participants. The cognitive task of patient prioritization consisted of three components: a brief overview of the entire cohort of patients to determine a general strategy; an individual chart review, whereby the participant created a functional patient story from information available in a file (i.e., vitals, brief clinical history); and creation of a relative priority list. Compared with residents, the attendings were better able to construct deeper and more complex patient stories.

CONCLUSIONS

The authors propose a conceptual framework for how EPs prioritize care for multiple patients in complex environments. This study may be useful to teachers who train physicians to function more efficiently in busy clinical environments.

Learning Analytics in Medical Education Assessment: The Past, the Present, and the Future

With the implementation of competency-based medical education (CBME) in emergency medicine, residency programs will amass substantial amounts of qualitative and quantitative data about trainees' performances. This increased volume of data will challenge traditional processes for assessing trainees and remediating training deficiencies. At the intersection of trainee performance data and statistical modeling lies the field of medical learning analytics. At a local training program level, learning analytics has the potential to assist program directors and competency committees with interpreting assessment data to inform decision making. On a broader level, learning analytics can be used to explore system questions and identify problems that may impact our educational programs. Scholars outside of health professions education have been exploring the use of learning analytics for years and their theories and applications have the potential to inform our implementation of CBME. The purpose of this review is to characterize the methodologies of learning analytics and explore their potential to guide new forms of assessment within medical education.

Performance of high-sensitivity cardiac troponin in the emergency department for myocardial infarction and a composite cardiac outcome across different estimated glomerular filtration rates

BACKGROUND

Clinicians regularly observe increased high-sensitivity cardiac troponin (hs-cTn) concentrations in patients with low estimated glomerular filtration rate (eGFR). The challenge is to differentiate acute coronary syndrome (ACS) from increased hs-cTn results across a range of eGFR. The objective of this study was to determined the optimal hs-cTn concentrations for acute myocardial infarction (MI) and a composite cardiovascular outcome across different eGFR ranges and to assess the utility of a low hs-cTn cutoff to rule-out events.

METHODS

We undertook an observational study in the emergency department of patients (n = 1212) with symptoms suggestive of ACS who had an eGFR and at least one Roche hs-cTnT and one Abbott hs-cTnI result. The 7-day outcomes were MI or a composite of MI, unstable angina, congestive heart failure, serious ventricular cardiac arrhythmia, or death. The maximum hs-cTn concentration was assessed across different eGFR ranges (<30,30-59,60-89,≥90 ml/min/1.73m²) by spearman correlation, ROC-curve analyses, and sensitivity and negative predictive value (NPV) for the proposed rule-out hs-cTn cutoffs (hs-cTnI<5 ng/l and hs-cTnT<6 ng/l) for the outcomes.

RESULTS

Both hs-cTnI and hs-cTnT concentrations were negatively correlated with eGFR. The lower the eGFR, the lower the AUC and the higher the optimal hs-cTn cutoffs for both MI and the composite outcome. The highest combined sensitivity (100%), NPV (100%) and proportion of low-risk for MI (45% of group) was observed for patients with hs-cTnT<6 ng/l with an eGFR≥90.

CONCLUSION

The test performance for hs-cTn for diagnosing or ruling-out an acute cardiac event varies per the eGFR. Accurate risk stratification requires knowledge of the eGFR.

JGME-ALiEM Hot Topics in Medical Education: An Analysis of a Virtual Discussion on Resident Well-Being

BACKGROUND

Physician well-being is garnering increasing attention. In 2016, the Journal of Graduate Medical Education (JGME) published a review by Kristin Raj, MD, entitled "Well-Being in Residency: A Systematic Review." There is benefit in contextualizing the literature on resident well-being through an academic journal club.

OBJECTIVE

We summarized an asynchronous, online journal club discussion about this systematic review and highlighted themes that were identified in the review.

METHODS

In January 2017, JGME and the Academic Life in Emergency Medicine (ALiEM) blog facilitated an open-access, online, weeklong journal club on the featured JGME article. Online discussions and interactions were facilitated via blog posts and comments, a video discussion on Google Hangouts on Air, and Twitter. We performed a thematic analysis of the discussion and captured web analytics.

RESULTS

Over the first 14 days, the blog post was viewed 1070 unique times across 52 different countries. A total of 130 unique participants on Twitter posted 480 tweets using the hashtag #JGMEscholar. Thematic analysis revealed 5 major domains: the multidimensional nature of well-being, measurement of well-being, description of wellness programs and interventions, creation of a culture of wellness, and critique of the methodology of the review.

CONCLUSIONS

Our online journal club highlighted several gaps in the current understanding of resident well-being, including the need for consensus on the operational definition, the need for effective instruments to evaluate wellness programs and identify residents in distress, and a national research collaboration to assess wellness programs and their impact on resident well-being.

Failure to flow: An exploration of learning and teaching in busy, multi-patient environments using an interpretive description method

INTRODUCTION

As patient volumes continue to increase, more attention must be paid to skills that foster efficiency without sacrificing patient safety. The emergency department is a fertile ground for examining leadership and management skills, especially those that concern prioritization in multi-patient environments. We sought to understand the needs of emergency physicians (EPs) and emergency medicine junior trainees with regards to teaching and learning about how best to handle busy, multi-patient environments.

METHOD

A cognitive task analysis was undertaken, using a qualitative approach to elicit knowledge of EPs and residents about handling busy emergency department situations. Ten experienced EPs and 10 junior emergency medicine residents were interviewed about their experiences in busy emergency departments. Transcripts of the interviews were analyzed inductively and iteratively by two independent coders using an interpretive description technique.

RESULTS

EP teachers and junior residents differed in their perceptions of what makes an emergency department busy. Moreover, they focused on different aspects of patient care that contributed to their busyness: EP teachers tended to focus on volume of patients, junior residents tended to focus on the complexity of certain cases. The most important barrier to effective teaching and learning of managerial skills was thought to be the lack of faculty development in this skill set.

CONCLUSIONS

This study presents qualitative data that helps us elucidate how patient volumes affect our learning environments, and how clinical teachers and residents operate within these environments.