26
|
Gottlieb M, Riddell J, Cooney R, King A, Fung CC, Sherbino J. Maximizing the Morning Commute: A Randomized Trial Assessing the Effect of Driving on Podcast Knowledge Acquisition and Retention. Ann Emerg Med 2021; 78:416-424. [PMID: 33931254 DOI: 10.1016/j.annemergmed.2021.02.030] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/05/2020] [Revised: 02/25/2021] [Accepted: 02/25/2021] [Indexed: 01/21/2023]
Abstract
STUDY OBJECTIVE Emergency medicine residents use podcasts as part of their learning process, often listening while driving. It is unclear how driving while listening to a podcast affects knowledge acquisition and retention. This study evaluated the knowledge gained from listening to podcasts while driving compared to that gained from undistracted listening. METHODS This was a multicenter, randomized, crossover trial among postgraduate year (PGY) 1 to 4 emergency medicine residents at 4 institutions. Residents were randomized with stratification by site and PGY level to listen to podcasts while driving first or sitting undistracted in a room first. Within 30 minutes of listening, they completed a 20-question test. They subsequently crossed over to the alternate intervention, serving as their own controls, and listened to a different podcast before completing a second 20-question test. Each of the podcasts was professionally recorded and based on 5 emergency medicine-relevant journal articles that had not been covered in a journal club or curriculum at any of the institutions. One month later, participants completed a delayed recall test composed of 40 new questions based on both podcasts. Questions were derived and validity evidence was collected prior to use. Data were compared using a paired-sample t test and ANOVA. RESULTS A total of 100 residents completed the initial recall tests, and 96 residents completed the delayed recall test. There was no statistically significant difference between the driving and undistracted cohorts on the initial recall (74.2% versus 73.3%) or delayed recall (52.2% versus 52.0%). CONCLUSION Driving while listening to a podcast does not meaningfully affect knowledge acquisition or retention when compared with undistracted podcast listening among emergency medicine residents.
Collapse
|
27
|
LoGiudice AB, Sherbino J, Norman G, Monteiro S, Sibbald M. Intuitive and deliberative approaches for diagnosing 'well' versus 'unwell': evidence from eye tracking, and potential implications for training. ADVANCES IN HEALTH SCIENCES EDUCATION : THEORY AND PRACTICE 2021; 26:811-825. [PMID: 33423154 DOI: 10.1007/s10459-020-10023-w] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 04/22/2020] [Accepted: 12/21/2020] [Indexed: 06/12/2023]
Abstract
Rapidly assessing how ill a patient is based on their immediate presentation-colloquially termed 'eyeballing' in practice-serves a vital role in acute care settings. Yet surprisingly little is known about how this diagnostic skill is learned or how it should be taught. Some authors have pointed to a dual-process model, suggesting that assessments of illness severity are driven by two distinct types of processing: an intuitive, fast, pattern recognition-like process (Type 1) that depends on many prior patient encounters and outcomes being stored in memory; and a deliberate, slow, analytic process (Type 2) characterized by additional data gathering, data scrutiny, or recollection of rules. But prior studies have supported a dual-process model for the assessment of illness severity only insofar as experienced clinicians chiefly displayed what was presumed to be Type 1 processing. Here we further explored a dual-process model by examining whether less experienced clinicians displayed both types of processing when assessing illness severity across a series of cases. Consistent with the model, a dissociation between Type 1 and Type 2 processing was observed through resident reports of deliberation, response times, and three eye tracking metrics associated with diagnostic expertise. We conclude by discussing potential implications for the training of this enigmatic diagnostic skill.
Collapse
|
28
|
Magee K, Sherbino J. Letter to the Editor. CAN J EMERG MED 2021; 23:565-566. [PMID: 33704693 DOI: 10.1007/s43678-021-00101-2] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/28/2021] [Accepted: 02/09/2021] [Indexed: 12/01/2022]
|
29
|
Gottlieb M, Yarris LM, Krzyzaniak SM, Natesan S, Sherbino J, Lin M, Chan TM. Faculty development using a virtual community of practice: Three-year outcomes of the Academic Life in Emergency Medicine Faculty Incubator program. AEM EDUCATION AND TRAINING 2021; 5:e10626. [PMID: 34222756 PMCID: PMC8241569 DOI: 10.1002/aet2.10626] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 03/19/2021] [Revised: 05/11/2021] [Accepted: 05/12/2021] [Indexed: 05/11/2023]
Abstract
INTRODUCTION The Academic Life in Emergency Medicine (ALiEM) Faculty Incubator program is a longitudinal, 1-year, virtual faculty development program for early- and mid-career faculty members that crosses specialties and institutions. This study sought to evaluate the outcomes among 3 years of participants. METHODS This cross-sectional survey study evaluated postcourse and 1-year outcomes from three graduated classes of the ALiEM Faculty Incubator program. The program evaluation survey was designed to collect outcomes across multiple Kirkpatrick levels using pre/post surveys and tracking of abstracts, publications, speaking opportunities, new leadership positions, and new curricula. RESULTS Over 3 years, 89 clinician educators participated in the program. Of those, 59 (66%) completed the initial survey and 33 (37%) completed the 1-year survey. Participants reported a significant increase in knowledge (4.1/9.0 vs. 7.0/9.0). The number of abstracts, publications, and invited presentations significantly increased after course completion and continued postcourse. A total of 37 of 59 (62.7%) developed a new curriculum during the course and 19 of 33 (57.6%) developed another new curriculum after the course. A total of 29 of 59 (49.2%) began a new leadership position upon course completion with 15 of 33 (45.5%) beginning another new leadership position 1 year later. DISCUSSION The ALiEM Faculty Incubator program demonstrated an increase in perceived knowledge and documented academic productivity among early- and mid-career medical educators.
Collapse
|
30
|
Ilgen JS, Regehr G, Teunissen PW, Sherbino J, de Bruin ABH. Skeptical self-regulation: Resident experiences of uncertainty about uncertainty. MEDICAL EDUCATION 2021; 55:749-757. [PMID: 33527454 DOI: 10.1111/medu.14459] [Citation(s) in RCA: 11] [Impact Index Per Article: 3.7] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 11/03/2020] [Revised: 01/18/2021] [Accepted: 01/20/2021] [Indexed: 06/12/2023]
Abstract
OBJECTIVES Managing uncertainty is central to expert practice, yet how novice trainees navigate these moments is likely different than what has been described by experienced clinicians. Exploring trainees' experiences with uncertainty could therefore help explicate the unique cues that they attend to, how they appraise their comfort in these moments and how they enact responses within the affordances of their training environment. METHODS Informed by constructivist grounded theory, we explored how novice emergency medicine trainees experienced and managed clinical uncertainty in practice. We used a critical incident technique to prompt participants to reflect on experiences with uncertainty immediately following a clinical shift, exploring the cues they attended to and the approaches they used to navigate these moments. Two investigators coded line-by-line using constant comparison, organising the data into focused codes. The research team discussed the relationships between these codes and developed a set of themes that supported our efforts to theorise about the phenomenon. RESULTS We enrolled 13 trainees in their first two years of postgraduate training across two institutions. They expressed uncertainty about the root causes of the patient problems they were facing and the potential management steps to take, but also expressed a pervasive sense of uncertainty about their own abilities and their appraisals of the situation. This, in turn, led to challenges with selecting, interpreting and using the cues in their environment effectively. Participants invoked several approaches to combat this sense of uncertainty about themselves, rehearsing steps before a clinical encounter, checking their interpretations with others and implicitly calibrating their appraisals to those of more experienced team members. CONCLUSIONS Trainees' struggles with the legitimacy of their interpretations impact their experiences with uncertainty. Recognising these ongoing struggles may enable supervisors and other team members to provide more effective scaffolding, validation and calibration of clinical judgments and patient management.
Collapse
|
31
|
Dowhos K, Sherbino J, Chan TM, Nagji A. Infographics, podcasts, and blogs: a multi-channel, asynchronous, digital faculty experience to improve clinical teaching (MAX FacDev). CAN J EMERG MED 2021; 23:390-393. [PMID: 33788176 PMCID: PMC8011059 DOI: 10.1007/s43678-020-00069-5] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/01/2020] [Accepted: 12/15/2020] [Indexed: 11/29/2022]
Abstract
Attendance at faculty development events are often limited to those with time and interest, but all clinical teachers should have access to continuing faculty development. A multi-channel, asynchronous, digital faculty experience strategy (MAX FacDev) was used to engage busy emergency medicine (EM) teachers associated with a distributed medical education network involving ten geographically distinct teaching sites. An evidence-informed education bundle on key principles for clinical teaching was developed. The education bundle included five topics, serialized via: an infographic series posted in distributed medical education EM departments, a podcast series and a blog. The target audience included 102 faculty members and 46 residents. Within 8 months of launching MAX FacDev, there were 1508 podcast listens and 7686 pageviews. An education bundle can efficiently deliver on-demand faculty development. Amplifying key messages via multiple channels increases the reach of faculty development and reinforces the messages.
Collapse
|
32
|
Kandasamy S, Vanstone M, Colvin E, Chan T, Sherbino J, Monteiro S. "I made a mistake!": A narrative analysis of experienced physicians' stories of preventable error. J Eval Clin Pract 2021; 27:236-245. [PMID: 33399266 DOI: 10.1111/jep.13531] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 08/08/2020] [Revised: 12/09/2020] [Accepted: 12/14/2020] [Indexed: 01/01/2023]
Abstract
RATIONALE, AIMS, AND OBJECTIVES The complexity of healthcare systems makes errors unavoidable. To strengthen the dialogue around how physicians experience and share medical errors, the objective of this study was to understand how generalist physicians make meaning of and grow from their medical errors. METHODS This study used a narrative inquiry approach to conduct and analyse in-depth interviews from 26 physicians from the generalist specialties of emergency, internal, and family medicine. We gathered stories via individual interview, analysed them for key components, and rewrote a "meta-story" in a chronological sequence. We conceptualized the findings into a metaphor to draw similarities, learn from, and apply new principles from other fields of practice. RESULTS Through analysis we interpreted the story of a physician who is required to make numerous decisions in a short period of time in different clinical environments among the patient's family and whilst abiding by existing rules and regulations. Through sharing stories of success and failure, the clinical supervisor can help optimize the physician's emotional growth and professional development. Similarly, through sharing and learning from stories, colleagues and trainees can also contribute to the growth of the protagonist's character and the development of clinic, hospital, and healthcare system. CONCLUSION We draw parallels between the clinical setting and a generalist physician's experiences of a medical error with the environment and practices within professional sports. Using this comparison, we discuss the potential for meaningful coaching in medical education.
Collapse
|
33
|
Neighbour H, Monteiro S, Lee M, Wu Y, Levinson A, Sherbino J. Effect of Gender on Feedback Associated with a Direct Clinical Observation Tool Used during Clerkship at McMaster University. J Gen Intern Med 2021:10.1007/s11606-020-06399-5. [PMID: 33506385 DOI: 10.1007/s11606-020-06399-5] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 07/13/2020] [Accepted: 12/03/2020] [Indexed: 10/22/2022]
|
34
|
Li S, Acai A, Sherbino J, Chan TM. The Teacher, the Assessor, and the Patient Protector: A Conceptual Model Describing How Context Interfaces With the Supervisory Roles of Academic Emergency Physicians. AEM EDUCATION AND TRAINING 2021; 5:52-62. [PMID: 33521491 PMCID: PMC7821073 DOI: 10.1002/aet2.10431] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 10/22/2019] [Revised: 12/05/2019] [Accepted: 12/12/2019] [Indexed: 05/10/2023]
Abstract
OBJECTIVES Emergency medicine is a fast-paced specialty that demands emergency physicians to respond to rapidly evolving patient presentations, while engaging in clinical supervision. Most research on supervisory roles has focused on the behaviors of attending physicians, including their individual preferences of supervision and level of entrustment of clinical tasks to trainees. However, less research has investigated how the clinical context (patient case complexity, workflow) influences clinical supervision. In this study, we examined how the context of the emergency department (ED) shapes the ways in which emergency physicians reconcile their competing roles in patient care and clinical supervision to optimize learning and ensure patient safety. METHODS Emergency physicians who regularly participated in clinical supervision in several academic teaching hospitals were individually interviewed using a semi-structured format. The interviews were transcribed and analyzed using a constructivist grounded theory approach. RESULTS Sixteen emergency physicians were asked to reflect on their clinical supervisory roles in the ED. We conceptualized a model that describes three prominent roles: teacher, assessor, and patient protector. Contextual features such as trainee competence, pace of the ED, patient complexity, and the culture of academic medicine influenced the extent to which certain roles were considered salient at any given time. CONCLUSIONS This conceptual model can inform researchers and medical educators about the role of context in accentuating or minimizing various roles of emergency physicians. Identifying how context interfaces with these roles may help design faculty development initiatives aimed to navigate the tension between urgent patient care and medical education for emergency physicians.
Collapse
|
35
|
Riddell JC, Robins L, Sherbino J, Brown A, Ilgen J. Residents' Perceptions of Effective Features of Educational Podcasts. West J Emerg Med 2020; 22:26-32. [PMID: 33439799 PMCID: PMC7806333 DOI: 10.5811/westjem.2020.10.49135] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/15/2020] [Accepted: 10/15/2020] [Indexed: 11/11/2022] Open
Abstract
INTRODUCTION Educational podcasts are used by emergency medicine (EM) trainees to supplement clinical learning and to foster a sense of connection to broader physician communities. Yet residents report difficulties remembering what they learned from listening, and the features of podcasts that residents find most effective for learning remain poorly understood. Therefore, we sought to explore residents' perceptions of the design features of educational podcasts that they felt most effectively promoted learning. METHODS We used a qualitative approach to explore EM trainees' experiences with educational podcasts, focusing on design features that they found beneficial to their learning. We conducted 16 semi-structured interviews with residents from three institutions from March 2016-August 2017. Interview transcripts were analyzed line-by-line using constant comparison and organized into focused codes, conceptual categories, and then key themes. RESULTS The five canons of classical rhetoric provided a framework for thematically grouping the disparate features of podcasts that residents reported enhanced their learning. Specifically, they reported valuing the following: 1) Invention: clinically relevant material presented from multiple perspectives with explicit learning points; 2) Arrangement: efficient communication; 3) Style: narrative incorporating humor and storytelling; 4) Memory: repetition of key content; and 5) Delivery: short episodes with good production quality. CONCLUSION This exploratory study describes features that residents perceived as effective for learning from educational podcasts and provides foundational guidance for ongoing research into the most effective ways to structure medical education podcasts.
Collapse
|
36
|
Kavsak PA, Mondoux SE, de Wit K, Rochwerg B, Main C, Yamamura D, Paré G, Ma J, Perri D, Sherbino J, Worster A. Admission High-Sensitivity Cardiac Troponin vs a Biochemical Score for Predicting Mortality in Patients With COVID-19. CJC Open 2020; 3:130-131. [PMID: 33073223 PMCID: PMC7553859 DOI: 10.1016/j.cjco.2020.10.008] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/20/2022] Open
|
37
|
Riddell J, Robins L, Lin M, Sherbino J, Ilgen J. 1EMF Hearing Is Believing: A Qualitative Exploration of Trust and Credibility Judgements in Educational Podcasts. Ann Emerg Med 2020. [DOI: 10.1016/j.annemergmed.2020.09.011] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/23/2022]
|
38
|
Gottlieb M, Riddell J, King A, Cooney R, Fung C, Sherbino J. 408EMF The Impact of Driving on Podcast Knowledge Acquisition and Retention among Emergency Medicine Resident Physicians. Ann Emerg Med 2020. [DOI: 10.1016/j.annemergmed.2020.09.424] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/23/2022]
|
39
|
Riddell J, Brown A, Robins L, Lin M, Sherbino J, Ilgen J. 300 Residents’ Perceptions of Effective Features of Educational Podcasts. Ann Emerg Med 2020. [DOI: 10.1016/j.annemergmed.2020.09.314] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/26/2022]
|
40
|
Kavsak PA, Cerasuolo JO, Ko DT, Ma J, Sherbino J, Mondoux SE, Clayton N, Hill SA, McQueen M, Griffith LE, Mehta SR, Perez R, Seow H, Devereaux PJ, Worster A. Using the clinical chemistry score in the emergency department to detect adverse cardiac events: a diagnostic accuracy study. CMAJ Open 2020; 8:E676-E684. [PMID: 33139388 PMCID: PMC7608943 DOI: 10.9778/cmajo.20200047] [Citation(s) in RCA: 15] [Impact Index Per Article: 3.8] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 12/14/2022] Open
Abstract
BACKGROUND The ability to rule out or in a major adverse cardiac event (MACE) in patients with suspected acute coronary syndrome at emergency department (ED) presentation would be beneficial to patient care and the health care system. The clinical chemistry score (CCS) was evaluated in this context. METHODS This diagnostic accuracy study evaluated 2 different ED cohorts with suspected acute coronary syndrome. For patients in cohort 1, who presented to the ED of 3 hospitals in Hamilton, Ontario, between May and August 2013, retrospective measurements were taken using the Ortho Clinical Diagnostics high-sensitivity cardiac troponin I (hs-cTnI) assay; for patients in cohort 2, who presented to the ED of the same 3 hospitals in Hamilton between November 2012 and February 2013, an ED cardiac presentation blood test panel was performed with the Abbott Diagnostics hs-cTnI assay. The sensitivity and specificity of the CCS (cut-offs of ≥ 1 and 5) and hs-cTnI alone (published cut-offs) were compared for MACE (composite of death, myocardial infarction, unstable angina, revascularization) at 30 days for both cohorts and at 90 days for cohort 2. RESULTS The incidence of MACE at 30 days was higher in cohort 1 (n = 1058) (19.4%, 95% confidence interval [CI] 16.8%-22.2%) than in cohort 2 (n = 5974) (14.6%, 95% CI 13.6%-15.6%). In cohort 1, a CCS of 1 or above yielded a sensitivity of 99.5% (95% CI 97.3%-99.9%). The sensitivity with an Ortho hs-cTnI cut-off of 1 ng/L or above was 91.2% (95% CI 86.5%-95.7%). The specificity of a CCS of 5 (97.8%, 95% CI 96.5%-98.7%) was higher than when the overall 99th-percentile cut-off for the Ortho hs-cTnI assay (> 11 ng/L; 90.1%, 95% CI 87.9%-92.0%) was used. A similar pattern was observed in cohort 2 at 30 days and persisted at 90 days with the Abbott hs-cTnI assay. INTERPRETATION The CCS derived with 2 different hs-cTnI assays and ED populations yielded higher sensitivity and specificity estimates for MACE than hs-cTnI alone. An intervention study is needed to evaluate the impact of the CCS at both the patient and hospital levels. TRIAL REGISTRATION ClinicalTrials.gov, no. NCT01994577.
Collapse
|
41
|
Kavsak PA, Cerasuolo JO, Mondoux SE, Sherbino J, Ma J, Hoard BK, Perez R, Seow H, Ko DT, Worster A. Risk Stratification for Patients with Chest Pain Discharged Home from the Emergency Department. J Clin Med 2020; 9:E2948. [PMID: 32932598 PMCID: PMC7565964 DOI: 10.3390/jcm9092948] [Citation(s) in RCA: 6] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/13/2020] [Revised: 09/10/2020] [Accepted: 09/10/2020] [Indexed: 12/17/2022] Open
Abstract
For patients with chest pain who are deemed clinically to be low risk and discharged home from the emergency department (ED), it is unclear whether further laboratory tests can improve risk stratification. Here, we investigated the utility of a clinical chemistry score (CCS), which comprises plasma glucose, the estimated glomerular filtration rate, and high-sensitivity cardiac troponin (I or T) to generate a common score for risk stratification. In a cohort of 14,676 chest pain patients in the province of Ontario, Canada and who were discharged home from the ED (November 2012-February 2013 and April 2013-September 2015) we evaluated the CCS as a risk stratification tool for all-cause mortality, plus hospitalization for myocardial infarction or unstable angina (primary outcome) at 30, 90, and 365 days post-discharge using Cox proportional hazard models. At 30 days the primary outcome occurred in 0.3% of patients with a CCS < 2 (n = 6404), 0.9% of patients with a CCS = 2 (n = 4336), and 2.3% of patients with a CCS > 2 (n = 3936) (p < 0.001). At 90 days, patients with CCS < 2 (median age = 52y (IQR = 46-60), 59.4% female) had an adjusted HR = 0.51 (95% confidence interval (CI) = 0.32-0.82) for the composite outcome and patients with a CCS > 2 (median age = 74y (IQR = 64-82), 48.0% female) had an adjusted HR = 2.80 (95%CI = 1.98-3.97). At 365 days, 1.3%, 3.4%, and 11.1% of patients with a CCS < 2, 2, or >2 respectively, had the composite outcome (p < 0.001). In conclusion, the CCS can risk stratify chest pain patients discharged home from the ED and identifies both low- and high-risk patients who may warrant different medical care.
Collapse
|
42
|
Monteiro S, Sherbino J, Ilgen JS, Hayden EM, Howey E, Norman G. The effect of prior experience on diagnostic reasoning: exploration of availability bias. ACTA ACUST UNITED AC 2020; 7:265-272. [PMID: 32776898 DOI: 10.1515/dx-2019-0091] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/23/2019] [Accepted: 06/08/2020] [Indexed: 11/15/2022]
Abstract
Objectives Diagnostic reasoning has been shown to be influenced by a prior similar patient case. However, it is unclear whether this process influences diagnostic error rates or whether clinicians at all experience levels are equally susceptible. The present study measured the influence of specific prior exposure and experience level on diagnostic accuracy. Methods To create the experience of prior exposure, participants (pre-clerkship medical students, emergency medicine residents, and faculty) first verified diagnoses of clinical vignettes. The influence of prior exposures was measured using equiprobable clinical vignettes; indicating two diagnoses. Participants diagnosed equiprobable cases that were: 1) matched to exposure cases (in one of three conditions: a) similar patient features, similar clinical features; b) dissimilar patient features, similar clinical features; c) similar patient features, dissimilar clinical features), or 2) not matched to any prior case (d) no exposure). A diagnosis consistent with a matched exposure case was scored correct. Cases with no prior exposure had no matched cases, hence validated the equiprobable design. Results Diagnosis A represented 47% of responses in condition d, but there was no influence of specific similarity of patient characteristics for Diagnosis A, F(3,712)=7.28, p=0.28 or Diagnosis B, F(3,712)=4.87, p=0.19. When re-scored based on matching both equiprobable diagnoses, accuracy was high, but favored faculty (n=40) 98%, and residents (n=39) 98% over medical students (n=32) 85%, F(2,712)=35.6, p<0.0001. Accuracy for medical students was 84, 87, 94, and 73% for conditions a-d, respectively, interaction F(2,712)=3.55, p<0.002. Conclusions The differential diagnosis of pre-clerkship medical students improved with prior exposure, but this was unrelated to specific case or patient features. The accuracy of medical residents and staff was not influenced by prior exposure.
Collapse
|
43
|
Chan TM, Mercuri M, Turcotte M, Gardiner E, Sherbino J, de Wit K. Making Decisions in the Era of the Clinical Decision Rule: How Emergency Physicians Use Clinical Decision Rules. ACADEMIC MEDICINE : JOURNAL OF THE ASSOCIATION OF AMERICAN MEDICAL COLLEGES 2020; 95:1230-1237. [PMID: 31789846 DOI: 10.1097/acm.0000000000003098] [Citation(s) in RCA: 15] [Impact Index Per Article: 3.8] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/10/2023]
Abstract
PURPOSE Physicians are often asked to integrate clinical decision rules (CDRs) with their own cognitive processes to reach a diagnosis. Clinicians, researchers, and educators must understand these cognitive processes to evaluate and improve the diagnostic process. The authors sought to explore emergency physicians' diagnostic processes and to examine how they integrated CDRs into their reasoning using simulated cases (with chest pain or leg pain). METHOD From August 2015 to July 2016, 16 practicing emergency physicians from 3 teaching hospitals associated with McMaster University, Ontario, Canada, were interviewed via a novel "teach aloud" protocol. Six videos of simulated patients with chest pain, breathlessness, or leg discomfort were used as prompts for the physicians to demonstrate their diagnostic thinking. Using a constructivist grounded theory analysis, 3 investigators independently reviewed the interview transcripts, meeting regularly to discuss identified themes and subthemes until sufficiency was reached. RESULTS A model to describe how clinicians integrate their own decision making with CDRs was developed, showing that physicians engage in an iterative diagnostic process that repeatedly refines the differential diagnosis list. The steps in the diagnostic process were: refinement of the differential diagnosis, ordering a hierarchy of risk, the decision to test, choosing the tests, and interpreting test results. Physicians applied CDRs when they had already decided to test. CONCLUSIONS To date, CDRs assume a static, linear model of clinical decision making. Findings demonstrate that participants engaged in iterative and dynamic decision-making processes that changed throughout their patient encounter, contingent on multiple contextual features. Understanding these processes could inform future development of CDRs and educational strategies around these decision aids.
Collapse
|
44
|
Kavsak PA, Mondoux SE, Ma J, Sherbino J, Hill SA, Clayton N, Mehta SR, Griffith LE, McQueen M, Devereaux PJ, Worster A. Comparison of two biomarker only algorithms for early risk stratification in patients with suspected acute coronary syndrome. Int J Cardiol 2020; 319:140-143. [PMID: 32634494 DOI: 10.1016/j.ijcard.2020.06.066] [Citation(s) in RCA: 11] [Impact Index Per Article: 2.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 04/02/2020] [Revised: 06/12/2020] [Accepted: 06/29/2020] [Indexed: 11/25/2022]
Abstract
BACKGROUND We developed a biomarker algorithm encompassing the clinical chemistry score (CCS; which includes the combination of a random glucose concentration, an estimated glomerular filtration rate and high-sensitivity cardiac troponin; hs-cTn) with the Ortho Clinical Diagnostics hs-cTnI assay (CCS-serial) and compared it to the cutoffs derived from Ortho Clinical Diagnostics 0/1 h (h) algorithm for 7-day myocardial infarction (MI) or cardiovascular (CV)-death. METHODS The study cohort was an emergency department (ED) population (n = 906) with symptoms suggestive of acute coronary syndrome (ACS) who had two Ortho hs-cTnI results approximately 3 h apart. Diagnostic parameters (sensitivity/specificity/negative predictive value; NPV/positive predictive value; PPV) were derived for the CCS-serial and the 0/1 h algorithm for 7-day MI/CV-death. A safety analysis was performed for patients in the rule-out arms of the algorithms for 30-day MI/death. RESULTS The CCS-serial algorithm yielded 100% sensitivity/NPV (32% low-risk) and 95.7% specificity/65% PPV (11% high-risk). The 0/1 h algorithm-cutoffs yielded sensitivity/NPV/specificity/PPV of 97.8%/99.4%/91.3%/50%, which classified 38% of patients as low-risk and 16% of patients as high-risk. Four patients (1.2%) in the 0/1 h algorithm-cutoff rule-out arm had a 30-day MI/death outcome as compared to zero patients in the CCS-serial rule-out arm (p = 0.06). CONCLUSION Both the CCS-serial and 0/1 h algorithm cutoffs yield high NPVs with a similar proportion of patients identified as low-risk. These data may be useful for sites who are unable to collect samples at 0/1 h in the emergency department.
Collapse
|
45
|
Kavsak PA, Cerasuolo JO, Ko DT, Ma J, Sherbino J, Mondoux SE, Perez R, Seow H, Worster A. High-Sensitivity Cardiac Troponin I vs a Clinical Chemistry Score for Predicting All-Cause Mortality in an Emergency Department Population. CJC Open 2020; 2:296-302. [PMID: 32695979 PMCID: PMC7365813 DOI: 10.1016/j.cjco.2020.03.004] [Citation(s) in RCA: 6] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/19/2020] [Accepted: 03/04/2020] [Indexed: 12/15/2022] Open
Abstract
BACKGROUND For patients investigated for suspected acute coronary syndrome, there is uncertainty if a single measurement of high-sensitivity cardiac troponin I (hs-cTnI) at emergency department (ED) presentation can identify patients at both low and high risk for mortality. METHODS We included consecutive adult patients in the ED who had a Clinical Chemistry Score (CCS) taken at presentation (ie, combination of glucose, creatinine for estimated glomerular filtration rate determination, and hs-cTnI assay) in a Canadian city between 2012 and 2013. Outcomes were 3-month, 1-year, and 5-year all-cause mortality using the provincial death registry. Mortality rates and test performance (eg, sensitivity and specificity) with 95% confidence intervals (CIs) were obtained for the CCS or hs-cTnI assay alone using established cutoffs for these tests. RESULTS Our cohort included 5974 patients with a 1-year mortality rate of 17.2% (95% CI, 16.2-18.3). A CCS ≥ 1 yielded a sensitivity of 99.2% (95% CI, 98.4-99.6) compared with the hs-cTnI ≥ 5 ng/L cutoff sensitivity of 88.4% (95% CI, 86.3-90.3), with the mortality rate being significantly lower for patients with CCS < 1 (2.0%; 95% CI, 0.9-4.0) vs patients with hs-cTnI < 5 ng/L (5.0%; 95% CI, 4.2-6.0) at 1 year (P = 0.01). A CCS of 5 also yielded a higher specificity (88.5%; 95% CI, 87.5-89.3) compared with hs-cTnI > 26 ng/L (83.9%; 95% CI, 82.9-84.9), with no difference in mortality rates (37.4% vs 36.3%; P = 0.66). This trend was consistent at 3-month and 5-year mortality. CONCLUSION For patients in the ED with a potential cardiac issue, using the CCS cutoffs can better identify patients at low and high risk for mortality than using published cutoffs for hs-cTnI alone.
Collapse
|
46
|
Odorizzi S, Cheung WJ, Sherbino J, Lee AC, Thurgur L, Frank JR. A Signal Through the Noise: Do Professionalism Concerns Impact the Decision Making of Competence Committees? ACADEMIC MEDICINE : JOURNAL OF THE ASSOCIATION OF AMERICAN MEDICAL COLLEGES 2020; 95:896-901. [PMID: 31577582 DOI: 10.1097/acm.0000000000003005] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/10/2023]
Abstract
PURPOSE To characterize how professionalism concerns influence individual reviewers' decisions about resident progression using simulated competence committee (CC) reviews. METHOD In April 2017, the authors conducted a survey of 25 Royal College of Physicians and Surgeons of Canada emergency medicine residency program directors and senior faculty who were likely to function as members of a CC (or equivalent) at their institution. Participants took a survey with 12 resident portfolios, each containing hypothetical formative and summative assessments. Six portfolios represented residents progressing as expected (PAE) and 6 represented residents not progressing as expected (NPAE). A professionalism variable (PV) was developed for each portfolio. Two counterbalanced surveys were developed in which 6 portfolios contained a PV and 6 portfolios did not (for each PV condition, 3 portfolios represented residents PAE and 3 represented residents NPAE). Participants were asked to make progression decisions based on each portfolio. RESULTS Without PVs, the consistency of participants giving scores of 1 or 2 (i.e., little or no need for educational intervention) to residents PAE and to those NPAE was 92% and 10%, respectively. When a PV was added, the consistency decreased by 34% for residents PAE and increased by 4% for those NPAE (P = .01). CONCLUSIONS When reviewing a simulated resident portfolio, individual reviewer scores for residents PAE were responsive to the addition of professionalism concerns. Considering this, educators using a CC should have a system to report, collect, and document professionalism issues.
Collapse
|
47
|
Hall AK, Rich J, Dagnone JD, Weersink K, Caudle J, Sherbino J, Frank JR, Bandiera G, Van Melle E. It's a Marathon, Not a Sprint: Rapid Evaluation of Competency-Based Medical Education Program Implementation. ACADEMIC MEDICINE : JOURNAL OF THE ASSOCIATION OF AMERICAN MEDICAL COLLEGES 2020; 95:786-793. [PMID: 31625995 DOI: 10.1097/acm.0000000000003040] [Citation(s) in RCA: 57] [Impact Index Per Article: 14.3] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/26/2023]
Abstract
PURPOSE Despite the broad endorsement of competency-based medical education (CBME), myriad difficulties have arisen in program implementation. The authors sought to evaluate the fidelity of implementation and identify early outcomes of CBME implementation using Rapid Evaluation to facilitate transformative change. METHOD Case-study methodology was used to explore the lived experience of implementing CBME in the emergency medicine postgraduate program at Queen's University, Canada, using iterative cycles of Rapid Evaluation in 2017-2018. After the intended implementation was explicitly described, stakeholder focus groups and interviews were conducted at 3 and 9 months post-implementation to evaluate the fidelity of implementation and early outcomes. Analyses were abductive, using the CBME core components framework and data-driven approaches to understand stakeholders' experiences. RESULTS In comparing planned with enacted implementation, important themes emerged with resultant opportunities for adaption. For example, lack of a shared mental model resulted in frontline difficulty with assessment and feedback and a concern that the granularity of competency-focused assessment may result in "missing the forest for the trees," prompting the return of global assessment. Resident engagement in personal learning plans was not uniformly adopted, and learning experiences tailored to residents' needs were slow to follow. CONCLUSIONS Rapid Evaluation provided critical insights into the successes and challenges of operationalizing CBME. Implementing the practical components of CBME was perceived as a sprint, while realizing the principles of CBME and changing culture in postgraduate training was a marathon requiring sustained effort in the form of frequent evaluation and continuous faculty and resident development.
Collapse
|
48
|
Kavsak PA, Mondoux SE, Sherbino J, Ma J, Clayton N, Hill SA, McQueen M, Mehta SR, Griffith LE, Devereaux PJ, Worster A. Clinical evaluation of Ortho Clinical Diagnostics high-sensitivity cardiac Troponin I assay in patients with symptoms suggestive of acute coronary syndrome. Clin Biochem 2020; 80:48-51. [PMID: 32304695 DOI: 10.1016/j.clinbiochem.2020.04.003] [Citation(s) in RCA: 12] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/30/2020] [Revised: 04/14/2020] [Accepted: 04/14/2020] [Indexed: 10/24/2022]
Abstract
BACKGROUND As more companies obtain regulatory approval for high-sensitivity cardiac troponin (hs-cTn) assays there is an urgent need for independent analytical and clinical evaluations. To this end, we have evaluated Ortho Clinical Diagnostics' hs-cTnI assay and compared it to their contemporary cTnI-ES assay in emergency department (ED) patients with suspected acute coronary syndrome (ACS). METHODS The study cohort consisted of ED patients (n = 906) with symptoms suggestive of ACS who had Ortho hs-cTnI and cTnI-ES results at presentation and 3 h (with calculated delta (0-3 h) defined as the absolute concentration difference between paired results). The primary composite outcome was 7-day myocardial infarction (MI) or cardiovascular death, with secondary analyses performed for 7-day MI and index-MI. Analytical imprecision testing (i.e., coefficient of variation; CV), receiver-operating characteristic (ROC) curve analyses with area under the curve (AUC), and diagnostic parameters (sensitivity/specificity/predictive values) were calculated. RESULTS The hs-cTnI assay had superior precision compared to the cTnI-ES assay below 5 ng/L in EDTA plasma (hs-cTnI CV ≤ 15% versus cTnI-ES CV ≥ 85%). The AUCs were higher for hs-cTnI as compared to cTnI-ES at 0 h (0.88 vs. 0.85), 3 h (0.94 vs. 0.92), and the delta (0-3 h) value (0.91 vs. 0.85) for the primary composite outcome (p < 0.05). At 3 h, the sensitivity/specificity for index-MI was ≥97%/≥82%, for 7-day MI was ≥89%/≥84%, and for the primary composite outcome was ≥90%/≥85% using the manufacturer's sex-specific 99th-percentile cutoffs. CONCLUSION The Ortho hs-cTnI assay has superior analytical and clinical performance over their contemporary cTnI-ES assay in evaluating ED patients with symptoms suggestive of ACS.
Collapse
|
49
|
Monteiro SD, Sherbino J, Schmidt H, Mamede S, Ilgen J, Norman G. It's the destination: diagnostic accuracy and reasoning. ADVANCES IN HEALTH SCIENCES EDUCATION : THEORY AND PRACTICE 2020; 25:19-29. [PMID: 31332589 DOI: 10.1007/s10459-019-09903-7] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 04/25/2019] [Accepted: 07/09/2019] [Indexed: 06/10/2023]
Abstract
While multiple theories exist to explain the diagnostic process, there are few available assessments that reliably determine diagnostic competence in trainees. Most methods focus on aspects of the process of diagnostic reasoning, such as the relation between case features and diagnostic hypotheses. Inevitably, detailed elucidation of aspects of the process requires substantial time per case and limits the number of cases that can be examined given a limited testing time. Shifting assessment to the outcome of diagnostic reasoning, accuracy of the diagnosis, may serve as a reliable measure of diagnostic competence and would allow increased sampling across cases. The present study is a retrospective analysis of 7 large studies, conducted by 3 research teams, that all used a series of brief written cases to examine the outcome of diagnostic reasoning-the diagnosis. The studies involved over 600 clinicians ranging from final year medical students to practicing emergency physicians. For 4 studies with usable reliability data, reliability for a 2 h test ranged from .63 to .94. On average speeded tests were more reliable (.85 vs. .73).To achieve a reliability of .75 required an average test time of 1.11 h for speeded tests and 1.99 for unspeeded tests. The measure was shown to be positively correlated with both written knowledge tests and measures of problem solving derived from OSCE performance tests. This retrospective analysis provides evidence to support the implementation of outcome-based assessments of clinical reasoning.
Collapse
|
50
|
Monteiro S, Sherbino J, Sibbald M, Norman G. Critical thinking, biases and dual processing: The enduring myth of generalisable skills. MEDICAL EDUCATION 2020; 54:66-73. [PMID: 31468581 DOI: 10.1111/medu.13872] [Citation(s) in RCA: 32] [Impact Index Per Article: 8.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 05/30/2018] [Revised: 10/10/2018] [Accepted: 02/13/2019] [Indexed: 06/10/2023]
Abstract
CONTEXT The myth of generalisable thinking skills in medical education is gaining popularity once again. The implications are significant as medical educators decide on how best to use limited resources to prepare trainees for safe medical practice. This myth-busting critical review cautions against the proliferation of curricular interventions based on the acquisition of generalisable skills. STRUCTURE This paper begins by examining the recent history of general thinking skills, as defined by research in cognitive psychology and medical education. We describe three distinct epochs: (a) the Renaissance, which marked the beginning of cognitive psychology as a discipline in the 1960s and 1970s and was paralleled by educational reforms in medical education focused on problem solving and problem-based learning; (b) the Enlightenment, when an accumulation of evidence in psychology and in medical education cast doubt on the assumption of general reasoning or problem-solving skill and shifted the focus to consideration of the role of knowledge in expert clinical performance; and (c) the Counter-Enlightenment, in the current time, when the notion of general thinking skills has reappeared under different guises, but the fundamental problems related to lack of generality of skills and centrality of knowledge remain. CONCLUSIONS The myth of general thinking skills persists, despite the lack of evidence. Progress in medical education is more likely to arise from devising strategies to improve the breadth and depth of experiential knowledge.
Collapse
|