1
|
Staal J, Waechter J, Allen J, Lee CH, Zwaan L. Deliberate practice of diagnostic clinical reasoning reveals low performance and improvement of diagnostic justification in pre-clerkship students. BMC MEDICAL EDUCATION 2023; 23:684. [PMID: 37735677 PMCID: PMC10515060 DOI: 10.1186/s12909-023-04541-5] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 02/24/2023] [Accepted: 07/27/2023] [Indexed: 09/23/2023]
Abstract
PURPOSE Diagnostic errors are a large burden on patient safety and improving clinical reasoning (CR) education could contribute to reducing these errors. To this end, calls have been made to implement CR training as early as the first year of medical school. However, much is still unknown about pre-clerkship students' reasoning processes. The current study aimed to observe how pre-clerkship students use clinical information during the diagnostic process. METHODS In a prospective observational study, pre-clerkship medical students completed 10-11 self-directed online simulated CR diagnostic cases. CR skills assessed included: creation of the differential diagnosis (Ddx), diagnostic justification (DxJ), ordering investigations, and identifying the most probable diagnosis. Student performances were compared to expert-created scorecards and students received detailed individualized formative feedback for every case. RESULTS 121 of 133 (91%) first- and second-year medical students consented to the research project. Students scored much lower for DxJ compared to scores obtained for creation of the Ddx, ordering tests, and identifying the correct diagnosis, (30-48% lower, p < 0.001). Specifically, students underutilized physical exam data (p < 0.001) and underutilized data that decreased the probability of incorrect diagnoses (p < 0.001). We observed that DxJ scores increased 40% after 10-11 practice cases (p < 0.001). CONCLUSIONS We implemented deliberate practice with formative feedback for CR starting in the first year of medical school. Students underperformed in DxJ, particularly with analyzing the physical exam data and pertinent negative data. We observed significant improvement in DxJ performance with increased practice.
Collapse
Affiliation(s)
- Justine Staal
- Erasmus Medical Center, Institute of Medical Education Research Rotterdam, Rotterdam, the Netherlands
| | - Jason Waechter
- Department of Critical Care, Cumming School of Medicine, University of Calgary, Calgary, AB, Canada
| | - Jon Allen
- Department of Medicine, University of North Dakota School of Medicine and Health Sciences, Grand Forks, ND, USA
| | - Chel Hee Lee
- Department of Mathematics, University of Calgary, Calgary, AB, Canada
| | - Laura Zwaan
- Department of Critical Care, Cumming School of Medicine, University of Calgary, Calgary, AB, Canada.
- Institute of Medical Education Research, Erasmus Medical Center, Dr. Molewaterplein 40, Rotterdam, 3015 GD, the Netherlands.
| |
Collapse
|
2
|
Staal J, Zegers R, Caljouw-Vos J, Mamede S, Zwaan L. Impact of diagnostic checklists on the interpretation of normal and abnormal electrocardiograms. Diagnosis (Berl) 2023; 10:121-129. [PMID: 36490202 DOI: 10.1515/dx-2022-0092] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/19/2022] [Accepted: 11/27/2022] [Indexed: 12/13/2022]
Abstract
OBJECTIVES Checklists that aim to support clinicians' diagnostic reasoning processes are often recommended to prevent diagnostic errors. Evidence on checklist effectiveness is mixed and seems to depend on checklist type, case difficulty, and participants' expertise. Existing studies primarily use abnormal cases, leaving it unclear how the diagnosis of normal cases is affected by checklist use. We investigated how content-specific and debiasing checklists impacted performance for normal and abnormal cases in electrocardiogram (ECG) diagnosis. METHODS In this randomized experiment, 42 first year general practice residents interpreted normal, simple abnormal, and complex abnormal ECGs without a checklist. One week later, they were randomly assigned to diagnose the ECGs again with either a debiasing or content-specific checklist. We measured residents' diagnostic accuracy, confidence, patient management, and time taken to diagnose. Additionally, confidence-accuracy calibration was assessed. RESULTS Accuracy, confidence, and patient management were not significantly affected by checklist use. Time to diagnose decreased with a checklist (M=147s (77)) compared to without a checklist (M=189s (80), Z=-3.10, p=0.002). Additionally, residents' calibration improved when using a checklist (phase 1: R2=0.14, phase 2: R2=0.40). CONCLUSIONS In both normal and abnormal cases, checklist use improved confidence-accuracy calibration, though accuracy and confidence were not significantly affected. Time to diagnose was reduced. Future research should evaluate this effect in more experienced GPs. Checklists appear promising for reducing overconfidence without negatively impacting normal or simple ECGs. Reducing overconfidence has the potential to improve diagnostic performance in the long term.
Collapse
Affiliation(s)
- Justine Staal
- Erasmus Medical Center Rotterdam, Institute of Medical Education Research Rotterdam, Rotterdam, The Netherlands
| | - Robert Zegers
- Department of General Practice, Erasmus Medical Center Rotterdam, Rotterdam, The Netherlands
| | | | - Sílvia Mamede
- Erasmus Medical Center Rotterdam, Institute of Medical Education Research Rotterdam, Rotterdam, The Netherlands
- Department of Psychology, Education and Child Studies, Erasmus School of Social and Behavioral Sciences, Rotterdam, The Netherlands
| | - Laura Zwaan
- Erasmus Medical Center Rotterdam, Institute of Medical Education Research Rotterdam, Rotterdam, The Netherlands
| |
Collapse
|
3
|
Behavioral Economics: A Primer and Applications to the UN Sustainable Development Goal of Good Health and Well-Being. REPORTS 2021. [DOI: 10.3390/reports4020016] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/16/2022] Open
Abstract
Behavioral economics (BE) is a relatively new field within economics that incorporates insights from psychology that can be harnessed to improve economic decision making with the potential to enhance good health and well-being of individuals and societies, the third of the United Nations Sustainable Development Goals. While some of the psychological principles of economic decision making were described as far back as the 1700s by Adam Smith, BE emerged as a discipline in the 1970s with the groundbreaking work of psychologists Daniel Kahneman and Amos Tversky. We describe the basic concepts of BE, heuristics (decision-making shortcuts) and their associated biases, and the BE strategies framing, incentives, and economic nudging to overcome these biases. We survey the literature to identify how BE techniques have been employed to improve individual choice (focusing on childhood obesity), health policy, and patient and healthcare provider decision making. Additionally, we discuss how these BE-based efforts to improve health-related decision making can lead to sustaining good health and well-being and identify additional health-related areas that may benefit from including principles of BE in decision making.
Collapse
|
4
|
Staal J, Alsma J, Mamede S, Olson APJ, Prins-van Gilst G, Geerlings SE, Plesac M, Sundberg MA, Frens MA, Schmidt HG, Van den Broek WW, Zwaan L. The relationship between time to diagnose and diagnostic accuracy among internal medicine residents: a randomized experiment. BMC MEDICAL EDUCATION 2021; 21:227. [PMID: 33882919 PMCID: PMC8061054 DOI: 10.1186/s12909-021-02671-2] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 01/08/2021] [Accepted: 04/08/2021] [Indexed: 05/02/2023]
Abstract
BACKGROUND Diagnostic errors have been attributed to cognitive biases (reasoning shortcuts), which are thought to result from fast reasoning. Suggested solutions include slowing down the reasoning process. However, slower reasoning is not necessarily more accurate than faster reasoning. In this study, we studied the relationship between time to diagnose and diagnostic accuracy. METHODS We conducted a multi-center within-subjects experiment where we prospectively induced availability bias (using Mamede et al.'s methodology) in 117 internal medicine residents. Subsequently, residents diagnosed cases that resembled those bias cases but had another correct diagnosis. We determined whether residents were correct, incorrect due to bias (i.e. they provided the diagnosis induced by availability bias) or due to other causes (i.e. they provided another incorrect diagnosis) and compared time to diagnose. RESULTS We did not successfully induce bias: no significant effect of availability bias was found. Therefore, we compared correct diagnoses to all incorrect diagnoses. Residents reached correct diagnoses faster than incorrect diagnoses (115 s vs. 129 s, p < .001). Exploratory analyses of cases where bias was induced showed a trend of time to diagnose for bias diagnoses to be more similar to correct diagnoses (115 s vs 115 s, p = .971) than to other errors (115 s vs 136 s, p = .082). CONCLUSIONS We showed that correct diagnoses were made faster than incorrect diagnoses, even within subjects. Errors due to availability bias may be different: exploratory analyses suggest a trend that biased cases were diagnosed faster than incorrect diagnoses. The hypothesis that fast reasoning leads to diagnostic errors should be revisited, but more research into the characteristics of cognitive biases is important because they may be different from other causes of diagnostic errors.
Collapse
Affiliation(s)
- J Staal
- Erasmus Medical Center Rotterdam, Institute of Medical Education Research Rotterdam, Rotterdam, The Netherlands.
| | - J Alsma
- Department of Internal Medicine, Erasmus Medical Center Rotterdam, Rotterdam, The Netherlands
| | - S Mamede
- Erasmus Medical Center Rotterdam, Institute of Medical Education Research Rotterdam, Rotterdam, The Netherlands
| | - A P J Olson
- Division of General Internal Medicine, University of Minnesota, Section of Hospital Medicine, Minneapolis, USA
| | - G Prins-van Gilst
- Department of Internal Medicine, Erasmus Medical Center Rotterdam, Rotterdam, The Netherlands
| | - S E Geerlings
- Department of Internal Medicine and Department of Infectious Diseases, Academic Medical Center Amsterdam, Amsterdam, The Netherlands
| | - M Plesac
- Division of General Internal Medicine, University of Minnesota, Section of Hospital Medicine, Minneapolis, USA
| | - M A Sundberg
- Division of General Internal Medicine, University of Minnesota, Section of Hospital Medicine, Minneapolis, USA
| | - M A Frens
- Department of Neuroscience, Erasmus Medical Center Rotterdam, Rotterdam, The Netherlands
| | - H G Schmidt
- Erasmus Medical Center Rotterdam, Institute of Medical Education Research Rotterdam, Rotterdam, The Netherlands
| | - W W Van den Broek
- Erasmus Medical Center Rotterdam, Institute of Medical Education Research Rotterdam, Rotterdam, The Netherlands
| | - L Zwaan
- Erasmus Medical Center Rotterdam, Institute of Medical Education Research Rotterdam, Rotterdam, The Netherlands
| |
Collapse
|
5
|
van Peppen LM, Verkoeijen PPJL, Kolenbrander SV, Heijltjes AEG, Janssen EM, van Gog T. Learning to avoid biased reasoning: effects of interleaved practice and worked examples. JOURNAL OF COGNITIVE PSYCHOLOGY 2021. [DOI: 10.1080/20445911.2021.1890092] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/22/2022]
Affiliation(s)
- Lara M. van Peppen
- Department of Psychology, Education and Child Studies, Erasmus University Rotterdam Rotterdam, the Netherlands
- Institute of Medical Education Research, Erasmus MC, University Medical Center Rotterdam, Rotterdam, the Netherlands
| | - Peter P. J. L. Verkoeijen
- Department of Psychology, Education and Child Studies, Erasmus University Rotterdam Rotterdam, the Netherlands
- Learning and Innovation Center, Avans University of Applied Sciences Breda, the Netherlands
| | - Stefan V. Kolenbrander
- Learning and Innovation Center, Avans University of Applied Sciences Breda, the Netherlands
| | - Anita E. G. Heijltjes
- Learning and Innovation Center, Avans University of Applied Sciences Breda, the Netherlands
| | - Eva M. Janssen
- Department of Education, Utrecht University Utrecht, the Netherlands
| | - Tamara van Gog
- Department of Education, Utrecht University Utrecht, the Netherlands
| |
Collapse
|
6
|
Hansen K. Cognitive bias in emergency medicine. Emerg Med Australas 2020; 32:852-855. [DOI: 10.1111/1742-6723.13622] [Citation(s) in RCA: 8] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/21/2020] [Accepted: 08/21/2020] [Indexed: 12/15/2022]
Affiliation(s)
- Kim Hansen
- Emergency Department St Andrew's War Memorial Hospital Brisbane Queensland Australia
- Emergency Department The Prince Charles Hospital Brisbane Queensland Australia
- Faculty of Medicine The University of Queensland Brisbane Queensland Australia
| |
Collapse
|
7
|
Catalano AS, Redford K, Margoluis R, Knight AT. Black swans, cognition, and the power of learning from failure. CONSERVATION BIOLOGY : THE JOURNAL OF THE SOCIETY FOR CONSERVATION BIOLOGY 2018; 32:584-596. [PMID: 29094402 DOI: 10.1111/cobi.13045] [Citation(s) in RCA: 15] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 06/27/2017] [Revised: 09/03/2017] [Accepted: 10/11/2017] [Indexed: 06/07/2023]
Abstract
Failure carries undeniable stigma and is difficult to confront for individuals, teams, and organizations. Disciplines such as commercial and military aviation, medicine, and business have long histories of grappling with it, beginning with the recognition that failure is inevitable in every human endeavor. Although conservation may arguably be more complex, conservation professionals can draw on the research and experience of these other disciplines to institutionalize activities and attitudes that foster learning from failure, whether they are minor setbacks or major disasters. Understanding the role of individual cognitive biases, team psychological safety, and organizational willingness to support critical self-examination all contribute to creating a cultural shift in conservation to one that is open to the learning opportunity that failure provides. This new approach to managing failure is a necessary next step in the evolution of conservation effectiveness.
Collapse
Affiliation(s)
- Allison S Catalano
- Department of Life Sciences, Imperial College London, Ascot, Berkshire SL5 7PY, U.K
- The Silwood Group, Imperial College London, Ascot, Berkshire SL5 7PY, U.K
| | - Kent Redford
- The Silwood Group, Imperial College London, Ascot, Berkshire SL5 7PY, U.K
- Archipelago Consulting, Box 4750, Portland, ME 04112, U.S.A
- Department of Environmental Studies, University of New England, 11 Hills Beach Rd., Biddeford, ME 04005, U.S.A
| | - Richard Margoluis
- The Gordon and Betty Moore Foundation, 1661 Page Mill Rd., Palo Alto, CA 94304, U.S.A
| | - Andrew T Knight
- Department of Life Sciences, Imperial College London, Ascot, Berkshire SL5 7PY, U.K
- The Silwood Group, Imperial College London, Ascot, Berkshire SL5 7PY, U.K
- ARC Centre of Excellence in Environmental Decisions, The University of Queensland, St. Lucia, Queensland 4072, Australia
- Department of Botany, Nelson Mandela University, PO Box 77000, Port Elizabeth 6031, South Africa
| |
Collapse
|
8
|
Mamede S, Schmidt HG. Reflection in Medical Diagnosis: A Literature Review. HEALTH PROFESSIONS EDUCATION 2017. [DOI: 10.1016/j.hpe.2017.01.003] [Citation(s) in RCA: 42] [Impact Index Per Article: 6.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/17/2022] Open
|
9
|
Cardiac Arrest and Gastrointestinal Bleeding: A Case of Medical Heuristics. Case Rep Med 2016; 2016:9621390. [PMID: 27366152 PMCID: PMC4913064 DOI: 10.1155/2016/9621390] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.4] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/21/2016] [Accepted: 05/17/2016] [Indexed: 11/17/2022] Open
Abstract
Insufficient clinical data from patients is a major cause of errors in medical diagnostics. In an attempt to make a diagnosis, initial clinical information provided to the physician may be overly relied on as the only information required in making diagnosis leading to anchoring. Failure to rely on differential diagnoses in spite of new signs and symptoms or rethinking of initial hypothesis may lead to fixation on a certain diagnosis, which may lead to significant morbidity and mortality. In the event that there is an anchoring heuristic, like in our patient, it is important to consider differential diagnoses; however, it is not wrong to rely on some form of anchor. We report a case of a 62-year-old male with a history of multiple medical conditions and a history of acetaminophen overdose who presented to the hospital with large amounts of coffee ground emesis. He was subsequently transferred to the liver transplant center on discovery that he was in fulminant hepatic failure and died two days later in spite of aggressive medical treatment.
Collapse
|