The effectiveness of cognitive forcing strategies to decrease diagnostic error: an exploratory study

Identifying, Understanding, and Minimizing Unconscious Cognitive Biases in Perioperative Crisis Management: A Narrative Review

Rapid clinical decision-making behavior is often based on pattern recognition and other mental shortcuts. Although such behavior is often faster than deliberative thinking, it can also lead to errors due to unconscious cognitive biases (UCBs). UCBs may contribute to inaccurate diagnoses, hamper interpersonal communication, trigger inappropriate clinical interventions, or result in management delays. The authors review the literature on UCBs and discuss their potential impact on perioperative crisis management. Using the Scale for the Assessment of Narrative Review Articles (SANRA), publications with the most relevance to UCBs in perioperative crisis management were selected for inclusion. Of the 19 UCBs that have been most investigated in the medical literature, the authors identified 9 that were judged to be clinically relevant or most frequently occurring during perioperative crisis management. Formal didactic training on concepts of deliberative thinking has had limited success in reducing the presence of UCBs during clinical decision-making. The evolution of clinical decision support tools (CDSTs) has demonstrated efficacy in improving deliberative clinical decision-making, possibly by reducing the intrusion of maladaptive UCBs and forcing reflective thinking. Anesthesiology remains a leader in perioperative crisis simulation and CDST implementation, but spearheading innovations to reduce the adverse impact of UCBs will further improve diagnostic precision and patient safety during perioperative crisis management.

Ten misconceptions regarding decision-making in critical care

Diagnostic errors are prevalent in critical care practice and are associated with patient harm and costs for providers and the healthcare system. Patient complexity, illness severity, and the urgency in initiating proper treatment all contribute to decision-making errors. Clinician-related factors such as fatigue, cognitive overload, and inexperience further interfere with effective decision-making. Cognitive science has provided insight into the clinical decision-making process that can be used to reduce error. This evidence-based review discusses ten common misconceptions regarding critical care decision-making. By understanding how practitioners make clinical decisions and examining how errors occur, strategies may be developed and implemented to decrease errors in Decision-making and improve patient outcomes.

Dual process models of clinical reasoning: The central role of knowledge in diagnostic expertise

RATIONALE

Research on diagnostic reasoning has been conducted for fifty years or more. There is growing consensus that there are two distinct processes involved in human diagnostic reasoning: System 1, a rapid retrieval of possible diagnostic hypotheses, largely automatic and based to a large part on experiential knowledge, and System 2, a slower, analytical, conscious application of formal knowledge to arrive at a diagnostic conclusion. However, within this broad framework, controversy and disagreement abound. In particular, many authors have suggested that the root cause of diagnostic errors is cognitive biases originating in System 1 and propose that educating learners about the types of cognitive biases and their impact on diagnosis would have a major influence on error reduction.

AIMS AND OBJECTIVES

In the present paper, we take issue with these claims.

METHOD

We reviewed the literature to examine the extent to which this theoretical model is supported by the evidence.

RESULTS

We show that evidence derived from fundamental research in human cognition and studies in clinical medicine challenges the basic assumptions of this theory-that errors arise in System 1 processing as a consequence of cognitive biases, and are corrected by slow, deliberative analytical processing. We claim that, to the contrary, errors derive from both System 1 and System 2 reasoning, that they arise from lack of access to the appropriate knowledge, not from errors of processing, and that the two processes are not essential to the process of diagnostic reasoning.

CONCLUSIONS

The two processing modes are better understood as a consequence of the nature of the knowledge retrieved, not as independent processes.

Using an experiential learning model to teach clinical reasoning theory and cognitive bias: an evaluation of a first-year medical student curriculum

BACKGROUND

Most medical students entering clerkships have limited understanding of clinical reasoning concepts. The value of teaching theories of clinical reasoning and cognitive biases to first-year medical students is unknown. This study aimed to evaluate the value of explicitly teaching clinical reasoning theory and cognitive bias to first-year medical students.

METHODS

Using Kolb's experiential learning model, we introduced dual process theory, script theory, and cognitive biases in teaching clinical reasoning to first-year medical students at an academic medical center in New York City between January and June 2020. Due to the COVID-19 pandemic, instruction was transitioned to a distance learning format in March 2020. The curriculum included a series of written clinical reasoning examinations with facilitated small group discussions. Written self-assessments prompted each student to reflect on the experience, draw conclusions about their clinical reasoning, and plan for future encounters involving clinical reasoning. We evaluated the value of the curriculum using mixed-methods to analyze faculty assessments, student self-assessment questionnaires, and an end-of-curriculum anonymous questionnaire eliciting student feedback.

RESULTS

Among 318 total examinations of 106 students, 254 (80%) had a complete problem representation, while 199 (63%) of problem representations were considered concise. The most common cognitive biases described by students in their clinical reasoning were anchoring bias, availability bias, and premature closure. Four major themes emerged as valuable outcomes of the CREs as identified by students: (1) synthesis of medical knowledge; (2) enhanced ability to generate differential diagnoses; (3) development of self-efficacy related to clinical reasoning; (4) raised awareness of personal cognitive biases.

CONCLUSIONS

We found that explicitly teaching clinical reasoning theory and cognitive biases using an experiential learning model provides first-year medical students with valuable opportunities for developing knowledge, skills, and self-efficacy related to clinical reasoning.

Pursuit of "endpoint diagnoses" as a cognitive forcing strategy to avoid premature diagnostic closure

Premature closure is often described as a significant contributor to diagnostic error. Therefore, developing strategies to mitigate premature closure could reduce diagnostic errors and improve patient care. Here we propose the novel concept of pursuit of an "endpoint diagnosis" as a cognitive forcing strategy (CFS) for avoiding premature diagnostic closure. We define an "endpoint diagnosis" as an underlying causative explanation for a patient's signs, symptoms, and laboratory and radiographic data that exhausts additional relevant diagnostic evaluation. We have observed four contexts in which the error of not pursuing an endpoint diagnosis most often occurs: (1) diagnoses that appear to result in the same treatment regardless of etiology, (2) cases that are particularly complex, (3) clinical scenarios that are vulnerable to systems errors, and (4) situations in which patients' problems are attributed to uncontrolled underlying risk factors or an exacerbation of a known condition. Additionally, we address why we believe endpoint diagnoses are not universally pursued, delineate when this approach might be particularly useful, attempt to reconcile the potential conflict between accepting diagnostic ambiguity in certain instances and pursuing endpoint diagnoses, and outline possible concerns that might arise with using this CFS, including the possibility of lengthy evaluations resulting in overdiagnosis and overtreatment. Our overarching goal is for this CFS to help clinicians in their daily clinical practice as they seek to optimize their diagnostic skill and patient care.

Diagnostic reasoning in cardiovascular medicine

Research in cognitive psychology shows that expert clinicians make a medical diagnosis through a two step process of hypothesis generation and hypothesis testing. Experts generate a list of possible diagnoses quickly and intuitively, drawing on previous experience. Experts remember specific examples of various disease categories as exemplars, which enables rapid access to diagnostic possibilities and gives them an intuitive sense of the base rates of various diagnoses. After generating diagnostic hypotheses, clinicians then test the hypotheses and subjectively estimate the probability of each diagnostic possibility by using a heuristic called anchoring and adjusting. Although both novices and experts use this two step diagnostic process, experts distinguish themselves as better diagnosticians through their ability to mobilize experiential knowledge in a manner that is content specific. Experience is clearly the best teacher, but some educational strategies have been shown to modestly improve diagnostic accuracy. Increased knowledge about the cognitive psychology of the diagnostic process and the pitfalls inherent in the process may inform clinical teachers and help learners and clinicians to improve the accuracy of diagnostic reasoning. This article reviews the literature on the cognitive psychology of diagnostic reasoning in the context of cardiovascular disease.

Teaching Culturally Safe Care in Simulated Cultural Communication Scenarios During the COVID-19 Pandemic: Virtual Visits with Indigenous Animators

Clinical learning activities involving Indigenous patient actors that specifically address the development of culturally safe care skills among medical students are important in order to improve health care for Indigenous people. In 2020, the COVID-19 pandemic led to strict physical distancing regulations and regional lockdowns that made the in-person delivery of Simulated Cultural Communication Scenarios (SCCS) with Indigenous patient actors impossible due to the disproportionate risk that public health emergencies pose for Indigenous communities. As the pandemic continued in 2021, we co-created a Virtual Visit approach to SCCS for the education of culturally safe care to pre-clerkship medical students. We report on student and tutor evaluation of these virtual sessions and contextualize our findings with our previous results delivering In-Person SCCSs. We found that Virtual Visit SCCS were highly effective in providing authentic exposure to and feedback from Indigenous patients. However, students rated their learning outcomes with Virtual Visit lower than the In-person approach to SCCS. We recommend formal training on interacting with patients in virtual care scenarios prior to Virtual Visit SCCS. We also found that exposure to SCCS with Indigenous animators has the potential to conjure up a diverse spectrum of sometimes unresolved negative feelings related to colonialism among students and tutors including discomfort, embarrassment, and anxiety. Our findings underscore the importance of resolving these sentiments within the safe environment of a classroom. To prepare Indigenous as well as non-Indigenous students and tutors adequately, it is important to acknowledge and critically deconstruct the embodiment of colonialism and Indigenous-settler relations when teaching physicians, as well as future physicians, preparedness for culturally safe care of Indigenous peoples.

Intensive Care Unit Decision-Making in Uncertain and Stressful Conditions Part 2: Cognitive Errors, Debiasing Strategies, and Enhancing Critical Thinking

Diagnostic errors are considered a blind spot of health care delivery and occur in up to 15% of patient cases. Cognitive failures are a leading cause of diagnostic error and often occur as a result of overreliance on system 1 thinking. This narrative review describes why diagnostic errors occur by shedding additional light on systems 1 and 2 forms of thinking, reviews literature on debiasing strategies in medicine, and provides a framework for teaching critical thinking in the intensive care unit as a strategy to promote learner development and minimize cognitive failures.

Cognitive Biases in Orthopaedic Surgery

INTRODUCTION

Cognitive biases are known to affect all aspects of human decision-making and reasoning. Examples include misjudgment of probability, preferential attention to evidence that confirms one's beliefs, and preference for certainty. It is not known whether cognitive biases influence orthopaedic surgeon decision-making. This study measured the influence of a few cognitive biases on orthopaedic decision-making in hypothetical vignettes. The questions we addressed were as follows: Do orthopaedic surgeons display the cognitive biases of base rate neglect and confirmation bias in hypothetical vignettes? Can anchoring and framing biases be demonstrated?

METHODS

One hundred ninety-six orthopaedic surgeons completed a survey consisting of three vignettes evaluating base rate neglect, five evaluating confirmation bias, and two separate vignettes each randomly exposing half of the group to different anchors and frames.

RESULTS

For the three vignettes evaluating base rate neglect, 43% (84 of 196) chose answers consistent with base rate neglect in vignette 1, 88% (173 of 196) in vignette 2, and 35% (69 of 196) in vignette 3. Regarding confirmation bias, 51% (100 of 196) chose an answer consistent with confirmation bias for vignette 1, 11% (22 of 196) for vignette 2, 22% (43 of 196) for vignette 3, 22% (44 of 196) for vignette 4, and 29% (56 of 196) for vignette 5. There was a measurable anchoring heuristic (56% versus 34%; a difference of 22%) and framing effect (77% versus 61%; a difference of 16%).

CONCLUSION

The influence of cognitive biases can be documented in patient vignettes presented to orthopaedic surgeons. Strategies can anticipate cognitive bias and develop practice debiasing strategies to limit potential error.

Specific Disease Knowledge as Predictor of Susceptibility to Availability Bias in Diagnostic Reasoning: a Randomized Controlled Experiment

BACKGROUND

Bias in reasoning rather than knowledge gaps has been identified as the origin of most diagnostic errors. However, the role of knowledge in counteracting bias is unclear.

OBJECTIVE

To examine whether knowledge of discriminating features (findings that discriminate between look-alike diseases) predicts susceptibility to bias.

DESIGN

Three-phase randomized experiment. Phase 1 (bias-inducing): Participants were exposed to a set of clinical cases (either hepatitis-IBD or AMI-encephalopathy). Phase 2 (diagnosis): All participants diagnosed the same cases; 4 resembled hepatitis-IBD, 4 AMI-encephalopathy (but all with different diagnoses). Availability bias was expected in the 4 cases similar to those encountered in phase 1. Phase 3 (knowledge evaluation): For each disease, participants decided (max. 2 s) which of 24 findings was associated with the disease. Accuracy of decisions on discriminating features, taken as a measure of knowledge, was expected to predict susceptibility to bias.

PARTICIPANTS

Internal medicine residents at Erasmus MC, Netherlands.

MAIN MEASURES

The frequency with which higher-knowledge and lower-knowledge physicians gave biased diagnoses based on phase 1 exposure (range 0-4). Time to diagnose was also measured.

KEY RESULTS

Sixty-two physicians participated. Higher-knowledge physicians yielded to availability bias less often than lower-knowledge physicians (0.35 vs 0.97; p = 0.001; difference, 0.62 [95% CI, 0.28-0.95]). Whereas lower-knowledge physicians tended to make more of these errors on subjected-to-bias than on not-subjected-to-bias cases (p = 0.06; difference, 0.35 [CI, - 0.02-0.73]), higher-knowledge physicians resisted the bias (p = 0.28). Both groups spent more time to diagnose subjected-to-bias than not-subjected-to-bias cases (p = 0.04), without differences between groups.

CONCLUSIONS

Knowledge of features that discriminate between look-alike diseases reduced susceptibility to bias in a simulated setting. Reflecting further may be required to overcome bias, but succeeding depends on having the appropriate knowledge. Future research should examine whether the findings apply to real practice and to more experienced physicians.

Training in statistical analysis reduces the framing effect among medical students and residents in Argentina

PURPOSE

The framing effect refers to a phenomenon wherein, when the same problem is presented using different representations of information, people make significant changes in their decisions. This study aimed to explore whether the framing effect could be reduced in medical students and residents by teaching them the statistical concepts of effect size, probability, and sampling for use in the medical decision-making process.

METHODS

Ninety-five second-year medical students and 100 second-year medical residents of Austral University and Buenos Aires University, Argentina were invited to participate in the study between March and June 2017. A questionnaire was developed to assess the different types of framing effects in medical situations. After an initial administration of the survey, students and residents were taught statistical concepts including effect size, probability, and sampling during 2 individual independent official biostatistics courses. After these interventions, the same questionnaire was randomly administered again, and pre- and post-intervention outcomes were compared among students and residents.

RESULTS

Almost every type of framing effect was reproduced either in the students or in the residents. After teaching medical students and residents the analytical process behind statistical concepts, a significant reduction in sample-size, risky-choice, pseudo-certainty, number-size, attribute, goal, and probabilistic formulation framing effects was observed.

CONCLUSION

The decision-making of medical students and residents in simulated medical situations may be affected by different frame descriptions, and these framing effects can be partially reduced by training individuals in probability analysis and statistical sampling methods.

Education initiatives in cognitive debiasing to improve diagnostic accuracy in student providers: A scoping review

BACKGROUND

The high prevalence of diagnostic errors by health care providers has prompted medical educators to examine cognitive biases and debiasing strategies in an effort to prevent these errors. The National Academy of Medicine hypothesized that explicit diagnostic reasoning education of all health care professionals can improve diagnostic accuracy.

OBJECTIVES

The purpose of this scoping review is to identify, analyze, and summarize the existing literature on student health care providers' use of cognitive debiasing strategies to reduce diagnostic error.

DATA SOURCES

The review was guided by the Joanna Briggs Institute methodology for scoping reviews. A systematic search of PubMed, CINAHL, PsychINFO, and Scopus databases for debiasing strategies in student provider education yielded 33 studies.

CONCLUSIONS

The 33 studies included in this review represent four categories of debiasing strategies: increased medical knowledge or experience (seven studies), guided reflection (eight studies), self-explanation of reasoning (nine studies), and checklists to expand diagnosis considerations (seven studies). The studies were inclusive of medical students and residents; no studies included nurse practitioner (NP) students. Guided reflection, the most clearly defined and implemented strategy, showed the most promise for improvement of diagnostic accuracy. Educational interventions were wide ranging in content and delivery but did yield a path for future research.

IMPLICATIONS FOR PRACTICE

There are myriad debiasing strategies student providers may use to mitigate cognitive bias. Structured reflection and education initiatives demonstrated the most consistent improvements in diagnostic accuracy. Future studies on debiasing strategies must include NP students to understand their response to these initiatives.

Making Decisions in the Era of the Clinical Decision Rule: How Emergency Physicians Use Clinical Decision Rules

PURPOSE

Physicians are often asked to integrate clinical decision rules (CDRs) with their own cognitive processes to reach a diagnosis. Clinicians, researchers, and educators must understand these cognitive processes to evaluate and improve the diagnostic process. The authors sought to explore emergency physicians' diagnostic processes and to examine how they integrated CDRs into their reasoning using simulated cases (with chest pain or leg pain).

METHOD

From August 2015 to July 2016, 16 practicing emergency physicians from 3 teaching hospitals associated with McMaster University, Ontario, Canada, were interviewed via a novel "teach aloud" protocol. Six videos of simulated patients with chest pain, breathlessness, or leg discomfort were used as prompts for the physicians to demonstrate their diagnostic thinking. Using a constructivist grounded theory analysis, 3 investigators independently reviewed the interview transcripts, meeting regularly to discuss identified themes and subthemes until sufficiency was reached.

RESULTS

A model to describe how clinicians integrate their own decision making with CDRs was developed, showing that physicians engage in an iterative diagnostic process that repeatedly refines the differential diagnosis list. The steps in the diagnostic process were: refinement of the differential diagnosis, ordering a hierarchy of risk, the decision to test, choosing the tests, and interpreting test results. Physicians applied CDRs when they had already decided to test.

CONCLUSIONS

To date, CDRs assume a static, linear model of clinical decision making. Findings demonstrate that participants engaged in iterative and dynamic decision-making processes that changed throughout their patient encounter, contingent on multiple contextual features. Understanding these processes could inform future development of CDRs and educational strategies around these decision aids.

Diagnosing, fast and slow

Diagnostic error is increasingly recognised as a source of significant morbidity and mortality in medicine. In this article, we will attempt to address several questions relating to clinical decision making; How do we decide on a diagnosis? Why do we so often get it wrong? Can we improve our critical faculties?We begin by describing a clinical vignette in which a medical error occurred and resulted in an adverse outcome for a patient. This case leads us to the concepts of heuristic thinking and cognitive bias. We then discuss how this is relevant to our current clinical paradigm, examples of heuristic thinking and potential mechanisms to mitigate bias.The aim of this article is to increase awareness of the role that cognitive bias and heuristic thinking play in medical decision making. We hope to motivate clinicians to reflect on their own patterns of thinking with an overall aim of improving patient care.

Why Physiology Is Critical to the Practice of Medicine: A 40-year Personal Perspective

Accuracy in diagnosis trumps all other elements in clinical decision making. If diagnosis is inaccurate, management is likely to prove futile if not dangerous. Knowledge of physiology provides a periscope for identifying abnormalities beneath the skin responsible for clinical manifestations on the surface. Expert diagnosticians suspect disorders based on pattern recognition and automatic retrieval of knowledge stored in memory. A superior diagnostician looks at the same findings other clinicians see but thinks of causes that others have not imagined. Solving clinical mysteries depends on a clinician's power of imagination, not the capacity to recite an algorithm or apply a protocol.

Immunising' physicians against availability bias in diagnostic reasoning: a randomised controlled experiment

Background

Diagnostic errors have often been attributed to biases in physicians’ reasoning. Interventions to ‘immunise’ physicians against bias have focused on improving reasoning processes and have largely failed.

Objective

To investigate the effect of increasing physicians’ relevant knowledge on their susceptibility to availability bias.

Design, settings and participants

Three-phase multicentre randomised experiment with second-year internal medicine residents from eight teaching hospitals in Brazil.

Interventions

Immunisation: Physicians diagnosed one of two sets of vignettes (either diseases associated with chronic diarrhoea or with jaundice) and compared/contrasted alternative diagnoses with feedback. Biasing phase (1 week later): Physicians were biased towards either inflammatory bowel disease or viral hepatitis. Diagnostic performance test: All physicians diagnosed three vignettes resembling inflammatory bowel disease, three resembling hepatitis (however, all with different diagnoses). Physicians who increased their knowledge of either chronic diarrhoea or jaundice 1 week earlier were expected to resist the bias attempt.

Main outcome measurements

Diagnostic accuracy, measured by test score (range 0–1), computed for subjected-to-bias and not-subjected-to-bias vignettes diagnosed by immunised and not-immunised physicians.

Results

Ninety-one residents participated in the experiment. Diagnostic accuracy differed on subjected-to-bias vignettes, with immunised physicians performing better than non-immunised physicians (0.40 vs 0.24; difference in accuracy 0.16 (95% CI 0.05 to 0.27); p=0.004), but not on not-subjected-to-bias vignettes (0.36 vs 0.41; difference −0.05 (95% CI −0.17 to 0.08); p=0.45). Bias only hampered non-immunised physicians, who performed worse on subjected-to-bias than not-subjected-to-bias vignettes (difference −0.17 (95% CI −0.28 to −0.05); p=0.005); immunised physicians’ accuracy did not differ (p=0.56).

Conclusions

An intervention directed at increasing knowledge of clinical findings that discriminate between similar-looking diseases decreased physicians’ susceptibility to availability bias, reducing diagnostic errors, in a simulated setting. Future research needs to examine the degree to which the intervention benefits other disease clusters and performance in clinical practice.

Trial registration number

68745917.1.1001.0068.

Critical thinking, biases and dual processing: The enduring myth of generalisable skills

CONTEXT

The myth of generalisable thinking skills in medical education is gaining popularity once again. The implications are significant as medical educators decide on how best to use limited resources to prepare trainees for safe medical practice. This myth-busting critical review cautions against the proliferation of curricular interventions based on the acquisition of generalisable skills.

STRUCTURE

This paper begins by examining the recent history of general thinking skills, as defined by research in cognitive psychology and medical education. We describe three distinct epochs: (a) the Renaissance, which marked the beginning of cognitive psychology as a discipline in the 1960s and 1970s and was paralleled by educational reforms in medical education focused on problem solving and problem-based learning; (b) the Enlightenment, when an accumulation of evidence in psychology and in medical education cast doubt on the assumption of general reasoning or problem-solving skill and shifted the focus to consideration of the role of knowledge in expert clinical performance; and (c) the Counter-Enlightenment, in the current time, when the notion of general thinking skills has reappeared under different guises, but the fundamental problems related to lack of generality of skills and centrality of knowledge remain.

CONCLUSIONS

The myth of general thinking skills persists, despite the lack of evidence. Progress in medical education is more likely to arise from devising strategies to improve the breadth and depth of experiential knowledge.

Cognitive bias in clinical medicine

Cognitive bias is increasingly recognised as an important source of medical error, and is both ubiquitous across clinical practice yet incompletely understood. This increasing awareness of bias has resulted in a surge in clinical and psychological research in the area and development of various 'debiasing strategies'. This paper describes the potential origins of bias based on 'dual process thinking', discusses and illustrates a number of the important biases that occur in clinical practice, and considers potential strategies that might be used to mitigate their effect.

Theory-guided teaching: Implementation of a clinical reasoning curriculum in residents

Introduction: Educators have theorized that interventions grounded in dual process theory (DPT) and script theory (ST) may improve the diagnostic reasoning process of physicians but little empirical evidence exists. Methods: In this quasi-experimental study, we assessed the impact of a clinical reasoning (CR) curriculum grounded in DPT and ST on medicine residents participating in one of three groups during a 6-month period: no, partial, or full intervention. Residents completed the diagnostic thinking inventory (DTI) at baseline and 6 months. At 6 months, participants also completed a post-survey assessing application of concepts to cases. Results: There was a significant difference between groups in application of concepts (no intervention 1.6 (0.65) compared to partial 2.3 (0.81) and full 2.2 (0.91), p = 0.05), as well as describing cases in problem representation format (no intervention 1.2 (0.38) and partial 1.5 (0.55) compared to full 2.1 (0.93), p = 0.004). There was no significant difference in change in DTI scores (no intervention 7.0 (16.3), partial 8.8 (9.8), full 7.8 (12.0)). Conclusions: Residents who participated in a CR curriculum grounded in DPT and ST were effective in applying principles of CR in cases from their practice. To our knowledge, this is the first workplace-based CR educational intervention study showing differences in the reasoning process residents apply to patients.

A meta-analysis for comparing effective teaching in clinical education

Aim: Many factors affect learning outcomes, however studies comparing the effectiveness of different clinical teaching methods are limited. We utilize the list of influences on educational achievement compiled by John Hattie to inform a meta-analysis of learning effect sizes (ESs) associated with teaching-learning factors (TLFs) in clinical education. Methods: A literature search was conducted in PubMed to identify articles examining clinically relevant TLFs. Selection criteria were applied to identify learner-focused studies, with subsequent categorization by study design (pretest-posttest or controlled group). The Cohen's ES (d) for each TLF was extracted and a pooled ES determined. Results: From 3454 studies, 132 suitable articles enabled analysis of 16 TLFs' ESs. In general, ESs derived from pretest-posttest data were larger than those from controlled group designs, probably due to learner maturation effect. The TLFs of mastery learning, small group learning and goal settings possessed the largest ESs (d ≥ 0.8), while worked examples, play programs, questioning, concept mapping, meta-cognitive strategies, visual-perception programs and teaching strategies demonstrated ESs between 0.4 and 0.8. Conclusions: This is the first study to provide a rigorous and comprehensive overview of the effectiveness of TLFs in clinical education. We discuss the practical traits shared by effective TLFs which may assist teaching design.

Mortality and morbidity rounds (MMR) in pathology: relative contribution of cognitive bias vs. systems failures to diagnostic error

Background Heuristics and cognitive biases are thought to play an important role in diagnostic medical error. How to systematically determine and capture these kinds of errors remains unclear. Morbidity and mortality rounds (MMRs) are generally focused on reducing medical error by identifying and correcting systems failures. However, they may also provide an educational platform for recognizing and raising awareness on cognitive errors. Methods A total of 49 MMR cases spanning the period 2008-2015 in our pathology department were examined for the presence of cognitive errors and/or systems failures by eight study participant raters who were trained on a subset of 16 of these MMR cases (excluded from the main study analysis) to identify such errors. The Delphi method was used to obtain group consensus on error classification on the remaining 33 study cases. Cases with <75% inter-rater agreement were subjected to subsequent rounds of Delphi analysis. Inter-rater agreement at each round was determined by Fleiss' kappa values. Results Thirty-six percent of the cases presented at our pathology MMRs over an 8-year period were found to contain errors likely due to cognitive bias. Conclusions These data suggest that the errors identified in our pathology MMRs represent not only systems failures but may also be composed of a significant proportion of cognitive errors. Teaching trainees and health professionals to correctly identify different types of cognitive errors may present an opportunity for quality improvement interventions in the interests of patient safety.

Debiasing versus knowledge retrieval checklists to reduce diagnostic error in ECG interpretation

There is an ongoing debate regarding the cause of diagnostic errors. One view is that errors result from unconscious application of cognitive heuristics; the alternative is that errors are a consequence of knowledge deficits. The objective of this study was to compare the effectiveness of checklists that (a) identify and address cognitive biases or (b) promote knowledge retrieval, as a means to reduce errors in ECG interpretation. Novice postgraduate year (PGY) 1 emergency medicine and internal medicine residents (n = 40) and experienced cardiology fellows (PGY 4-6) (n = 21) were randomly allocated to three conditions: a debiasing checklist, a content (knowledge) checklist, or control (no checklist) to be used while interpreting 20 ECGs. Half of the ECGs were deliberately engineered to predispose to bias. Diagnostic performance under either checklist intervention was not significantly better than the control. As expected, more errors occurred when cases were designed to induce bias (F = 96.9, p < 0.0001). There was no significant interaction between the instructional condition and level of learner. Checklists attempting to help learners identify cognitive bias or mobilize domain-specific knowledge did not have an overall effect in reducing diagnostic errors in ECG interpretation, although they may help novices. Even when cognitive biases are deliberately inserted in cases, cognitive debiasing checklists did not improve participants' performance.

Pediatric faculty knowledge and comfort discussing diagnostic errors: a pilot survey to understand barriers to an educational program

Background

Improving Diagnosis in Healthcare calls for improved training in diagnostic reasoning and establishing non-judgmental forums to learn from diagnostic errors arising from heuristic-driven reasoning. Little is known about pediatric providers’ familiarity with heuristics or the culture surrounding forums where diagnostic errors are discussed. This study aimed to describe pediatric providers’ familiarity with common heuristics and perceptions surrounding public discussions of diagnostic errors.

Methods

We surveyed pediatric providers at a university-affiliated children’s hospital. The survey asked participants to identify common heuristics used during clinical reasoning (five definitions; four exemplar clinical vignettes). Participants answered questions regarding comfort publicly discussing their own diagnostic errors and barriers to sharing them.

Results

Seventy (30.6% response rate) faculty completed the survey. The mean number of correctly selected heuristics was 1.60/5 [standard deviation (SD)=1.13] and 1.01/4 (SD=1.06) for the definitions and vignettes, respectively. A low but significant correlation existed between correctly identifying a definition and selecting the correct heuristic in vignettes (Spearman’s ρ=0.27, p=0.02). Clinicians were significantly less likely to be “pretty” or “very” comfortable discussing diagnostic errors in public vs. private conversations (28.3% vs. 74.3%, p<0.01). The most frequently cited barriers to discussing errors were loss of reputation (62.9%) and fear of knowledge-base (58.6%) or decision-making (57.1%) being judged.

Conclusions

Pediatric providers demonstrated limited familiarity with common heuristics leading to diagnostic error. Greater years in practice is associated with more comfort discussing diagnostic errors, but negative peer and personal perceptions of diagnostic performance are common barriers to discussing errors publicly.

Teaching Critical Thinking: A Case for Instruction in Cognitive Biases to Reduce Diagnostic Errors and Improve Patient Safety

Diagnostic errors contribute to as many as 70% of medical errors. Prevention of diagnostic errors is more complex than building safety checks into health care systems; it requires an understanding of critical thinking, of clinical reasoning, and of the cognitive processes through which diagnoses are made. When a diagnostic error is recognized, it is imperative to identify where and how the mistake in clinical reasoning occurred. Cognitive biases may contribute to errors in clinical reasoning. By understanding how physicians make clinical decisions, and examining how errors due to cognitive biases occur, cognitive bias awareness training and debiasing strategies may be developed to decrease diagnostic errors and patient harm. Studies of the impact of teaching critical thinking skills have mixed results but are limited by methodological problems.This Perspective explores the role of clinical reasoning and cognitive bias in diagnostic error, as well as the effect of instruction in metacognitive skills on improvement of diagnostic accuracy for both learners and practitioners. Recent literature questioning whether teaching critical thinking skills increases diagnostic accuracy is critically examined, as are studies suggesting that metacognitive practices result in better patient care and outcomes. Instruction in metacognition, reflective practice, and cognitive bias awareness may help learners move toward adaptive expertise and help clinicians improve diagnostic accuracy. The authors argue that explicit instruction in metacognition in medical education, including awareness of cognitive biases, has the potential to reduce diagnostic errors and thus improve patient safety.

Cognitive error in an academic emergency department

Background

Medical error is a leading cause of death nationwide. While systems issues have been closely investigated as a contributor to error, little is known about the cognitive factors that contribute to diagnostic error in an emergency department (ED) environment.

Methods

Eight months of patient revisits within 72 h where patients were admitted on their second visit were examined. Fifty-two cases of confirmed error were identified and classified using a modified version of the Australian Patient Safety Foundation classification system for medical errors by a group of trained physicians.

Results

Faulty information processing was the most frequently identified category of error (45% of cases), followed by faulty verification (31%). Faulty knowledge (6%) and faulty information gathering (18%) occurred relatively infrequently. “Misjudging the salience of a finding” and “premature closure” were the individual errors that occurred most frequently (13%).

Conclusions

Despite the complex nature of diagnostic reasoning, cognitive errors of information processing appear to occur at higher rates than other errors, and in a similar pattern to an internal medicine service despite a different clinical environment. Further research is needed to elucidate why these errors occur and how to mitigate them.

The evaluation of a continuing professional development package for primary care dentists designed to reduce stress, build resilience and improve clinical decision-making

Introduction Stress and burnout are widely accepted as a problem for primary care dental practitioners. Previous programmes to address this issue have met with some success. Burnout is associated with poor coping skills and emotion regulation, and increased rates of clinical errors. Anxiety is associated with poor decision-making and is thought to be associated with poor clinical decision-making. Attempts to improve decision-making use increasing meta-awareness and review of thinking processes. Bibliotherapy is an effective method of delivering cognitive behavioural therapy as self-help or guided self-help (with some therapist input) formats.Objective To evaluate the efficacy of a specially designed CPD package which was designed to improve coping skills, build resilience and reduce the impact of anxiety on dentists' clinical decision-making.Design A multi-centred quasi-experiment.Setting Lincolnshire and Nottinghamshire (England) 2014.Materials and methods Thirty-five volunteer primary care dentists used two versions (self-help [SH] and guided self-help [GSH], which included a 3 hour workshop) of a specially written cognitive behavioural therapy bibliotherapy programme designed to improve well-being and decision-making.Main Outcome Measures The main outcome measures were dentists' burnout, depression, anxiety, stress and decision-making style. Data were also collected on use and evaluation of the programme.Results At 6 weeks there was a clinically and statistically significant reduction in depression, anxiety and stress levels, a statistically significant reduction in burnout (emotional exhaustion) and hypervigilant decision-making, and an increase in personal achievement. The improvements in depression, stress, emotional exhaustion and hypervigilant decision-making were maintained at 6 months. Dentists were overwhelmingly positive in their evaluation of the project and used most of its contents.Conclusion With the caveat of small numbers and the lack of a no-treatment control, this project demonstrated that a self-help package can be highly acceptable to dentists and, in the short-to-medium term, improve dentists' well-being and decision-making with implications for patient safety.

BET 2: Poor evidence on whether teaching cognitive debiasing, or cognitive forcing strategies, lead to a reduction in errors attributable to cognition in emergency medicine students or doctors

A short review was carried out to see if teaching cognitive forcing strategies reduces cognitive error in the practice of emergency medicine. Two relevant papers were found using the described search strategy. The author, date and country of publication, patient group studied, study type, relevant outcomes, results and study weaknesses of these papers are tabulated. There is currently little evidence that teaching cognitive forcing strategies reduces cognitive error in the practice of emergency medicine.

Diagnostic Errors: Impact of an Educational Intervention on Pediatric Primary Care

INTRODUCTION

The purpose of our study was to determine the impact of an educational program on a provider's knowledge related to diagnostic errors and diagnostic reasoning strategies.

METHODS

A quasi-experimental interventional study with a multimedia approach, case study discussion, and trigger-generated medical record review at two time points was conducted. Measurement tools included a test developed by the National Patient Safety Foundation, Reducing Diagnostic Errors: Strategies for Solutions Quiz, additional diagnostic reasoning questions, and a trigger-generated process to analyze medical records.

RESULTS

Knowledge related to diagnostic errors statistically improved from the pretest to posttest scores with sustained 60-day differences (p < .025). Although there was a decline in the proportion of patients returning with the same chief complaint within 14 days, this was not statistically significant (p < .15). When providers were confronted with an unrecognizable clinical presentation, they reported an increased use of a "diagnostic timeout" (p < .038).

DISCUSSION

Providers developed an increased awareness of the presence of diagnostic errors in the primary care setting, the contributing risk factors for a diagnostic error, and possible strategies to reduce diagnostic errors. These factors had an unexpected impact on changing the primary care practice model to enhance the continuity of patient care.

Evaluation of critical congenital heart defects screening using pulse oximetry in the neonatal intensive care unit

OBJECTIVE

To evaluate the implementation of early screening for critical congenital heart defects (CCHDs) in the neonatal intensive care unit (NICU) and potential exclusion of sub-populations from universal screening.

STUDY DESIGN

Prospective evaluation of CCHD screening at multiple time intervals was conducted in 21 NICUs across five states (n=4556 infants).

RESULTS

Of the 4120 infants with complete screens, 92% did not have prenatal CHD diagnosis or echocardiography before screening, 72% were not receiving oxygen at 24 to 48 h and 56% were born ⩾2500 g. Thirty-seven infants failed screening (0.9%); none with an unsuspected CCHD. False positive rates were low for infants not receiving oxygen (0.5%) and those screened after weaning (0.6%), yet higher among infants born at <28 weeks (3.8%). Unnecessary echocardiograms were minimal (0.2%).

CONCLUSION

Given the majority of NICU infants were ⩾2500 g, not on oxygen and not preidentified for CCHD, systematic screening at 24 to 48 h may be of benefit for early detection of CCHD with minimal burden.

Debiasing Health-Related Judgments and Decision Making: A Systematic Review

BACKGROUND

Being confronted with uncertainty in the context of health-related judgments and decision making can give rise to the occurrence of systematic biases. These biases may detrimentally affect lay persons and health experts alike. Debiasing aims at mitigating these negative effects by eliminating or reducing the biases. However, little is known about its effectiveness. This study seeks to systematically review the research on health-related debiasing to identify new opportunities and challenges for successful debiasing strategies.

METHODS

A systematic search resulted in 2748 abstracts eligible for screening. Sixty-eight articles reporting 87 relevant studies met the predefined inclusion criteria and were categorized and analyzed with regard to content and quality. All steps were undertaken independently by 2 reviewers, and inconsistencies were resolved through discussion.

RESULTS

The majority of debiasing interventions ( n = 60) was at least partially successful. Optimistic biases ( n = 25), framing effects ( n = 14), and base rate neglects ( n = 10) were the main targets of debiasing efforts. Cognitive strategies ( n = 36) such as "consider-the-opposite" and technological interventions ( n = 33) such as visual aids were mainly tested. Thirteen studies aimed at debiasing health care professionals' judgments, while 74 interventions addressed the general population. Studies' methodological quality ranged from 26.2% to 92.9%, with an average rating of 68.7%.

DISCUSSION

In the past, the usefulness of debiasing was often debated. Yet most of the interventions reviewed here are found to be effective, pointing to the utility of debiasing in the health context. In particular, technological strategies offer a novel opportunity to pursue large-scale debiasing outside the laboratory. The need to strengthen the transfer of debiasing interventions to real-life settings and a lack of conceptual rigor are identified as the main challenges requiring further research.

How Expert Clinicians Intuitively Recognize a Medical Diagnosis

Research has shown that expert clinicians make a medical diagnosis through a process of hypothesis generation and verification. Experts begin the diagnostic process by generating a list of diagnostic hypotheses using intuitive, nonanalytic reasoning. Analytic reasoning then allows the clinician to test and verify or reject each hypothesis, leading to a diagnostic conclusion. In this article, we focus on the initial step of hypothesis generation and review how expert clinicians use experiential knowledge to intuitively recognize a medical diagnosis.

The Causes of Errors in Clinical Reasoning: Cognitive Biases, Knowledge Deficits, and Dual Process Thinking

Contemporary theories of clinical reasoning espouse a dual processing model, which consists of a rapid, intuitive component (Type 1) and a slower, logical and analytical component (Type 2). Although the general consensus is that this dual processing model is a valid representation of clinical reasoning, the causes of diagnostic errors remain unclear. Cognitive theories about human memory propose that such errors may arise from both Type 1 and Type 2 reasoning. Errors in Type 1 reasoning may be a consequence of the associative nature of memory, which can lead to cognitive biases. However, the literature indicates that, with increasing expertise (and knowledge), the likelihood of errors decreases. Errors in Type 2 reasoning may result from the limited capacity of working memory, which constrains computational processes. In this article, the authors review the medical literature to answer two substantial questions that arise from this work: (1) To what extent do diagnostic errors originate in Type 1 (intuitive) processes versus in Type 2 (analytical) processes? (2) To what extent are errors a consequence of cognitive biases versus a consequence of knowledge deficits?The literature suggests that both Type 1 and Type 2 processes contribute to errors. Although it is possible to experimentally induce cognitive biases, particularly availability bias, the extent to which these biases actually contribute to diagnostic errors is not well established. Educational strategies directed at the recognition of biases are ineffective in reducing errors; conversely, strategies focused on the reorganization of knowledge to reduce errors have small but consistent benefits.

A neural marker of medical visual expertise: implications for training

Researchers have identified a component of the EEG that discriminates visual experts from novices. The marker indexes a comprehensive model of visual processing, and if it is apparent in physicians, it could be used to investigate the development and training of their visual expertise. The purpose of this study was to determine whether a neural marker of visual expertise-the enhanced N170 event-related potential-is apparent in the EEGs of physicians as they interpret diagnostic images. We conducted a controlled trial with 10 cardiologists and 9 pulmonologists. Each participant completed 520 trials of a standard visual processing task involving the rapid evaluation of EKGs and CXRs-indicating-lung-disease. Ostensibly, each participant is expert with one type of image and competent with the other. We collected behavioral data on the participants' expertise with EKGs and CXRs and electrophysiological data on the magnitude, latency, and scalp location of their N170 ERPs as they interpreted the two types of images. Cardiologists demonstrated significantly more expertise with EKGs than CXRs, and this was reflected in an increased amplitude of their N170 ERPs while reading EKGs compared to CXRs. Pulmonologists demonstrated equal expertise with both types of images, and this was reflected in equal N170 ERP amplitudes for EKGs and CXRs. The results suggest provisionally that visual expertise has a similar substrate in medical practice as it does in other domains that have been studied extensively. This provides support for applying a sophisticated body of literature to questions about training and assessment of visual expertise among physicians.

Reflecting on Diagnostic Errors: Taking a Second Look is Not Enough

BACKGROUND

An experimenter controlled form of reflection has been shown to improve the detection and correction of diagnostic errors in some situations; however, the benefits of participant-controlled reflection have not been assessed.

OBJECTIVE

The goal of the current study is to examine how experience and a self-directed decision to reflect affect the accuracy of revised diagnoses.

DESIGN

Medical residents diagnosed 16 medical cases (pass 1). Participants were then given the opportunity to reflect on each case and revise their diagnoses (pass 2).

PARTICIPANTS

Forty-seven medical Residents in post-graduate year (PGY) 1, 2 and 3 were recruited from Hamilton Health Care Centres.

MAIN MEASURES

Diagnoses were scored as 0 (incorrect), 1 (partially correct) and 2 (correct). Accuracies and response times in pass 1 were analyzed using an ANOVA with three factors-PGY, Decision to revise yes/no, and Case 1-16, averaged across residents. The extent to which additional reflection affected accuracy was examined by analyzing only those cases that were revised, using a repeated measures ANOVA, with pass 1 or 2 as a within subject factor, and PGY and Case or Resident as a between-subject factor.

KEY RESULTS

The mean score at pass 1 for each level was PGY1, 1.17 (SE 0.50); PGY2, 1.35 (SE 0.67) and PGY3, 1.27 (SE 0.94). While there was a trend for increased accuracy with level, this did not achieve significance. The number of residents at each level who revised at least one diagnosis was 12/19 PGY1 (63 %), 9/11 PGY2 (82 %) and 8/17 PGY3 (47 %). Only 8 % of diagnoses were revised resulting in a small but significant increase in scores from Pass 1 to 2, from 1.20/2 to 1.22 /2 (t = 2.15, p = 0.03).

CONCLUSIONS

Participants did engage in self-directed reflection for incorrect diagnoses; however, this strategy provided minimal benefits compared to knowing the correct answer. Education strategies should be directed at improving formal and experiential knowledge.

Complex clinical reasoning in the critical care unit - difficulties, pitfalls and adaptive strategies

Diagnostic error has implications for both clinical outcome and resource utilisation, and may often be traced to impaired data gathering, processing or synthesis because of the influence of cognitive bias. Factors inherent to the intensive/acute care environment afford multiple additional opportunities for such errors to occur. This article illustrates many of these with reference to a case encountered on our intensive care unit. Strategies to improve completeness of data gathering, processing and synthesis in the acute care environment are critically appraised in the context of early detection and amelioration of cognitive bias. These include reflection, targeted simulation training and the integration of social media and IT based aids in complex diagnostic processes. A framework which can be quickly and easily employed in a variety of clinical environments is then presented.

Disrupting diagnostic reasoning: do interruptions, instructions, and experience affect the diagnostic accuracy and response time of residents and emergency physicians?

PURPOSE

Others have suggested that increased time pressure, sometimes caused by interruptions, may result in increased diagnostic errors. The authors previously found, however, that increased time pressure alone does not result in increased errors, but they did not test the effect of interruptions. It is unclear whether experience modulates the combined effects of time pressure and interruptions. This study investigated whether increased time pressure, interruptions, and experience level affect diagnostic accuracy and response time.

METHOD

In October 2012, 152 residents were recruited at five Medical Council of Canada Qualifying Examination Part II test sites. Forty-six emergency physicians were recruited from one Canadian and one U.S. academic health center. Participants diagnosed 20 written general medicine cases. They were randomly assigned to receive fast (time pressure) or slow condition instructions. Visual and auditory case interruptions were manipulated as a within-subject factor.

RESULTS

Diagnostic accuracy was not affected by interruptions or time pressure but was related to experience level: Emergency physicians were more accurate (71%) than residents (43%) (F = 234.0, P < .0001) and responded more quickly (54 seconds) than residents (65 seconds) (F = 9.0, P < .005). Response time was shorter for participants in the fast condition (55 seconds) than in the slow condition (73 seconds) (F = 22.2, P < .0001). Interruptions added about 8 seconds to response time.

CONCLUSIONS

Experienced emergency physicians were both faster and more accurate than residents. Instructions to proceed quickly and interruptions had a small effect on response time but no effect on accuracy.

Ineffectiveness of cognitive forcing strategies to reduce biases in diagnostic reasoning: a controlled trial

Objectives:

Cognitive forcing strategies (CFS)may reduce error arising from cognitive biases. This is the first experimental test to determine the effect of CFS training in medical students.

Methods:

Students were allocated to CFS training or control during a 4-week emergency medicine rotation (n = 191). At the end of the rotation examination, students were tested using computer-based cases. Application of CFS could enable reduction of diagnostic error, as evidenced by identifying multiple correct diagnoses for the two cases prone to search satisficing bias (SSB) and uncommon diagnoses for the two cases prone to availability bias (AB). Two “false positive” cases were included to test for possible “oversearching.”

Results:

There were 145 students in the intervention and 46 in the control group. For the SSB cases, 52% of students with CFS training and 48% in the control group initiated a search for the second diagnosis (χ2 = 0.13, df = 1, p = 0.91). More than half (54%) correctly identified the second diagnosis in the CFS group, and 48% identified it in the control group. The difference was not significant (χ2 = 2.25, df = 1, p = 0.13). For the second diagnosis in the false positive cases, 64% of the CFS group and 77% of the control group incorrectly identified it. There were no significant differences between groups (χ2 = 2.38, df = 1, p = 0.12). In the AB cases, only 45% in each group identified the uncommon correct diagnosis (χ2 = 0.001, df = 1, p = 0.98).

Conclusions:

The educational interventions suggested by experts in clinical reasoning and employed in our study to teach CFS failed to show any reduction in diagnostic error by novices.

Cognitive debiasing 2: impediments to and strategies for change

In a companion paper, we proposed that cognitive debiasing is a skill essential in developing sound clinical reasoning to mitigate the incidence of diagnostic failure. We reviewed the origins of cognitive biases and some proposed mechanisms for how debiasing processes might work. In this paper, we first outline a general schema of how cognitive change occurs and the constraints that may apply. We review a variety of individual factors, many of them biases themselves, which may be impediments to change. We then examine the major strategies that have been developed in the social sciences and in medicine to achieve cognitive and affective debiasing, including the important concept of forcing functions. The abundance and rich variety of approaches that exist in the literature and in individual clinical domains illustrate the difficulties inherent in achieving cognitive change, and also the need for such interventions. Ongoing cognitive debiasing is arguably the most important feature of the critical thinker and the well-calibrated mind. We outline three groups of suggested interventions going forward: educational strategies, workplace strategies and forcing functions. We stress the importance of ambient and contextual influences on the quality of individual decision making and the need to address factors known to impair calibration of the decision maker. We also emphasise the importance of introducing these concepts and corollary development of training in critical thinking in the undergraduate level in medical education.

Educational agenda for diagnostic error reduction

Diagnostic errors are a major patient safety concern. Although the majority of diagnostic errors are partially attributable to cognitive mistakes, the most effective means of improving clinician cognition in order to achieve gains in diagnostic reliability are unclear. We propose a tripartite educational agenda for improving diagnostic performance among students, residents and practising physicians. This agenda includes strengthening the metacognitive abilities of clinicians, fostering intuitive reasoning and increasing awareness of the role of systems in the diagnostic process. The evidence supporting initiatives in each of these realms is reviewed and a course of future implementation and study is proposed. The barriers to designing and implementing this agenda are substantial and include limited evidence supporting these initiatives and the challenges of changing the practice patterns of practising physicians. Implementation will need to be accompanied by rigorous evaluation.

Medical error, disclosure and patient safety: a global view of quality care

Medical errors are a prominent issue in health care. Numerous studies point at the high prevalence of adverse events, many of which are preventable. Although there is a range of severity in errors, they all cause harm, to the patient, to the system, or both. While errors have many causes, including human interactions and system inadequacies, the focus on individuals rather than the system has led to an unsuitable culture for improving patient safety. Important areas of focus are diagnostic procedures and clinical laboratories because their results play a major role in guiding clinical decisions in patient management. Proper disclosure of medical errors and adverse events is also a key area for improvement. Globally, system improvements are beginning to take place, however, in Canada, policies on disclosure, error reporting and protection for physicians remain non-uniform. Achieving a national standard with mandatory reporting, in addition to a non-punitive system is recommended to move forward.

Critical appraisal of emergency medicine educational research: the best publications of 2011

OBJECTIVES

The objective was to critically appraise and highlight medical education research studies published in 2011 that were methodologically superior and whose outcomes were pertinent to teaching and education in emergency medicine (EM).

METHODS

A search of the English language literature in 2011 querying PubMed, Scopus, Education Resources Information Center (ERIC), and PsychInfo identified EM studies that used hypothesis-testing or observational investigations of educational interventions. Six reviewers independently ranked all publications based on 10 criteria, including four related to methodology, that were chosen a priori to standardize evaluation by reviewers. This method was used previously to appraise medical education published in 2008, 2009, and 2010.

RESULTS

Forty-eight educational research papers were identified. Comparing the literature of 2011 to that of 2008 through 2010, the number of published educational research papers meeting the criteria increased over time from 30, to 36, to 41, and now to 48. Five medical education research studies met the a priori criteria for inclusion as exemplary and are reviewed and summarized in this article. The number of funded studies remained fairly stable over the past 3 years, at 13 (2008), 16 (2009), 9 (2010), and 13 (2011). As in past years, research involving the use of technology accounted for almost half (n = 22) of the publications. Observational study designs accounted for 28 of the papers, while nine studies featured an experimental design.

CONCLUSIONS

Forty-eight EM educational studies published in 2011 and meeting the criteria were identified. This critical appraisal reviews and highlights five studies that met a priori quality indicators. Current trends and common methodologic pitfalls in the 2011 papers are noted.

Diagnostic reasoning: where we've been, where we're going

Recently, clinical diagnostic reasoning has been characterized by "dual processing" models, which postulate a fast, unconscious (System 1) component and a slow, logical, analytical (System 2) component. However, there are a number of variants of this basic model, which may lead to conflicting claims. This paper critically reviews current theories and evidence about the nature of clinical diagnostic reasoning. We begin by briefly discussing the history of research in clinical reasoning. We then focus more specifically on the evidence to support dual-processing models. We conclude by identifying knowledge gaps about clinical reasoning and provide suggestions for future research. In contrast to work on analytical and nonanalytical knowledge as a basis for reasoning, these theories focus on the thinking process, not the nature of the knowledge retrieved. Ironically, this appears to be a revival of an outdated concept. Rather than defining diagnostic performance by problem-solving skills, it is now being defined by processing strategy. The version of dual processing that has received most attention in the literature in medical diagnosis might be labeled a "default/interventionist" model,(17) which suggests that a default system of cognitive processes (System 1) is responsible for cognitive biases that lead to diagnostic errors and that System 2 intervenes to correct these errors. Consequently, from this model, the best strategy for reducing errors is to make students aware of the biases and to encourage them to rely more on System 2. However, an accumulation of evidence suggests that (a) strategies directed at increasing analytical (System 2) processing, by slowing down, reducing distractions, paying conscious attention, and (b) strategies directed at making students aware of the effect of cognitive biases, have no impact on error rates. Conversely, strategies based on increasing application of relevant knowledge appear to have some success and are consistent with basic research on concept formation.

The relationship between response time and diagnostic accuracy

PURPOSE

Psychologists theorize that cognitive reasoning involves two distinct processes: System 1, which is rapid, unconscious, and contextual, and System 2, which is slow, logical, and rational. According to the literature, diagnostic errors arise primarily from System 1 reasoning, and therefore they are associated with rapid diagnosis. This study tested whether accuracy is associated with shorter or longer times to diagnosis.

METHOD

Immediately after the 2010 administration of the Medical Council of Canada Qualifying Examination (MCCQE) Part II at three test centers, the authors recruited participants, who read and diagnosed a series of 25 written cases of varying difficulty. The authors computed accuracy and response time (RT) for each case.

RESULTS

Seventy-five Canadian medical graduates (of 95 potential participants) participated. The overall correlation between RT and accuracy was -0.54; accuracy, then, was strongly associated with more rapid RT. This negative relationship with RT held for 23 of 25 cases individually and overall when the authors controlled for participants' knowledge, as judged by their MCCQE Part I and II scores. For 19 of 25 cases, accuracy on each case was positively related to experience with that specific diagnosis. A participant's performance on the test overall was significantly correlated with his or her performance on both the MCCQE Part I and II.

CONCLUSIONS

These results are inconsistent with clinical reasoning models that presume that System 1 reasoning is necessarily more error prone than System 2. These results suggest instead that rapid diagnosis is accurate and relates to other measures of competence.