Using voluntary reports from physicians to learn from diagnostic errors in emergency medicine

Minimizing the Risk of Diagnostic Errors in Acute Care for Older Adults: An Interdisciplinary Patient Safety Challenge

Modern healthcare systems are increasingly organized according to diagnosis-specific clinical pathways and treatment protocols. At the same time, the number of patients with complex problems and needs that do not fit the single-diagnosis approach is rising, contributing to a high prevalence of diagnostic errors. In this article, we focus on the risk of diagnostic errors arising from missed or incomplete diagnosis and assessment of older adult patients' care needs in the first hours of acute hospitalizations in EDs. This focus is important for improving patient safety, as clinical decisions made in EDs impact patient safety in the subsequent steps of the process, thereby potentially causing new risks to arise. Based on our discussion of clinical decision-making and diagnostic errors in the acute care context, we propose a more comprehensive interdisciplinary approach to improvements in patient safety that integrates organizational and clinical research and examines where, when, how, and why risks to patient safety arise in and across different clinical-organizational contexts.

Types of diagnostic errors reported by paediatric emergency providers in a global paediatric emergency care research network

Background

Diagnostic errors, reframed as missed opportunities for improving diagnosis (MOIDs), are poorly understood in the paediatric emergency department (ED) setting. We investigated the clinical experience, harm and contributing factors related to MOIDs reported by physicians working in paediatric EDs.

Methods

We developed a web-based survey in which physicians participating in the international Paediatric Emergency Research Network representing five out of six WHO regions, described examples of MOIDs involving their own or a colleague’s patients. Respondents provided case summaries and answered questions regarding harm and factors contributing to the event.

Results

Of 1594 physicians surveyed, 412 (25.8%) responded (mean age=43 years (SD=9.2), 42.0% female, mean years in practice=12 (SD=9.0)). Patient presentations involving MOIDs had common undifferentiated symptoms at initial presentation, including abdominal pain (21.1%), fever (17.2%) and vomiting (16.5%). Patients were discharged from the ED with commonly reported diagnoses, including acute gastroenteritis (16.7%), viral syndrome (10.2%) and constipation (7.0%). Most reported MOIDs (65%) were detected on ED return visits (46% within 24 hours and 76% within 72 hours). The most common reported MOID was appendicitis (11.4%), followed by brain tumour (4.4%), meningitis (4.4%) and non-accidental trauma (4.1%). More than half (59.1%) of the reported MOIDs involved the patient/parent–provider encounter (eg, misinterpreted/ignored history or an incomplete/inadequate physical examination). Types of MOIDs and contributing factors did not differ significantly between countries. More than half of patients had either moderate (48.7%) or major (10%) harm due to the MOID.

Conclusions

An international cohort of paediatric ED physicians reported several MOIDs, often in children who presented to the ED with common undifferentiated symptoms. Many of these were related to patient/parent–provider interaction factors such as suboptimal history and physical examination. Physicians’ personal experiences offer an underexplored source for investigating and mitigating diagnostic errors in the paediatric ED.

Diagnostic Delays in Sepsis: Lessons Learned From a Retrospective Study of Canadian Medico-Legal Claims

Although rapid treatment improves outcomes for patients presenting with sepsis, early detection can be difficult, especially in otherwise healthy adults.

OBJECTIVES

Using medico-legal data, we aimed to identify areas of focus to assist with early recognition of sepsis.

DESIGN SETTING AND PARTICIPANTS

Retrospective descriptive design. We analyzed closed medico-legal cases involving physicians from a national database repository at the Canadian Medical Protective Association. The study included cases closed between 2011 and 2020 that had documented peer expert criticism of a diagnostic issue related to sepsis or relevant infections.

MAIN OUTCOMES AND MEASURES

We used univariate statistics to describe patients and physicians and applied published frameworks to classify contributing factors (provider, team, system) and diagnostic pitfalls based on peer expert criticisms.

RESULTS

Of 162 involved patients, the median age was 53 years (interquartile range [IQR], 34-66 yr) and mortality was 49%. Of 218 implicated physicians, 169 (78%) were from family medicine, emergency medicine, or surgical specialties. Eighty patients (49%) made multiple visits to outpatient care leading up to sepsis recognition/hospitalization (median = two visits; IQR, 2-4). Almost 40% of patients were admitted to the ICU. Deficient assessments, such as failing to consider sepsis or not reassessing the patient prior to discharge, contributed to the majority of cases (81%).

CONCLUSIONS AND RELEVANCE

Sepsis continues to be a challenging diagnosis for clinicians. Multiple visits to outpatient care may be an early warning sign requiring vigilance in the patient assessment.

Creating a Learning Health System for Improving Diagnostic Safety: Pragmatic Insights from US Health Care Organizations

OBJECTIVE

To identify challenges and pragmatic strategies for improving diagnostic safety at an organizational level using concepts from learning health systems METHODS: We interviewed 32 safety leaders across the USA on how their organizations approach diagnostic safety. Participants were recruited through email and represented geographically diverse academic and non-academic settings. The interview included questions on culture of reporting and learning from diagnostic errors; data gathering and analysis activities; diagnostic training and educational activities; and engagement of clinical leadership, staff, patients, and families in diagnostic safety activities. We conducted an inductive content analysis of interview transcripts and two reviewers coded all data.

RESULTS

Of 32 participants, 12 reported having a specific program to address diagnostic errors. Multiple barriers to implement diagnostic safety activities emerged: serious concerns about psychological safety associated with diagnostic error; lack of infrastructure for measurement, monitoring, and improvement activities related to diagnosis; lack of leadership investment, which was often diverted to competing priorities related to publicly reported measures or other incentives; and lack of dedicated teams to work on diagnostic safety. Participants provided several strategies to overcome barriers including adapting trigger tools to identify safety events, engaging patients in diagnostic safety, and appointing dedicated diagnostic safety champions.

CONCLUSIONS

Several foundational building blocks related to learning health systems could inform organizational efforts to reduce diagnostic error. Promoting an organizational culture specific to diagnostic safety, using science and informatics to improve measurement and analysis, leadership incentives to build institutional capacity to address diagnostic errors, and patient engagement in diagnostic safety activities can enable progress.

Frequency and predictors of unspecific medical diagnoses in the emergency department: a prospective observational study

Background

Misdiagnosis is a major public health problem, causing increased morbidity and mortality. In the busy setting of an emergency department (ED) patients are diagnosed under difficult circumstances. As a consequence, the ED diagnosis at hospital admittance may often be a descriptive diagnosis, such as “decreased general condition”. Our objective was to determine in how far patients with such an unspecific ED diagnosis differ from patients with a specific ED diagnosis and whether they experience a worse outcome.

Methods

We conducted a prospective observational study in Bern university hospital in Switzerland for all adult non-trauma patients admitted to any internal medicine ward from August 15th 2015 to December 7th 2015. Unspecific ED diagnoses were defined through the clinical classification software for ICD-10 by two outcome assessors. As outcome parameters, we assessed in-hospital mortality and length of hospital stay.

Results

Six hundred eighty six consecutive patients were included. Unspecific diagnoses were identified in 100 (14.6%) of all consultations.

Patients receiving an unspecific diagnosis at ED discharge were significantly more often women (56.0% vs. 43.9%, p = 0.024), presented more often with a non-specific complaint (34% vs. 21%, p = 0.004), were less often demonstrating an abnormal heart rate (5.0% vs. 12.5%, p = 0.03), and less often on antibiotics (32.0% vs. 49.0%, p = 0.002). Apart from these, no studied drug intake, laboratory or clinical data including change in diagnosis was associated significantly with an unspecific diagnosis. Unspecific diagnoses were neither associated with in-hospital mortality in multivariable analysis (OR = 1.74, 95% CI: 0.60–5.04; p = 0.305) adjusted for relevant confounders nor with length of hospital stay (GMR = 0.87, 95% CI: 0.23–3.32; p = 0.840).

Conclusions

Women and patients with non-specific presenting complaints and no abnormal heart rate are at risk of receiving unspecific ED diagnoses that do not allow for targeted treatment, discharge and prognosis. This study did not find an effect of such diagnoses on length of hospital stay nor in-hospital mortality.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12873-022-00665-x.

Diagnostic Error in Pediatrics: A Narrative Review

A priority topic for patient safety research is diagnostic errors. However, despite the significant growth in awareness of their unacceptably high incidence and associated harm, a relative paucity of large, high-quality studies of diagnostic error in pediatrics exists. In this narrative review, we present what is known about the incidence and epidemiology of diagnostic error in pediatrics as well as the established research methods for identifying, evaluating, and reducing diagnostic errors, including their strengths and weaknesses. Additionally, we highlight that pediatric diagnostic error remains an area in need of both innovative research and quality improvement efforts to apply learnings from a rapidly growing evidence base. We propose several key research questions aimed at addressing persistent gaps in the pediatric diagnostic error literature that focus on the foundational knowledge needed to inform effective interventions to reduce the incidence of diagnostic errors and their associated harm. Additional research is needed to better establish the epidemiology of diagnostic error in pediatrics, including identifying high-risk clinical scenarios, patient populations, and groups of diagnoses. A critical need exists for validated measures of both diagnostic errors and diagnostic processes that can be adapted for different clinical settings and standardized for use across varying institutions. Pediatric researchers will need to work collaboratively on large-scale, high-quality studies to accomplish the ultimate goal of reducing diagnostic errors and their associated harm in children by addressing these fundamental gaps in knowledge.

Harnessing Event Report Data to Identify Diagnostic Error During the COVID-19 Pandemic

INTRODUCTION

COVID-19 exposed systemic gaps with increased potential for diagnostic error. This project implemented a new approach leveraging electronic safety reporting to identify and categorize diagnostic errors during the pandemic.

METHODS

All safety event reports from March 1, 2020, to February 28, 2021, at an academic medical center were evaluated using two complementary pathways (Pathway 1: all reports with explicit mention of COVID-19; Pathway 2: all reports without explicit mention of COVID-19 where natural language processing [NLP] plus logic-based stratification was applied to identify potential cases). Cases were evaluated by manual review to identify diagnostic error/delay and categorize error type using a recently proposed classification framework of eight categories of pandemic-related diagnostic errors.

RESULTS

A total of 14,230 reports were included, with 95 (0.7%) identified as cases of diagnostic error/delay. Pathway 1 (n = 1,780 eligible reports) yielded 45 reports with diagnostic error/delay (positive predictive value [PPV] = 2.5%), of which 35.6% (16/45) were attributed to pandemic-related strain. In Pathway 2, the NLP-based algorithm flagged 110 safety reports for manual review from 12,450 eligible reports. Of these, 50 reports had diagnostic error/delay (PPV = 45.5%); 94.0% (47/50) were related to strain. Errors from all eight categories of the taxonomy were found on analysis.

CONCLUSION

An event reporting-based strategy including use of simple-NLP-identified COVID-19-related diagnostic errors/delays uncovered several safety concerns related to COVID-19. An NLP-based approach can complement traditional reporting and be used as a just-in-time monitoring system to enable early detection of emerging risks from large volumes of safety reports.

Diagnostic Error: Why Now?

Diagnostic errors remain relatively understudied and underappreciated. They are particularly concerning in the intensive care unit, where they are more likely to result in harm to patients. There is a lack of consensus on the definition of diagnostic error, and current methods to quantify diagnostic error have numerous limitations as noted in the sentinel report by the National Academy of Medicine. Although definitive definition and measurement remain elusive goals, increasing our understanding of diagnostic error is crucial if we are to make progress in reducing the incidence and harm caused by errors in diagnosis.

Monitoring Diagnostic Safety Risks in Emergency Departments: Protocol for a Machine Learning Study

BACKGROUND

Diagnostic decision making, especially in emergency departments, is a highly complex cognitive process that involves uncertainty and susceptibility to errors. A combination of factors, including patient factors (eg, history, behaviors, complexity, and comorbidity), provider-care team factors (eg, cognitive load and information gathering and synthesis), and system factors (eg, health information technology, crowding, shift-based work, and interruptions) may contribute to diagnostic errors. Using electronic triggers to identify records of patients with certain patterns of care, such as escalation of care, has been useful to screen for diagnostic errors. Once errors are identified, sophisticated data analytics and machine learning techniques can be applied to existing electronic health record (EHR) data sets to shed light on potential risk factors influencing diagnostic decision making.

OBJECTIVE

This study aims to identify variables associated with diagnostic errors in emergency departments using large-scale EHR data and machine learning techniques.

METHODS

This study plans to use trigger algorithms within EHR data repositories to generate a large data set of records that are labeled trigger-positive or trigger-negative, depending on whether they meet certain criteria. Samples from both data sets will be validated using medical record reviews, upon which we expect to find a higher number of diagnostic safety events in the trigger-positive subset. Machine learning will be used to evaluate relationships between certain patient factors, provider-care team factors, and system-level risk factors and diagnostic safety signals in the statistically matched groups of trigger-positive and trigger-negative charts.

RESULTS

This federally funded study was approved by the institutional review board of 2 academic medical centers with affiliated community hospitals. Trigger queries are being developed at both organizations, and sample cohorts will be labeled using the triggers. Machine learning techniques such as association rule mining, chi-square automated interaction detection, and classification and regression trees will be used to discover important variables that could be incorporated within future clinical decision support systems to help identify and reduce risks that contribute to diagnostic errors.

CONCLUSIONS

The use of large EHR data sets and machine learning to investigate risk factors (related to the patient, provider-care team, and system-level) in the diagnostic process may help create future mechanisms for monitoring diagnostic safety.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

DERR1-10.2196/24642.

What Contributes to Diagnostic Error or Delay? A Qualitative Exploration Across Diverse Acute Care Settings in the United States

OBJECTIVES

Diagnostic error and delay is a prevalent and impactful problem. This study was part of a mixed-methods approach to understand the organizational, clinician, and patient factors contributing to diagnostic error and delay among acutely ill patients within a health system, as well as recommendations for the development of tailored, targeted, feasible, and effective interventions.

METHODS

We did a multisite qualitative study using focus group methodology to explore the perspectives of key clinician stakeholders. We used a conceptual framework that characterized diagnostic error and delay as occurring within 1 of 3 stages of the patient's diagnostic journey-critical information gathering, synthesis of key information, and decision making and communication. We developed our moderator guide based on the sociotechnical frameworks previously described by Holden and Singh for understanding noncognitive factors that lead to diagnostic error and delay. Deidentified focus group transcripts were coded in triplicate and to consensus over a series of meetings. A final coded data set was then uploaded into NVivo software. The data were then analyzed to generate overarching themes and categories.

RESULTS

We recruited a total of 64 participants across 4 sites from emergency departments, hospital floor, and intensive care unit settings into 11 focus groups. Clinicians perceive that diverse organizational, communication and coordination, individual clinician, and patient factors interact to impede the process of making timely and accurate diagnoses.

CONCLUSIONS

This study highlights the complex sociotechnical system within which individual clinicians operate and the contributions of systems, processes, and institutional factors to diagnostic error and delay.

Cognitive biases, environmental, patient and personal factors associated with critical care decision making: A scoping review

PURPOSE

Cognitive biases and factors affecting decision making in critical care can potentially lead to life-threatening errors. We aimed to examine the existing evidence on the influence of cognitive biases and factors on decision making in critical care.

MATERIALS AND METHODS

We conducted a scoping review by searching MEDLINE for articles from 2004 to November 2020. We included studies conducted in physicians that described cognitive biases or factors associated with decision making. During the study process we decided on the method to summarize the evidence, and based on the obtained studies a descriptive summary of findings was the best fit.

RESULTS

Thirty heterogenous studies were included. Four main biases or factors were observed, e.g. cognitive biases, personal factors, environmental factors, and patient factors. Six (20%) studies reported biases associated with decision making comprising omission-, status quo-, implicit-, explicit-, outcome-, and overconfidence bias. Nineteen (63%) studies described personal factors, twenty-two (73%) studies described environmental factors, and sixteen (53%) studies described patient factors.

CONCLUSIONS

The current evidence on cognitive biases and factors is heterogenous, but shows they influence clinical decision. Future studies should investigate the prevalence of cognitive biases and factors in clinical practice and their impact on clinical outcomes.

Increasing Physician Reporting of Diagnostic Learning Opportunities

BACKGROUND

An estimated 10% of Americans experience a diagnostic error annually, yet little is known about pediatric diagnostic errors. Physician reporting is a promising method for identifying diagnostic errors. However, our pediatric hospital medicine (PHM) division had only 1 diagnostic-related safety report in the preceding 4 years. We aimed to improve attending physician reporting of suspected diagnostic errors from 0 to 2 per 100 PHM patient admissions within 6 months.

METHODS

Our improvement team used the Model for Improvement, targeting the PHM service. To promote a safe reporting culture, we used the term diagnostic learning opportunity (DLO) rather than diagnostic error, defined as a "potential opportunity to make a better or more timely diagnosis." We developed an electronic reporting form and encouraged its use through reminders, scheduled reflection time, and monthly progress reports. The outcome measure, the number of DLO reports per 100 patient admissions, was tracked on an annotated control chart to assess the effect of our interventions over time. We evaluated DLOs using a formal 2-reviewer process.

RESULTS

Over the course of 13 weeks, there was an increase in the number of reports filed from 0 to 1.6 per 100 patient admissions, which met special cause variation, and was subsequently sustained. Most events (66%) were true diagnostic errors and were found to be multifactorial after formal review.

CONCLUSIONS

We used quality improvement methodology, focusing on psychological safety, to increase physician reporting of DLOs. This growing data set has generated nuanced learnings that will guide future improvement work.

Identifying trigger concepts to screen emergency department visits for diagnostic errors

OBJECTIVES

The diagnostic process is a vital component of safe and effective emergency department (ED) care. There are no standardized methods for identifying or reliably monitoring diagnostic errors in the ED, impeding efforts to enhance diagnostic safety. We sought to identify trigger concepts to screen ED records for diagnostic errors and describe how they can be used as a measurement strategy to identify and reduce preventable diagnostic harm.

METHODS

We conducted a literature review and surveyed ED directors to compile a list of potential electronic health record (EHR) trigger (e-triggers) and non-EHR based concepts. We convened a multidisciplinary expert panel to build consensus on trigger concepts to identify and reduce preventable diagnostic harm in the ED.

RESULTS

Six e-trigger and five non-EHR based concepts were selected by the expert panel. E-trigger concepts included: unscheduled ED return to ED resulting in hospital admission, death following ED visit, care escalation, high-risk conditions based on symptom-disease dyads, return visits with new diagnostic/therapeutic interventions, and change of treating service after admission. Non-EHR based signals included: cases from mortality/morbidity conferences, risk management/safety office referrals, ED medical director case referrals, patient complaints, and radiology/laboratory misreads and callbacks. The panel suggested further refinements to aid future research in defining diagnostic error epidemiology in ED settings.

CONCLUSIONS

We identified a set of e-trigger concepts and non-EHR based signals that could be developed further to screen ED visits for diagnostic safety events. With additional evaluation, trigger-based methods can be used as tools to monitor and improve ED diagnostic performance.

Handshake antimicrobial stewardship as a model to recognize and prevent diagnostic errors

OBJECTIVES

Few studies describe the impact of antimicrobial stewardship programs (ASPs) on recognizing and preventing diagnostic errors. Handshake stewardship (HS-ASP) is a novel ASP model that prospectively reviews hospital-wide antimicrobial usage with recommendations made in person to treatment teams. The purpose of this study was to determine if HS-ASP could identify and intervene on potential diagnostic errors for children hospitalized at a quaternary care children's hospital.

METHODS

Previously self-identified "Great Catch" (GC) interventions by the Children's Hospital Colorado HS-ASP team from 10/2014 through 5/2018 were retrospectively reviewed. Each GC was categorized based on the types of recommendations from HS-ASP, including if any diagnostic recommendations were made to the treatment team. Each GC was independently scored using the "Safer Dx Instrument" to determine presence of diagnostic error based on a previously determined cut-off score of ≤1.50. Interrater reliability for the instrument was measured using a randomized subset of one third of GCs.

RESULTS

During the study period, there were 162 GC interventions. Of these, 65 (40%) included diagnostic recommendations by HS-ASP and 19 (12%) had a Safer Dx Score of ≤1.50, (Κ=0.44; moderate agreement). Of those GCs associated with diagnostic errors, the HS-ASP team made a diagnostic recommendation to the primary treatment team 95% of the time.

CONCLUSIONS

Handshake stewardship has the potential to identify and intervene on diagnostic errors for hospitalized children.

Evolving from Morbidity and Mortality to a Case-based Error Reduction Conference: Evidence-based Best Practices from the Council of Emergency Medicine Residency Directors

Morbidity and mortality conferences are common among emergency medicine residency programs and are an important part of quality improvement initiatives. Here we review the key components of running an effective morbidity and mortality conference with a focus on goals and objectives, case identification and selection, session structure, and case presentation.

Operational measurement of diagnostic safety: state of the science

Reducing the incidence of diagnostic errors is increasingly a priority for government, professional, and philanthropic organizations. Several obstacles to measurement of diagnostic safety have hampered progress toward this goal. Although a coordinated national strategy to measure diagnostic safety remains an aspirational goal, recent research has yielded practical guidance for healthcare organizations to start using measurement to enhance diagnostic safety. This paper, concurrently published as an Issue Brief by the Agency for Healthcare Research and Quality, issues a "call to action" for healthcare organizations to begin measurement efforts using data sources currently available to them. Our aims are to outline the state of the science and provide practical recommendations for organizations to start identifying and learning from diagnostic errors. Whether by strategically leveraging current resources or building additional capacity for data gathering, nearly all organizations can begin their journeys to measure and reduce preventable diagnostic harm.

Rate of diagnostic errors and serious misdiagnosis-related harms for major vascular events, infections, and cancers: toward a national incidence estimate using the “Big Three”

Background

Missed vascular events, infections, and cancers account for ~75% of serious harms from diagnostic errors. Just 15 diseases from these “Big Three” categories account for nearly half of all serious misdiagnosis-related harms in malpractice claims. As part of a larger project estimating total US burden of serious misdiagnosis-related harms, we performed a focused literature review to measure diagnostic error and harm rates for these 15 conditions.

Methods

We searched PubMed, Google, and cited references. For errors, we selected high-quality, modern, US-based studies, if available, and best available evidence otherwise. For harms, we used literature-based estimates of the generic (disease-agnostic) rate of serious harms (morbidity/mortality) per diagnostic error and applied claims-based severity weights to construct disease-specific rates. Results were validated via expert review and comparison to prior literature that used different methods. We used Monte Carlo analysis to construct probabilistic plausible ranges (PPRs) around estimates.

Results

Rates for the 15 diseases were drawn from 28 published studies representing 91,755 patients. Diagnostic error (false negative) rates ranged from 2.2% (myocardial infarction) to 62.1% (spinal abscess), with a median of 13.6% [interquartile range (IQR) 9.2–24.7] and an aggregate mean of 9.7% (PPR 8.2–12.3). Serious misdiagnosis-related harm rates per incident disease case ranged from 1.2% (myocardial infarction) to 35.6% (spinal abscess), with a median of 5.5% (IQR 4.6–13.6) and an aggregate mean of 5.2% (PPR 4.5–6.7). Rates were considered face valid by domain experts and consistent with prior literature reports.

Conclusions

Diagnostic improvement initiatives should focus on dangerous conditions with higher diagnostic error and misdiagnosis-related harm rates.

Incident Reporting in Emergency Medicine: A Thematic Analysis of Events

BACKGROUND

Incident reporting is a recognized tool for healthcare quality improvement. These systems, which aim to capture near-misses and harm events, enable organizations to gather critical information about failure modes and design mitigation strategies. Although many hospitals have employed these systems, little is known about safety themes in emergency medicine incident reporting. Our objective was to systematically analyze and thematically code 1 year of incident reports.

METHODS

A mixed-methods analysis was performed on 1 year of safety reporting data from a large, urban tertiary-care emergency department using a modified grounded theory approach.

RESULTS

Between January 1 and December 31, 2015, there were 108,436 emergency department visits. During this time, 750 incident reports were filed. Twenty-nine themes were used to code the reports, with 744 codes applied. The most common themes were related to delays (138/750, 18.4%), medication safety (136/750, 18.1%), and failures in communication (110/750, 14.7%). A total of 48.8% (366/750) of reports were submitted by nurses.

CONCLUSIONS

The most prominent themes during 1 year of incident reports were related to medication safety, delays, and communication. Relative to hospital-wide reporting patterns, a higher proportion of reports were submitted by physicians. Despite this, overall incident reporting remains low, and more is needed to engage physicians in reporting.

Factors Associated With Potentially Missed Diagnosis of Appendicitis in the Emergency Department

IMPORTANCE

Appendicitis may be missed during initial emergency department (ED) presentation.

OBJECTIVE

To compare patients with a potentially missed diagnosis of appendicitis (ie, patients with symptoms associated with appendicitis, including abdominal pain, constipation, nausea and/or vomiting, fever, and diarrhea diagnosed within 1-30 days after initial ED presentation) with patients diagnosed with appendicitis on the same day of ED presentation to identify factors associated with potentially missed appendicitis.

DESIGN, SETTING, AND PARTICIPANTS

In this cohort study, a retrospective analysis of commercially insured claims data was conducted from January 1 to December 15, 2019. Patients who presented to the ED with undifferentiated symptoms associated with appendicitis between January 1, 2010, and December 31, 2017, were identified using the Clinformatics Data Mart administrative database (Optum Insights). The study sample comprised eligible adults (aged ≥18 years) and children (aged <18 years) who had previous ED visits within 30 days of an appendicitis diagnosis.

MAIN OUTCOMES AND MEASURES

Potentially missed diagnosis of appendicitis. Adjusted odds ratios (AORs) for abdominal pain and its combinations with other symptoms associated with appendicitis were compared between patients with a same-day diagnosis of appendicitis and patients with a potentially missed diagnosis of appendicitis.

RESULTS

Of 187 461 patients with a diagnosis of appendicitis, a total of 123 711 (66%; 101 375 adults [81.9%] and 22 336 children [18.1%]) were eligible for analysis. Among adults, 51 923 (51.2%) were women, with a mean (SD) age of 44.3 (18.2) years; among children, 9631 (43.1%) were girls, with a mean (SD) age of 12.2 (18.2) years. The frequency of potentially missed appendicitis was 6060 of 101 375 adults (6.0%) and 973 of 22 336 children (4.4%). Patients with isolated abdominal pain (adults, AOR, 0.65; 95% CI, 0.62-0.69; P < .001; children, AOR, 0.79; 95% CI, 0.69-0.90; P < .001) or with abdominal pain and nausea and/or vomiting (adults, AOR, 0.90; 95% CI, 0.84-0.97; P = .003; children, AOR, 0.84; 95% CI, 0.71-0.98; P = .03) were less likely to have missed appendicitis. Patients with abdominal pain and constipation (adults, AOR, 1.51; 95% CI, 1.31-1.75; P < .001; children, AOR, 2.43; 95% CI, 1.86-3.17; P < .001) were more likely to have missed appendicitis. Stratified by the presence of undifferentiated symptoms, women (abdominal pain, AOR, 1.68; 95% CI, 1.58-1.78; nausea and/or vomiting, AOR, 1.68; 95% CI, 1.52-1.85; fever, AOR, 1.32; 95% CI, 1.10-1.59; diarrhea, AOR, 1.19; 95% CI, 1.01-1.40; and constipation, AOR, 1.50; 95% CI, 1.24-1.82) and girls (abdominal pain, AOR, 1.64; 95% CI, 1.43-1.88; nausea and/or vomiting, AOR, 1.74; 95% CI, 1.42-2.13; fever, AOR, 1.55; 95% CI, 1.14-2.11; diarrhea, AOR, 1.80; 95% CI, 1.19-2.74; and constipation, AOR, 1.25; 95% CI, 0.88-1.78) as well as patients with a comorbidity index of 2 or greater (adults, abdominal pain, AOR, 3.33; 95% CI, 3.09-3.60; nausea and/or vomiting, AOR, 3.66; 95% CI, 3.23-4.14; fever, AOR, 5.00; 95% CI, 3.79-6.60; diarrhea, AOR, 4.27; 95% CI, 3.39-5.38; and constipation, AOR, 4.17; 95% CI, 3.08-5.65; children, abdominal pain, AOR, 2.42; 95% CI, 1.93-3.05; nausea and/or vomiting, AOR, 2.55; 95% CI, 1.89-3.45; fever, AOR, 4.12; 95% CI, 2.71-6.25; diarrhea, AOR, 2.17; 95% CI, 1.18-3.97; and constipation, AOR, 2.19; 95% CI, 1.30-3.70) were more likely to have missed appendicitis. Adult patients who received computed tomographic scans at the initial ED visit (abdominal pain, AOR, 0.58; 95% CI, 0.52-0.65; nausea and/or vomiting, AOR, 0.63; 95% CI, 0.52-0.75; fever, AOR, 0.41; 95% CI, 0.29-0.58; diarrhea, AOR, 0.83; 95% CI, 0.58-1.20; and constipation, AOR, 0.60; 95% CI, 0.39-0.94) were less likely to have missed appendicitis.

CONCLUSIONS AND RELEVANCE

Regardless of age, a missed diagnosis of appendicitis was more likely to occur in women, patients with comorbidities, and patients who experienced abdominal pain accompanied by constipation. Population-based estimates of the rates of potentially missed appendicitis reveal opportunities for improvement and identify factors that may mitigate the risk of a missed diagnosis.

Pediatric Clinician Comfort Discussing Diagnostic Errors for Improving Patient Safety: A Survey

INTRODUCTION

Meaningful conversations about diagnostic errors require safety cultures where clinicians are comfortable discussing errors openly. However, clinician comfort discussing diagnostic errors publicly and barriers to these discussions remain unexplored. We compared clinicians' comfort discussing diagnostic errors to other medical errors and identified barriers to open discussion.

METHODS

Pediatric clinicians at 4 hospitals were surveyed between May and June 2018. The survey assessed respondents' comfort discussing medical errors (with varying degrees of system versus individual clinician responsibility) during morbidity and mortality conferences and privately with peers. Respondents reported the most significant barriers to discussing diagnostic errors publicly. Poststratification weighting accounted for nonresponse bias; the Benjamini-Hochberg adjustment was applied to control for false discovery (significance set at P < 0.018).

RESULTS

Clinicians (n = 838; response rate 22.6%) were significantly less comfortable discussing all error types during morbidity and mortality conferences than privately (P < 0.004) and significantly less comfortable discussing diagnostic errors compared with other medical errors (P < 0.018). Comfort did not differ by clinician type or years in practice; clinicians at one institution were significantly less comfortable discussing diagnostic errors compared with peers at other institutions. The most frequently cited barriers to discussing diagnostic errors publicly included feeling like a bad clinician, loss of reputation, and peer judgment of knowledge base and decision-making.

CONCLUSIONS

Clinicians are more uncomfortable discussing diagnostic errors than other types of medical errors. The most frequent barriers involve the public perception of clinical performance. Addressing this aspect of safety culture may improve clinician participation in efforts to reduce harm from diagnostic errors.

Diagnostic error in the emergency department: learning from national patient safety incident report analysis

BACKGROUND

Diagnostic error occurs more frequently in the emergency department than in regular in-patient hospital care. We sought to characterise the nature of reported diagnostic error in hospital emergency departments in England and Wales from 2013 to 2015 and to identify the priority areas for intervention to reduce their occurrence.

METHODS

A cross-sectional mixed-methods design using an exploratory descriptive analysis and thematic analysis of patient safety incident reports. Primary data were extracted from a national database of patient safety incidents. Reports were filtered for emergency department settings, diagnostic error (as classified by the reporter), from 2013 to 2015. These were analysed for the chain of events, contributory factors and harm outcomes.

RESULTS

There were 2288 cases of confirmed diagnostic error: 1973 (86%) delayed and 315 (14%) wrong diagnoses. One in seven incidents were reported to have severe harm or death. Fractures were the most common condition (44%), with cervical-spine and neck of femur the most frequent types. Other common conditions included myocardial infarctions (7%) and intracranial bleeds (6%). Incidents involving both delayed and wrong diagnoses were associated with insufficient assessment, misinterpretation of diagnostic investigations and failure to order investigations. Contributory factors were predominantly human factors, including staff mistakes, healthcare professionals' inadequate skillset or knowledge and not following protocols.

CONCLUSIONS

Systems modifications are needed that provide clinicians with better support in performing patient assessment and investigation interpretation. Interventions to reduce diagnostic error need to be evaluated in the emergency department setting, and could include standardised checklists, structured reporting and technological investigation improvements.

Improving Team-Based Decision Making Using Data Analytics and Informatics: Protocol for a Collaborative Decision Support Design

Background

According to the September 2015 Institute of Medicine report, Improving Diagnosis in Health Care, each of us is likely to experience one diagnostic error in our lifetime, often with devastating consequences. Traditionally, diagnostic decision making has been the sole responsibility of an individual clinician. However, diagnosis involves an interaction among interprofessional team members with different training, skills, cultures, knowledge, and backgrounds. Moreover, diagnostic error is prevalent in the interruption-prone environment, such as the emergency department, where the loss of information may hinder a correct diagnosis.

Objective

The overall purpose of this protocol is to improve team-based diagnostic decision making by focusing on data analytics and informatics tools that improve collective information management.

Methods

To achieve this goal, we will identify the factors contributing to failures in team-based diagnostic decision making (aim 1), understand the barriers of using current health information technology tools for team collaboration (aim 2), and develop and evaluate a collaborative decision-making prototype that can improve team-based diagnostic decision making (aim 3).

Results

Between 2019 to 2020, we are collecting data for this study. The results are anticipated to be published between 2020 and 2021.

Conclusions

The results from this study can shed light on improving diagnostic decision making by incorporating diagnostics rationale from team members. We believe a positive direction to move forward in solving diagnostic errors is by incorporating all team members, and using informatics.

International Registered Report Identifier (IRRID)

DERR1-10.2196/16047

Cognitive bias in clinical medicine

Cognitive bias is increasingly recognised as an important source of medical error, and is both ubiquitous across clinical practice yet incompletely understood. This increasing awareness of bias has resulted in a surge in clinical and psychological research in the area and development of various 'debiasing strategies'. This paper describes the potential origins of bias based on 'dual process thinking', discusses and illustrates a number of the important biases that occur in clinical practice, and considers potential strategies that might be used to mitigate their effect.

Serious misdiagnosis-related harms in malpractice claims: The “Big Three” – vascular events, infections, and cancers

Background

Diagnostic errors cause substantial preventable harm, but national estimates vary widely from 40,000 to 4 million annually. This cross-sectional analysis of a large medical malpractice claims database was the first phase of a three-phase project to estimate the US burden of serious misdiagnosis-related harms.

Methods

We sought to identify diseases accounting for the majority of serious misdiagnosis-related harms (morbidity/mortality). Diagnostic error cases were identified from Controlled Risk Insurance Company (CRICO)’s Comparative Benchmarking System (CBS) database (2006–2015), representing 28.7% of all US malpractice claims. Diseases were grouped according to the Agency for Healthcare Research and Quality (AHRQ) Clinical Classifications Software (CCS) that aggregates the International Classification of Diseases diagnostic codes into clinically sensible groupings. We analyzed vascular events, infections, and cancers (the “Big Three”), including frequency, severity, and settings. High-severity (serious) harms were defined by scores of 6–9 (serious, permanent disability, or death) on the National Association of Insurance Commissioners (NAIC) Severity of Injury Scale.

Results

From 55,377 closed claims, we analyzed 11,592 diagnostic error cases [median age 49, interquartile range (IQR) 36–60; 51.7% female]. These included 7379 with high-severity harms (53.0% death). The Big Three diseases accounted for 74.1% of high-severity cases (vascular events 22.8%, infections 13.5%, and cancers 37.8%). In aggregate, the top five from each category (n = 15 diseases) accounted for 47.1% of high-severity cases. The most frequent disease in each category, respectively, was stroke, sepsis, and lung cancer. Causes were disproportionately clinical judgment factors (85.7%) across categories (range 82.0–88.8%).

Conclusions

The Big Three diseases account for about three-fourths of serious misdiagnosis-related harms. Initial efforts to improve diagnosis should focus on vascular events, infections, and cancers.

Teaching Critical Thinking: A Case for Instruction in Cognitive Biases to Reduce Diagnostic Errors and Improve Patient Safety

Diagnostic errors contribute to as many as 70% of medical errors. Prevention of diagnostic errors is more complex than building safety checks into health care systems; it requires an understanding of critical thinking, of clinical reasoning, and of the cognitive processes through which diagnoses are made. When a diagnostic error is recognized, it is imperative to identify where and how the mistake in clinical reasoning occurred. Cognitive biases may contribute to errors in clinical reasoning. By understanding how physicians make clinical decisions, and examining how errors due to cognitive biases occur, cognitive bias awareness training and debiasing strategies may be developed to decrease diagnostic errors and patient harm. Studies of the impact of teaching critical thinking skills have mixed results but are limited by methodological problems.This Perspective explores the role of clinical reasoning and cognitive bias in diagnostic error, as well as the effect of instruction in metacognitive skills on improvement of diagnostic accuracy for both learners and practitioners. Recent literature questioning whether teaching critical thinking skills increases diagnostic accuracy is critically examined, as are studies suggesting that metacognitive practices result in better patient care and outcomes. Instruction in metacognition, reflective practice, and cognitive bias awareness may help learners move toward adaptive expertise and help clinicians improve diagnostic accuracy. The authors argue that explicit instruction in metacognition in medical education, including awareness of cognitive biases, has the potential to reduce diagnostic errors and thus improve patient safety.

A cognitive forcing tool to mitigate cognitive bias - a randomised control trial

BACKGROUND

Cognitive bias is an important source of diagnostic error yet is a challenging area to understand and teach. Our aim was to determine whether a cognitive forcing tool can reduce the rates of error in clinical decision making. A secondary objective was to understand the process by which this effect might occur.

METHODS

We hypothesised that using a cognitive forcing tool would reduce diagnostic error rates. To test this hypothesis, a novel online case-based approach was used to conduct a single blinded randomized clinical trial conducted from January 2017 to September 2018. In addition, a qualitative series of "think aloud" interviews were conducted with 20 doctors from a UK teaching hospital in 2018. The primary outcome was the diagnostic error rate when solving bias inducing clinical vignettes. A volunteer sample of medical professionals from across the UK, Republic of Ireland and North America. They ranged in seniority from medical student to Attending Physician.

RESULTS

Seventy six participants were included in the study. The data showed doctors of all grades routinely made errors related to cognitive bias. There was no difference in error rates between groups (mean 2.8 cases correct in intervention vs 3.1 in control group, 95% CI -0.94 - 0.45 P = 0.49). The qualitative protocol revealed that the cognitive forcing strategy was well received and a produced a subjectively positive impact on doctors' accuracy and thoughtfulness in clinical cases.

CONCLUSIONS

The quantitative data failed to show an improvement in accuracy despite a positive qualitative experience. There is insufficient evidence to recommend this tool in clinical practice, however the qualitative data suggests such an approach has some merit and face validity to users.

Diagnostic Decision-Making in the Emergency Department

Emergency medicine requires diagnosing unfamiliar patients with undifferentiated acute presentations. This requires hypothesis generation and questioning, examination, and testing. Balancing patient load, care across the severity spectrum, and frequent interruptions create time pressures that predispose humans to fast thinking or cognitive shortcuts, including cognitive biases. Diagnostic error is the failure to establish an accurate and timely explanation of the problem or communicate that to the patient, often contributing to physical, emotional, or financial harm. Methods for monitoring diagnostic error in the emergency department are needed to establish frequency and serve as a foundation for future interventions.

How common are cognitive errors in cases presented at emergency medicine resident morbidity and mortality conferences?

Background

Cognitive errors are a major contributor to medical error. Traditionally, medical errors at teaching hospitals are analyzed in morbidity and mortality (M&M) conferences. We aimed to describe the frequency of cognitive errors in relation to the occurrence of diagnostic and other error types, in cases presented at an emergency medicine (EM) resident M&M conference.

Methods

We conducted a retrospective study of all cases presented at a suburban US EM residency monthly M&M conference from September 2011 to August 2016. Each case was reviewed using the electronic medical record (EMR) and notes from the M&M case by two EM physicians. Each case was categorized by type of primary medical error that occurred as described by Okafor et al. When a diagnostic error occurred, the case was reviewed for contributing cognitive and non-cognitive factors. Finally, when a cognitive error occurred, the case was classified into faulty knowledge, faulty data gathering or faulty synthesis, as described by Graber et al. Disagreements in error type were mediated by a third EM physician.

Results

A total of 87 M&M cases were reviewed; the two reviewers agreed on 73 cases, and 14 cases required mediation by a third reviewer. Forty-eight cases involved diagnostic errors, 47 of which were cognitive errors. Of these 47 cases, 38 involved faulty synthesis, 22 involved faulty data gathering and only 11 involved faulty knowledge. Twenty cases contained more than one type of cognitive error. Twenty-nine cases involved both a resident and an attending physician, while 17 cases involved only an attending physician. Twenty-one percent of the resident cases involved all three cognitive errors, while none of the attending cases involved all three. Forty-one percent of the resident cases and only 6% of the attending cases involved faulty knowledge. One hundred percent of the resident cases and 94% of the attending cases involved faulty synthesis.

Conclusions

Our review of 87 EM M&M cases revealed that cognitive errors are commonly involved in cases presented, and that these errors are less likely due to deficient knowledge and more likely due to faulty synthesis. M&M conferences may therefore provide an excellent forum to discuss cognitive errors and how to reduce their occurrence.

Tracking Progress in Improving Diagnosis: A Framework for Defining Undesirable Diagnostic Events

Diagnostic error is a prevalent, harmful, and costly phenomenon. Multiple national health care and governmental organizations have recently identified the need to improve diagnostic safety as a high priority. A major barrier, however, is the lack of standardized, reliable methods for measuring diagnostic safety. Given the absence of reliable and valid measures for diagnostic errors, we need methods to help establish some type of baseline diagnostic performance across health systems, as well as to enable researchers and health systems to determine the impact of interventions for improving the diagnostic process. Multiple approaches have been suggested but none widely adopted. We propose a new framework for identifying "undesirable diagnostic events" (UDEs) that health systems, professional organizations, and researchers could further define and develop to enable standardized measurement and reporting related to diagnostic safety. We propose an outline for UDEs that identifies both conditions prone to diagnostic error and the contexts of care in which these errors are likely to occur. Refinement and adoption of this framework across health systems can facilitate standardized measurement and reporting of diagnostic safety.

The global burden of diagnostic errors in primary care

Diagnosis is one of the most important tasks performed by primary care physicians. The World Health Organization (WHO) recently prioritized patient safety areas in primary care, and included diagnostic errors as a high-priority problem. In addition, a recent report from the Institute of Medicine in the USA, 'Improving Diagnosis in Health Care', concluded that most people will likely experience a diagnostic error in their lifetime. In this narrative review, we discuss the global significance, burden and contributory factors related to diagnostic errors in primary care. We synthesize available literature to discuss the types of presenting symptoms and conditions most commonly affected. We then summarize interventions based on available data and suggest next steps to reduce the global burden of diagnostic errors. Research suggests that we are unlikely to find a 'magic bullet' and confirms the need for a multifaceted approach to understand and address the many systems and cognitive issues involved in diagnostic error. Because errors involve many common conditions and are prevalent across all countries, the WHO's leadership at a global level will be instrumental to address the problem. Based on our review, we recommend that the WHO consider bringing together primary care leaders, practicing frontline clinicians, safety experts, policymakers, the health IT community, medical education and accreditation organizations, researchers from multiple disciplines, patient advocates, and funding bodies among others, to address the many common challenges and opportunities to reduce diagnostic error. This could lead to prioritization of practice changes needed to improve primary care as well as setting research priorities for intervention development to reduce diagnostic error.

Voluntary Medical Incident Reporting Tool to Improve Physician Reporting of Medical Errors in an Emergency Department

Introduction

Medical errors are frequently under-reported, yet their appropriate analysis, coupled with remediation, is essential for continuous quality improvement. The emergency department (ED) is recognized as a complex and chaotic environment prone to errors. In this paper, we describe the design and implementation of a web-based ED-specific incident reporting system using an iterative process.

Methods

A web-based, password-protected tool was developed by members of a quality assurance committee for ED providers to report incidents that they believe could impact patient safety.

Results

The utilization of this system in one residency program with two academic sites resulted in an increase from 81 reported incidents in 2009, the first year of use, to 561 reported incidents in 2012. This is an increase in rate of reported events from 0.07% of all ED visits to 0.44% of all ED visits. In 2012, faculty reported 60% of all incidents, while residents and midlevel providers reported 24% and 16% respectively. The most commonly reported incidents were delays in care and management concerns.

Conclusion

Error reporting frequency can be dramatically improved by using a web-based, user-friendly, voluntary, and non-punitive reporting system.