Accuracy of the Safer Dx Instrument to Identify Diagnostic Errors in Primary Care

Assessing the Revised Safer Dx Instrument® in the understanding of ambulatory system design changes for type 1 diabetes and autism spectrum disorder in pediatrics

OBJECTIVES

We sought within an ambulatory safety study to understand if the Revised Safer Dx instrument may be helpful in identification of diagnostic missed opportunities in care of children with type 1 diabetes (T1D) and autism spectrum disorder (ASD).

METHODS

We reviewed two months of emergency department (ED) encounters for all patients at our tertiary care site with T1D and a sample of such encounters for patients with ASD over a 15-month period, and their pre-visit communication methods to better understand opportunities to improve diagnosis. We applied the Revised Safer Dx instrument to each diagnostic journey. We chose potentially preventable ED visits for hyperglycemia, diabetic ketoacidosis, and behavioral crises, and reviewed electronic health record data over the prior three months related to the illness that resulted in the ED visit.

RESULTS

We identified 63 T1D and 27 ASD ED visits. Using the Revised Safer Dx instrument, we did not identify any potentially missed opportunities to improve diagnosis in T1D. We found two potential missed opportunities (Safer Dx overall score of 5) in ASD, related to potential for ambulatory medical management to be improved. Over this period, 40 % of T1D and 52 % of ASD patients used communication prior to the ED visit.

CONCLUSIONS

Using the Revised Safer Dx instrument, we uncommonly identified missed opportunities to improve diagnosis in patients who presented to the ED with potentially preventable complications of their chronic diseases. Future researchers should consider prospectively collected data as well as development or adaptation of tools like the Safer Dx.

Development and Implementation of a Digital Quality Measure of Emergency Cancer Diagnosis

PURPOSE

Missed and delayed cancer diagnoses are common, harmful, and often preventable. Automated measures of quality of cancer diagnosis are lacking but could identify gaps and guide interventions. We developed and implemented a digital quality measure (dQM) of cancer emergency presentation (EP) using electronic health record databases of two health systems and characterized the measure's association with missed opportunities for diagnosis (MODs) and mortality.

METHODS

On the basis of literature and expert input, we defined EP as a new cancer diagnosis within 30 days after emergency department or inpatient visit. We identified EPs for lung cancer and colorectal cancer (CRC) in the Department of Veterans Affairs (VA) and Geisinger from 2016 to 2020. We validated measure accuracy and identified preceding MODs through standardized chart review of 100 records per cancer per health system. Using VA's longitudinal encounter and mortality data, we applied logistic regression to assess EP's association with 1-year mortality, adjusting for cancer stage and demographics.

RESULTS

Among 38,565 and 2,914 patients with lung cancer and 14,674 and 1,649 patients with CRCs at VA and Geisinger, respectively, our dQM identified EPs in 20.9% and 9.4% of lung cancers, and 22.4% and 7.5% of CRCs. Chart reviews revealed high positive predictive values for EPs across sites and cancer types (72%-90%), and a substantial percent represented MODs (48.8%-84.9%). EP was associated with significantly higher odds of 1-year mortality for lung cancer and CRC (adjusted odds ratio, 1.78 and 1.83, respectively, 95% CI, 1.63 to 1.86 and 1.61 to 2.07).

CONCLUSION

A dQM for cancer EP was strongly associated with both mortality and MODs. The findings suggest a promising automated approach to measuring quality of cancer diagnosis in US health systems.

Diagnostic errors and characteristics of patients seen at a general internal medicine outpatient clinic with a referral for diagnosis

OBJECTIVES

Patients referred to general internal medicine (GIM) outpatient clinics may face a higher risk of diagnostic errors than non-referred patients. This difference in risk is assumed to be due to the differences in diseases and clinical presentations between referred and non-referred patients; however, clinical data regarding this issue are scarce. This study aimed to determine the frequency of diagnostic errors and compare the characteristics of referred and non-referred patients visit GIM outpatient clinics.

METHODS

This study included consecutive outpatients who visited the GIM outpatient clinic at a university hospital, with or without referral. Data on age, sex, chief complaints, referral origin, and final diagnosis were collected from medical records. The Revised Safer Dx Instrument was used to detect diagnostic errors.

RESULTS

Data from 534 referred and 599 non-referred patients were analyzed. The diagnostic error rate was higher in the referral group than that in the non-referral group (2.2 % vs. 0.5 %, p=0.01). The prevalence of abnormal test results and sensory disturbances was higher in the chief complaints, and the prevalence of musculoskeletal system disorders, connective tissue diseases, and neoplasms was higher in the final diagnoses of referred patients compared with non-referred patients. Among referred patients with diagnostic errors, abnormal test results and sensory disturbances were the two most common chief complaints, whereas neoplasia was the most common final diagnosis. Problems with data integration and interpretation were found to be the most common factors contributing to diagnostic errors.

CONCLUSIONS

Paying more attention to patients with abnormal test results and sensory disturbances and considering a higher pre-test probability for neoplasms may prevent diagnostic errors in patients referred to GIM outpatient clinics.

Evaluation of a Structured Review Process for Emergency Department Return Visits with Admission

BACKGROUND

Review of emergency department (ED) revisits with admission allows the identification of improvement opportunities. Applying a health equity lens to revisits may highlight potential disparities in care transitions. Universal definitions or practicable frameworks for these assessments are lacking. The authors aimed to develop a structured methodology for this quality assurance (QA) process, with a layered equity analysis.

METHODS

The authors developed a classification instrument to identify potentially preventable 72-hour returns with admission (PPRA-72), accounting for directed, unrelated, unanticipated, or disease progression returns. A second review team assessed the instrument reliability. A self-reported race/ethnicity (R/E) and language algorithm was developed to minimize uncategorizable data. Disposition distribution, return rates, and PPRA-72 classifications were analyzed for disparities using Pearson chi-square and Fisher's exact tests.

RESULTS

The PPRA-72 rate was 4.8% for 2022 ED return visits requiring admission. Review teams achieved 93% agreement (κ = 0.51) for the binary determination of PPRA-72 vs. nonpreventable returns. There were significant differences between R/E and language in ED dispositions (p < 0.001), with more frequent admissions for the R/E White at the index visit and Other at the 72-hour return visit. Rates of return visits within 72 hours differed significantly by R/E (p < 0.001) but not by language (p = 0.156), with the R/E Black most frequent to have a 72-hour return. There were no differences between R/E (p = 0.446) or language (p = 0.248) in PPRA-72 rates. The initiative led to system improvements through informatics optimizations, triage protocols, provider feedback, and education.

CONCLUSION

The authors developed a review methodology for identifying improvement opportunities across ED 72-hour returns. This QA process enabled the identification of areas of disparity, with the continuous aim to develop next steps in ensuring health equity in care transitions.

The Safety of Outpatient Health Care : Review of Electronic Health Records

BACKGROUND

Despite considerable emphasis on delivering safe care, substantial patient harm occurs. Although most care occurs in the outpatient setting, knowledge of outpatient adverse events (AEs) remains limited.

OBJECTIVE

To measure AEs in the outpatient setting.

DESIGN

Retrospective review of the electronic health record (EHR).

SETTING

11 outpatient sites in Massachusetts in 2018.

PATIENTS

3103 patients who received outpatient care.

MEASUREMENTS

Using a trigger method, nurse reviewers identified possible AEs and physicians adjudicated them, ranked severity, and assessed preventability. Generalized estimating equations were used to assess the association of having at least 1 AE with age, sex, race, and primary insurance. Variation in AE rates was analyzed across sites.

RESULTS

The 3103 patients (mean age, 52 years) were more often female (59.8%), White (75.1%), English speakers (90.8%), and privately insured (70.4%) and had a mean of 4 outpatient encounters in 2018. Overall, 7.0% (95% CI, 4.6% to 9.3%) of patients had at least 1 AE (8.6 events per 100 patients annually). Adverse drug events were the most common AE (63.8%), followed by health care-associated infections (14.8%) and surgical or procedural events (14.2%). Severity was serious in 17.4% of AEs, life-threatening in 2.1%, and never fatal. Overall, 23.2% of AEs were preventable. Having at least 1 AE was less often associated with ages 18 to 44 years than with ages 65 to 84 years (standardized risk difference, -0.05 [CI, -0.09 to -0.02]) and more often associated with Black race than with Asian race (standardized risk difference, 0.09 [CI, 0.01 to 0.17]). Across study sites, 1.8% to 23.6% of patients had at least 1 AE and clinical category of AEs varied substantially.

LIMITATION

Retrospective EHR review may miss AEs.

CONCLUSION

Outpatient harm was relatively common and often serious. Adverse drug events were most frequent. Rates were higher among older adults. Interventions to curtail outpatient harm are urgently needed.

PRIMARY FUNDING SOURCE

Controlled Risk Insurance Company and the Risk Management Foundation of the Harvard Medical Institutions.

Effect of contextual factors on the prevalence of diagnostic errors among patients managed by physicians of the same specialty: a single-centre retrospective observational study

BACKGROUND

There has been growing recognition that contextual factors influence the physician's cognitive processes. However, given that cognitive processes may depend on the physicians' specialties, the effects of contextual factors on diagnostic errors reported in previous studies could be confounded by difference in physicians.

OBJECTIVE

This study aimed to clarify whether contextual factors such as location and consultation type affect diagnostic accuracy.

METHODS

We reviewed the medical records of 1992 consecutive outpatients consulted by physicians from the Department of Diagnostic and Generalist Medicine in a university hospital between 1 January and 31 December 2019. Diagnostic processes were assessed using the Revised Safer Dx Instrument. Patients were categorised into three groups according to contextual factors (location and consultation type): (1) referred patients with scheduled visit to the outpatient department; (2) patients with urgent visit to the outpatient department; and (3) patients with emergency visit to the emergency room. The effect of the contextual factors on the prevalence of diagnostic errors was investigated using logistic regression analysis.

RESULTS

Diagnostic errors were observed in 12 of 534 referred patients with scheduled visit to the outpatient department (2.2%), 3 of 599 patients with urgent visit to the outpatient department (0.5%) and 13 of 859 patients with emergency visit to the emergency room (1.5%). Multivariable logistic regression analysis showed a significantly higher prevalence of diagnostic errors in referred patients with scheduled visit to the outpatient department than in patients with urgent visit to the outpatient department (OR 4.08, p=0.03), but no difference between patients with emergency and urgent visit to the emergency room and outpatient department, respectively.

CONCLUSION

Contextual factors such as consultation type may affect diagnostic errors; however, since the differences in the prevalence of diagnostic errors were small, the effect of contextual factors on diagnostic accuracy may be small in physicians working in different care settings.

An electronic trigger to detect telemedicine-related diagnostic errors

INTRODUCTION

The COVID-19 pandemic advanced the use of telehealth-facilitated care. However, little is known about how to measure safety of clinical diagnosis made through telehealth-facilitated primary care.

METHODS

We used the seven-step Safer Dx Trigger Tool framework to develop an electronic trigger (e-trigger) tool to identify potential missed opportunities for more timely diagnosis during primary care telehealth visits at a large Department of Veterans Affairs facility. We then applied the e-trigger algorithm to electronic health record data related to primary care visits during a 1-year period (1 April 2020-31 March 2021). The algorithm identified patients with unexpected visits within 10 days of an index telemedicine visit and classified such records as e-trigger positive. We then validated the e-trigger's ability to detect missed opportunities in diagnosis using chart reviews based on a structured data collection instrument (the Revised Safer Dx instrument).

RESULTS

We identified 128,761 telehealth visits (32,459 unique patients), of which 434 visits led to subsequent unplanned emergency department (ED), hospital, or primary care visits within 10 days of the index visit. Of these, 116 were excluded for clinical reasons (trauma, injury, or childbirth), leaving 318 visits (240 unique patients) needing further evaluation. From these, 100 records were randomly selected for review, of which four were falsely flagged due to invalid data (visits by non-providers or those incorrectly flagged as completed telehealth visits). Eleven patients had a missed opportunity in diagnosis, yielding a positive predictive value of 11%.

DISCUSSION

Electronic triggers that identify missed opportunities for additional evaluation could help advance the understanding of safety of clinical diagnosis made in telehealth-enabled care. Better measurement can help determine which patients can safely be cared for via telemedicine versus traditional in-person visits.

Japanese Internists' Most Memorable Diagnostic Error Cases: A Self-reflection Survey

Objective The etiologies of diagnostic errors among internal medicine physicians are unclear. To understand the causes and characteristics of diagnostic errors through reflection by those involved in them. Methods We conducted a cross-sectional study using a web-based questionnaire in Japan in January 2019. Over a 10-day period, a total of 2,220 participants agreed to participate in the study, of whom 687 internists were included in the final analysis. Participants were asked about their most memorable diagnostic error cases, in which the time course, situational factors, and psychosocial context could be most vividly recalled and where the participant provided care. We categorized diagnostic errors and identified contributing factors (i.e., situational factors, data collection/interpretation factors, and cognitive biases). Results Two-thirds of the identified diagnostic errors occurred in the clinic or emergency department. Errors were most frequently categorized as wrong diagnoses, followed by delayed and missed diagnoses. Errors most often involved diagnoses related to malignancy, circulatory system disorders, or infectious diseases. Situational factors were the most cited error cause, followed by data collection factors and cognitive bias. Common situational factors included limited consultation during office hours and weekends and barriers that prevented consultation with a supervisor or another department. Conclusion Internists reported situational factors as a significant cause of diagnostic errors. Other factors, such as cognitive biases, were also evident, although the difference in clinical settings may have influenced the proportions of the etiologies of the errors that were observed. Furthermore, wrong, delayed, and missed diagnoses may have distinctive associated cognitive biases.

Prevalence and Characteristics of Diagnostic Error in Pediatric Critical Care: A Multicenter Study

OBJECTIVES

Effective interventions to prevent diagnostic error among critically ill children should be informed by diagnostic error prevalence and etiologies. We aimed to determine the prevalence and characteristics of diagnostic errors and identify factors associated with error in patients admitted to the PICU.

DESIGN

Multicenter retrospective cohort study using structured medical record review by trained clinicians using the Revised Safer Dx instrument to identify diagnostic error (defined as missed opportunities in diagnosis). Cases with potential errors were further reviewed by four pediatric intensivists who made final consensus determinations of diagnostic error occurrence. Demographic, clinical, clinician, and encounter data were also collected.

SETTING

Four academic tertiary-referral PICUs.

PATIENTS

Eight hundred eighty-two randomly selected patients 0-18 years old who were nonelectively admitted to participating PICUs.

INTERVENTIONS

None.

MEASUREMENTS AND MAIN RESULTS

Of 882 patient admissions, 13 (1.5%) had a diagnostic error up to 7 days after PICU admission. Infections (46%) and respiratory conditions (23%) were the most common missed diagnoses. One diagnostic error caused harm with a prolonged hospital stay. Common missed diagnostic opportunities included failure to consider the diagnosis despite a suggestive history (69%) and failure to broaden diagnostic testing (69%). Unadjusted analysis identified more diagnostic errors in patients with atypical presentations (23.1% vs 3.6%, p = 0.011), neurologic chief complaints (46.2% vs 18.8%, p = 0.024), admitting intensivists greater than or equal to 45 years old (92.3% vs 65.1%, p = 0.042), admitting intensivists with more service weeks/year (mean 12.8 vs 10.9 wk, p = 0.031), and diagnostic uncertainty on admission (77% vs 25.1%, p < 0.001). Generalized linear mixed models determined that atypical presentation (odds ratio [OR] 4.58; 95% CI, 0.94-17.1) and diagnostic uncertainty on admission (OR 9.67; 95% CI, 2.86-44.0) were significantly associated with diagnostic error.

CONCLUSIONS

Among critically ill children, 1.5% had a diagnostic error up to 7 days after PICU admission. Diagnostic errors were associated with atypical presentations and diagnostic uncertainty on admission, suggesting possible targets for intervention.

Opportunities for Diagnostic Improvement Among Pediatric Hospital Readmissions

OBJECTIVES

Diagnostic errors, termed "missed opportunities for improving diagnosis" (MOIDs), are known sources of harm in children but have not been well characterized in pediatric hospital medicine. Our objectives were to systematically identify and describe MOIDs among general pediatric patients who experienced hospital readmission, outline improvement opportunities, and explore factors associated with increased risk of MOID.

PATIENTS AND METHODS

Our retrospective cohort study included unplanned readmissions within 15 days of discharge from a freestanding children's hospital (October 2018-September 2020). Health records from index admissions and readmissions were independently reviewed and discussed by practicing inpatient physicians to identify MOIDs using an established instrument, SaferDx. MOIDs were evaluated using a diagnostic-specific tool to identify improvement opportunities within the diagnostic process.

RESULTS

MOIDs were identified in 22 (6.3%) of 348 readmissions. Opportunities for improvement included: delay in considering the correct diagnosis (n = 11, 50%) and failure to order needed test(s) (n = 10, 45%). Patients with MOIDs were older (median age: 3.8 [interquartile range 1.5-11.2] vs 1.0 [0.3-4.9] years) than patients without MOIDs but similar in sex, primary language, race, ethnicity, and insurance type. We did not identify conditions associated with higher risk of MOID. Lower respiratory tract infections accounted for 26% of admission diagnoses but only 1 (4.5%) case of MOID.

CONCLUSIONS

Standardized review of pediatric readmissions identified MOIDs and opportunities for improvement within the diagnostic process, particularly in clinician decision-making. We identified conditions with low incidence of MOID. Further work is needed to better understand pediatric populations at highest risk for MOID.

A structured approach to EHR surveillance of diagnostic error in acute care: an exploratory analysis of two institutionally-defined case cohorts

OBJECTIVES

To test a structured electronic health record (EHR) case review process to identify diagnostic errors (DE) and diagnostic process failures (DPFs) in acute care.

METHODS

We adapted validated tools (Safer Dx, Diagnostic Error Evaluation Research [DEER] Taxonomy) to assess the diagnostic process during the hospital encounter and categorized 13 postulated e-triggers. We created two test cohorts of all preventable cases (n=28) and an equal number of randomly sampled non-preventable cases (n=28) from 365 adult general medicine patients who expired and underwent our institution's mortality case review process. After excluding patients with a length of stay of more than one month, each case was reviewed by two blinded clinicians trained in our process and by an expert panel. Inter-rater reliability was assessed. We compared the frequency of DE contributing to death in both cohorts, as well as mean DPFs and e-triggers for DE positive and negative cases within each cohort.

RESULTS

Twenty-seven (96.4%) preventable and 24 (85.7%) non-preventable cases underwent our review process. Inter-rater reliability was moderate between individual reviewers (Cohen's kappa 0.41) and substantial with the expert panel (Cohen's kappa 0.74). The frequency of DE contributing to death was significantly higher for the preventable compared to the non-preventable cohort (56% vs. 17%, OR 6.25 [1.68, 23.27], p<0.01). Mean DPFs and e-triggers were significantly and non-significantly higher for DE positive compared to DE negative cases in each cohort, respectively.

CONCLUSIONS

We observed substantial agreement among final consensus and expert panel reviews using our structured EHR case review process. DEs contributing to death associated with DPFs were identified in institutionally designated preventable and non-preventable cases. While e-triggers may be useful for discriminating DE positive from DE negative cases, larger studies are required for validation. Our approach has potential to augment institutional mortality case review processes with respect to DE surveillance.

Frequency of diagnostic errors in the neonatal intensive care unit: a retrospective cohort study

OBJECTIVE

To determine the frequency and etiology of diagnostic errors during the first 7 days of admission for inborn neonatal intensive care unit (NICU) patients.

STUDY DESIGN

We conducted a retrospective cohort study of 600 consecutive inborn admissions. A physician used the "Safer Dx NICU Instrument" to review the electronic health record for the first 7 days of admission, and categorized cases as "yes," "unclear," or "no" for diagnostic error. A secondary reviewer evaluated all "yes" charts plus a random sample of charts in the other categories. Subsequently, all secondary reviewers reviewed records with discordance between primary and secondary review to arrive at consensus.

RESULTS

We identified 37 diagnostic errors (6.2% of study patients) with "substantial agreement" between reviewers (κ = 0.66). The most common diagnostic process breakdown was missed maternal history (51%).

CONCLUSION

The frequency of diagnostic error in inborn NICU patients during the first 7 days of admission is 6.2%.

Inviting patients to identify diagnostic concerns through structured evaluation of their online visit notes

BACKGROUND

The 21st Century Cures Act mandates patients' access to their electronic health record (EHR) notes. To our knowledge, no previous work has systematically invited patients to proactively report diagnostic concerns while documenting and tracking their diagnostic experiences through EHR-based clinician note review.

OBJECTIVE

To test if patients can identify concerns about their diagnosis through structured evaluation of their online visit notes.

METHODS

In a large integrated health system, patients aged 18-85 years actively using the patient portal and seen between October 2019 and February 2020 were invited to respond to an online questionnaire if an EHR algorithm detected any recent unexpected return visit following an initial primary care consultation ("at-risk" visit). We developed and tested an instrument (Safer Dx Patient Instrument) to help patients identify concerns related to several dimensions of the diagnostic process based on notes review and recall of recent "at-risk" visits. Additional questions assessed patients' trust in their providers and their general feelings about the visit. The primary outcome was a self-reported diagnostic concern. Multivariate logistic regression tested whether the primary outcome was predicted by instrument variables.

RESULTS

Of 293 566 visits, the algorithm identified 1282 eligible patients, of whom 486 responded. After applying exclusion criteria, 418 patients were included in the analysis. Fifty-one patients (12.2%) identified a diagnostic concern. Patients were more likely to report a concern if they disagreed with statements "the care plan the provider developed for me addressed all my medical concerns" [odds ratio (OR), 2.65; 95% confidence interval [CI], 1.45-4.87) and "I trust the provider that I saw during my visit" (OR, 2.10; 95% CI, 1.19-3.71) and agreed with the statement "I did not have a good feeling about my visit" (OR, 1.48; 95% CI, 1.09-2.01).

CONCLUSION

Patients can identify diagnostic concerns based on a proactive online structured evaluation of visit notes. This surveillance strategy could potentially improve transparency in the diagnostic process.

Applying a diagnostic excellence framework to assess opportunities to improve recognition of child physical abuse

OBJECTIVES

Diagnostic excellence is an important domain of healthcare quality. Delays in diagnosis have been described in 20-30% of children with abusive injuries. Despite the well characterized epidemiology, improvement strategies remain elusive. We sought to assess the applicability of diagnostic improvement instruments to cases of non-accidental trauma and to identify potential opportunities for system improvement in child physical abuse diagnosis.

METHODS

We purposefully sampled 10 cases identified as having potential for system level interventions and in which the child had prior outpatient encounters to review. Experts in pediatrics, child abuse, and diagnostic improvement independently reviewed each case and completed SaferDx, a validated instrument used to evaluate the diagnostic process. Cases were subsequently discussed to map potential opportunities for improving the diagnostic process to the DEER Taxonomy, which classifies opportunities by type and phase of the diagnostic process.

RESULTS

The most frequent improvement opportunities identified by the SaferDx were in recognition of potential alarm symptoms and in expanding differential diagnosis (5 of 10 cases). The most frequent DEER taxonomy process opportunities were in history taking (8 of 10) and hypothesis generation (7 of 10). Discussion elicited additional opportunities in reconsideration of provisional diagnoses, understanding biopsychosocial risk, and addressing information scatter within the electronic health record (EHR).

CONCLUSIONS

Applying a diagnostic excellence framework facilitated identification of systems opportunities to improve recognition of child abuse including integration of EHR information to support recognition of alarm symptoms, collaboration to support vulnerable families, and communication about diagnostic reasoning.

Diagnostic Error in Pediatrics: A Narrative Review

A priority topic for patient safety research is diagnostic errors. However, despite the significant growth in awareness of their unacceptably high incidence and associated harm, a relative paucity of large, high-quality studies of diagnostic error in pediatrics exists. In this narrative review, we present what is known about the incidence and epidemiology of diagnostic error in pediatrics as well as the established research methods for identifying, evaluating, and reducing diagnostic errors, including their strengths and weaknesses. Additionally, we highlight that pediatric diagnostic error remains an area in need of both innovative research and quality improvement efforts to apply learnings from a rapidly growing evidence base. We propose several key research questions aimed at addressing persistent gaps in the pediatric diagnostic error literature that focus on the foundational knowledge needed to inform effective interventions to reduce the incidence of diagnostic errors and their associated harm. Additional research is needed to better establish the epidemiology of diagnostic error in pediatrics, including identifying high-risk clinical scenarios, patient populations, and groups of diagnoses. A critical need exists for validated measures of both diagnostic errors and diagnostic processes that can be adapted for different clinical settings and standardized for use across varying institutions. Pediatric researchers will need to work collaboratively on large-scale, high-quality studies to accomplish the ultimate goal of reducing diagnostic errors and their associated harm in children by addressing these fundamental gaps in knowledge.

Diagnostic Accuracy of Suspected Appendicitis: A Comparative Analysis of Misdiagnosed Appendicitis in Children

OBJECTIVE

When evaluating suspected appendicitis, limited data support quality benchmarks for negative appendectomy (NA); none exist for delayed diagnosis of appendicitis (DDA). The objectives of this study are the following: (1) to provide preliminary evidence supporting a quality benchmark for DDA and 2) to compare presenting features and diagnostic evaluations of children with NA and DDA with those with pathology-confirmed appendicitis (PCA) diagnosed during initial emergency department (ED) encounter.

METHODS

Secondary analysis of data from a QI project designed to reduce the use computed tomography when evaluating suspected appendicitis using a case-control design. Patients undergoing appendectomy in an academic tertiary care children's hospital system between January 1, 2015, and December 31, 2016 (n = 1,189) were eligible for inclusion in this case-control study. Negative appendectomy was defined as no pathologic change or findings consistent with a different diagnosis. Delayed diagnosis of appendicitis was defined as patients undergoing appendectomy within 7 days of a prior ED visit for a related complaint. Controls of PCA (n = 150) were randomly selected from all cases undergoing appendectomy.

RESULTS

There were 42 NA (3.5%) and 31 DDA (2.6%). Cases of PCA and NA exhibited similar histories, examination findings, and underwent comparable diagnostic evaluations. Cases of PCA more frequently demonstrated a white blood cell count greater than 10 × 103/μL (85% vs 67%; P = 0.01), a left-shift (77% vs 45%; P < 0.001), and an ultrasound interpretation with high probability for appendicitis (73% vs 54%; P = 0.03). Numerous significant differences in history, examination findings, and diagnostic tests performed existed between cases of PCA and DDA.

CONCLUSIONS

Children with PCA and NA present similarly and undergo comparable evaluations resulting in appendectomy. A 3% to 4% NA rate may be unavoidable given these similarities. Presenting features in DDA significantly differ from those of PCA. An irreducible proportion of appendicitis diagnoses may be delayed.

Cerebrovascular disease hospitalizations following emergency department headache visits: A nested case-control study

BACKGROUND

Delayed diagnosis of cerebrovascular disease (CVD) among patients can result in substantial harm. If diagnostic process failures can be identified at emergency department (ED) visits that precede CVD hospitalization, interventions to improve diagnostic accuracy can be developed.

METHODS

We conducted a nested case-control study using a cohort of adult ED patients discharged from a single medical center with a benign headache diagnosis from October 1, 2015 to March 31, 2018. Hospitalizations for CVD within 1 year of index ED visit were identified using a regional health information exchange. Patients with subsequent CVD hospitalization (cases) were individually matched to patients without subsequent hospitalization (controls) using patient age and visit date. Demographic, clinical, and ED process characteristics were assessed via detailed chart review. McNemar's test for categorical and paired t-test for continuous variables were used with statistical significance set at ≤0.05.

RESULTS

Of the 9157 patients with ED headache visits, 57 (0.6%, 95% confidence interval [CI] = 0.5-0.8) had a subsequent CVD hospitalization. Median time from ED visit to hospitalization was 107 days. In 25 patients (43.9%, 25/57) the CVD hospitalization and the index ED visit were at different hospitals. Fifty-three cases and 53 matched controls were included in the final study analysis. Cases and controls had similar baseline demographic and headache characteristics. Cases more often had a history of stroke (32.1% vs. 13.2%, p = 0.02) and neurosurgery (13.2% vs. 1.9%, p = 0.03) prior to the index ED visit. Cases more often had less than two components of the neurologic examination documented (30.2% vs. 11.3%, p = 0.03).

CONCLUSION

We found that 0.6% of patients with an ED headache visit had subsequent CVD hospitalization, often at another medical center. ED visits for headache complaints among patients with prior stroke or neurosurgical procedures may be important opportunities for CVD prevention. Documented neurologic examinations were poorer among cases, which may represent an opportunity for ED process improvement.

Incidence, origins and avoidable harm of missed opportunities in diagnosis: longitudinal patient record review in 21 English general practices

BACKGROUND

Diagnostic error is a global patient safety priority.

OBJECTIVES

To estimate the incidence, origins and avoidable harm of diagnostic errors in English general practice. Diagnostic errors were defined as missed opportunities to make a correct or timely diagnosis based on the evidence available (missed diagnostic opportunities, MDOs).

METHOD

Retrospective medical record reviews identified MDOs in 21 general practices. In each practice, two trained general practitioner reviewers independently conducted case note reviews on 100 randomly selected adult consultations performed during 2013-2014. Consultations where either reviewer identified an MDO were jointly reviewed.

RESULTS

Across 2057 unique consultations, reviewers agreed that an MDO was possible, likely or certain in 89 cases or 4.3% (95% CI 3.6% to 5.2%) of reviewed consultations. Inter-reviewer agreement was higher than most comparable studies (Fleiss' kappa=0.63). Sixty-four MDOs (72%) had two or more contributing process breakdowns. Breakdowns involved problems in the patient-practitioner encounter such as history taking, examination or ordering tests (main or secondary factor in 61 (68%) cases), performance and interpretation of diagnostic tests (31; 35%) and follow-up and tracking of diagnostic information (43; 48%). 37% of MDOs were rated as resulting in moderate to severe avoidable patient harm.

CONCLUSIONS

Although MDOs occurred in fewer than 5% of the investigated consultations, the high numbers of primary care contacts nationally suggest that several million patients are potentially at risk of avoidable harm from MDOs each year. Causes of MDOs were frequently multifactorial, suggesting the need for development and evaluation of multipronged interventions, along with policy changes to support them.

Incidence of diagnostic errors in unplanned hospitalized patients using an automated medical history-taking system with differential diagnosis generator: retrospective observational study (Preprint)

Background

Automated medical history–taking systems that generate differential diagnosis lists have been suggested to contribute to improved diagnostic accuracy. However, the effect of these systems on diagnostic errors in clinical practice remains unknown.

Objective

This study aimed to assess the incidence of diagnostic errors in an outpatient department, where an artificial intelligence (AI)–driven automated medical history–taking system that generates differential diagnosis lists was implemented in clinical practice.

Methods

We conducted a retrospective observational study using data from a community hospital in Japan. We included patients aged 20 years and older who used an AI-driven, automated medical history–taking system that generates differential diagnosis lists in the outpatient department of internal medicine for whom the index visit was between July 1, 2019, and June 30, 2020, followed by unplanned hospitalization within 14 days. The primary endpoint was the incidence of diagnostic errors, which were detected using the Revised Safer Dx Instrument by at least two independent reviewers. To evaluate the effect of differential diagnosis lists from the AI system on the incidence of diagnostic errors, we compared the incidence of these errors between a group where the AI system generated the final diagnosis in the differential diagnosis list and a group where the AI system did not generate the final diagnosis in the list; the Fisher exact test was used for comparison between these groups. For cases with confirmed diagnostic errors, further review was conducted to identify the contributing factors of these errors via discussion among three reviewers, using the Safer Dx Process Breakdown Supplement as a reference.

Results

A total of 146 patients were analyzed. A final diagnosis was confirmed for 138 patients and was observed in the differential diagnosis list from the AI system for 69 patients. Diagnostic errors occurred in 16 out of 146 patients (11.0%, 95% CI 6.4%-17.2%). Although statistically insignificant, the incidence of diagnostic errors was lower in cases where the final diagnosis was included in the differential diagnosis list from the AI system than in cases where the final diagnosis was not included in the list (7.2% vs 15.9%, P=.18).

Conclusions

The incidence of diagnostic errors among patients in the outpatient department of internal medicine who used an automated medical history–taking system that generates differential diagnosis lists seemed to be lower than the previously reported incidence of diagnostic errors. This result suggests that the implementation of an automated medical history–taking system that generates differential diagnosis lists could be beneficial for diagnostic safety in the outpatient department of internal medicine.

Diagnostic Error in the Critically Ill: A Hidden Epidemic?

Epidemiologic studies of diagnostic error in the intensive care unit (ICU) consist mostly of descriptive autopsy series. In these studies, rates of diagnostic errors are approximately 5% to 10%. Recently validated methods for retrospectively measuring error have expanded our understanding of the scope of the problem. These alternative measurement strategies have yielded similar estimates for the frequency of diagnostic error in the ICU. Although there is a fair understanding of the frequency of errors, further research is needed to better define the risk factors for diagnostic error in the ICU.

Learning from Missed Opportunities Through Reflective Practice

Identification of diagnostic errors is difficult but is not alone sufficient for performance improvement. Instead, cases must be reflected on to identify ways to improve decision-making in the future. There are many tools and modalities to retrospectively reflect on action to study medical decisions and outcomes and improve future performance. Reflection in action-in which diagnostic decisions are considered in real-time-may also improve medical decision-making especially through strategies such as structured reflection. Ongoing regular feedback can normalize the discussion about improving decision-making, enable reflective practice, and improve decision making.

nalyzing diagnostic errors in the acute setting: a process-driven approach

OBJECTIVES

We describe an approach for analyzing failures in diagnostic processes in a small, enriched cohort of general medicine patients who expired during hospitalization and experienced medical error. Our objective was to delineate a systematic strategy for identifying frequent and significant failures in the diagnostic process to inform strategies for preventing adverse events due to diagnostic error.

METHODS

Two clinicians independently reviewed detailed records of purposively sampled cases identified from established institutional case review forums and assessed the likelihood of diagnostic error using the Safer Dx instrument. Each reviewer used the modified Diagnostic Error Evaluation and Research (DEER) taxonomy, revised for acute care (41 possible failure points across six process dimensions), to characterize the frequency of failure points (FPs) and significant FPs in the diagnostic process.

RESULTS

Of 166 cases with medical error, 16 were sampled: 13 (81.3%) had one or more diagnostic error(s), and a total of 113 FPs and 30 significant FPs were identified. A majority of significant FPs (63.3%) occurred in "Diagnostic Information and Patient Follow-up" and "Patient and Provider Encounter and Initial Assessment" process dimensions. Fourteen (87.5%) cases had a significant FP in at least one of these dimensions.

CONCLUSIONS

Failures in the diagnostic process occurred across multiple dimensions in our purposively sampled cohort. A systematic analytic approach incorporating the modified DEER taxonomy, revised for acute care, offered critical insights into key failures in the diagnostic process that could serve as potential targets for preventative interventions.

Comparison of Diagnostic Recommendations from Individual Physicians versus the Collective Intelligence of Multiple Physicians in Ambulatory Cases Referred for Specialist Consultation

BACKGROUND

Studies report higher diagnostic accuracy using the collective intelligence (CI) of multiple clinicians compared with individual clinicians. However, the diagnostic process is iterative, and unexplored is the value of CI in improving clinical recommendations leading to a final diagnosis.

METHODS

To compare the appropriateness of diagnostic recommendations advised by individual physicians versus the CI of physicians, we entered actual consultation requests sent by primary care physicians to specialists onto a web-based CI platform capable of collecting diagnostic recommendations (next steps for care) from multiple physicians. We solicited responses to 35 cases (12 endocrinology, 13 gynecology, 10 neurology) from ≥3 physicians of any specialty through the CI platform, which aggregated responses into a CI output. The primary outcome was the appropriateness of individual physician recommendations versus the CI output recommendations, using recommendations agreed upon by 2 specialists in the same specialty as a gold standard. The secondary outcome was the recommendations' potential for harm.

RESULTS

A total of 177 physicians responded. Cases had a median of 7 respondents (interquartile range: 5-10). Diagnostic recommendations in the CI output achieved higher levels of appropriateness (69%) than recommendations from individual physicians (45%; χ² = 5.95, P = 0.015). Of the CI recommendations, 54% were potentially harmful, as compared with 41% of individuals' recommendations (χ² = 2.49, P = 0.11).

LIMITATIONS

Cases were from a single institution. CI was solicited using a single algorithm/platform.

CONCLUSIONS

When seeking specialist guidance, diagnostic recommendations from the CI of multiple physicians are more appropriate than recommendations from most individual physicians, measured against specialist recommendations. Although CI provides useful recommendations, some have potential for harm. Future research should explore how to use CI to improve diagnosis while limiting harm from inappropriate tests/therapies.

Safer Stroke-Dx Instrument: Identifying Stroke Misdiagnosis in the Emergency Department

BACKGROUND

Missed or delayed diagnosis of acute stroke, or false-negative stroke (FNS), at initial emergency department (ED) presentation occurs in ≈9% of confirmed stroke patients. Failure to rapidly diagnose stroke can preclude time-sensitive treatments, resulting in higher risks of severe sequelae and disability. In this study, we developed and tested a modified version of a structured medical record review tool, the Safer Dx Instrument, to identify FNS in a subgroup of hospitalized patients with stroke to gain insight into sources of ED stroke misdiagnosis.

METHODS

We conducted a retrospective cohort study at 2 unaffiliated comprehensive stroke centers. In the development and confirmatory cohorts, we applied the Safer Stroke-Dx Instrument to report the prevalence and documented sources of ED diagnostic error in FNS cases among confirmed stroke patients upon whom an acute stroke was suspected by the inpatient team, as evidenced by stroke code activation or urgent neurological consultation, but not by the ED team. Inter-rater reliability and agreement were assessed using interclass coefficient and kappa values (κ).

RESULTS

Among 183 cases in the development cohort, the prevalence of FNS was 20.2% (95% CI, 15.0-26.7). Too narrow a differential diagnosis and limited neurological examination were common potential sources of error. The interclass coefficient for the Safer Stroke-Dx Instrument items ranged from 0.42 to 0.91, and items were highly correlated with each other. The κ for diagnostic error identification was 0.90 (95% CI, 0.821-0.978) using the Safer Stroke-Dx Instrument. In the confirmatory cohort of 99 cases, the prevalence of FNS was 21.2% (95% CI, 14.2-30.3) with similar sources of diagnostic error identified.

CONCLUSIONS

Hospitalized patients identified by stroke codes and requests for urgent neurological consultation represent an enriched population for the study of diagnostic error in the ED. The Safer Stroke-Dx Instrument is a reliable tool for identifying FNS and sources of diagnostic error.

What Contributes to Diagnostic Error or Delay? A Qualitative Exploration Across Diverse Acute Care Settings in the United States

OBJECTIVES

Diagnostic error and delay is a prevalent and impactful problem. This study was part of a mixed-methods approach to understand the organizational, clinician, and patient factors contributing to diagnostic error and delay among acutely ill patients within a health system, as well as recommendations for the development of tailored, targeted, feasible, and effective interventions.

METHODS

We did a multisite qualitative study using focus group methodology to explore the perspectives of key clinician stakeholders. We used a conceptual framework that characterized diagnostic error and delay as occurring within 1 of 3 stages of the patient's diagnostic journey-critical information gathering, synthesis of key information, and decision making and communication. We developed our moderator guide based on the sociotechnical frameworks previously described by Holden and Singh for understanding noncognitive factors that lead to diagnostic error and delay. Deidentified focus group transcripts were coded in triplicate and to consensus over a series of meetings. A final coded data set was then uploaded into NVivo software. The data were then analyzed to generate overarching themes and categories.

RESULTS

We recruited a total of 64 participants across 4 sites from emergency departments, hospital floor, and intensive care unit settings into 11 focus groups. Clinicians perceive that diverse organizational, communication and coordination, individual clinician, and patient factors interact to impede the process of making timely and accurate diagnoses.

CONCLUSIONS

This study highlights the complex sociotechnical system within which individual clinicians operate and the contributions of systems, processes, and institutional factors to diagnostic error and delay.

Increasing Physician Reporting of Diagnostic Learning Opportunities

BACKGROUND

An estimated 10% of Americans experience a diagnostic error annually, yet little is known about pediatric diagnostic errors. Physician reporting is a promising method for identifying diagnostic errors. However, our pediatric hospital medicine (PHM) division had only 1 diagnostic-related safety report in the preceding 4 years. We aimed to improve attending physician reporting of suspected diagnostic errors from 0 to 2 per 100 PHM patient admissions within 6 months.

METHODS

Our improvement team used the Model for Improvement, targeting the PHM service. To promote a safe reporting culture, we used the term diagnostic learning opportunity (DLO) rather than diagnostic error, defined as a "potential opportunity to make a better or more timely diagnosis." We developed an electronic reporting form and encouraged its use through reminders, scheduled reflection time, and monthly progress reports. The outcome measure, the number of DLO reports per 100 patient admissions, was tracked on an annotated control chart to assess the effect of our interventions over time. We evaluated DLOs using a formal 2-reviewer process.

RESULTS

Over the course of 13 weeks, there was an increase in the number of reports filed from 0 to 1.6 per 100 patient admissions, which met special cause variation, and was subsequently sustained. Most events (66%) were true diagnostic errors and were found to be multifactorial after formal review.

CONCLUSIONS

We used quality improvement methodology, focusing on psychological safety, to increase physician reporting of DLOs. This growing data set has generated nuanced learnings that will guide future improvement work.

Handshake antimicrobial stewardship as a model to recognize and prevent diagnostic errors

OBJECTIVES

Few studies describe the impact of antimicrobial stewardship programs (ASPs) on recognizing and preventing diagnostic errors. Handshake stewardship (HS-ASP) is a novel ASP model that prospectively reviews hospital-wide antimicrobial usage with recommendations made in person to treatment teams. The purpose of this study was to determine if HS-ASP could identify and intervene on potential diagnostic errors for children hospitalized at a quaternary care children's hospital.

METHODS

Previously self-identified "Great Catch" (GC) interventions by the Children's Hospital Colorado HS-ASP team from 10/2014 through 5/2018 were retrospectively reviewed. Each GC was categorized based on the types of recommendations from HS-ASP, including if any diagnostic recommendations were made to the treatment team. Each GC was independently scored using the "Safer Dx Instrument" to determine presence of diagnostic error based on a previously determined cut-off score of ≤1.50. Interrater reliability for the instrument was measured using a randomized subset of one third of GCs.

RESULTS

During the study period, there were 162 GC interventions. Of these, 65 (40%) included diagnostic recommendations by HS-ASP and 19 (12%) had a Safer Dx Score of ≤1.50, (Κ=0.44; moderate agreement). Of those GCs associated with diagnostic errors, the HS-ASP team made a diagnostic recommendation to the primary treatment team 95% of the time.

CONCLUSIONS

Handshake stewardship has the potential to identify and intervene on diagnostic errors for hospitalized children.

Prevalence and characterisation of diagnostic error among 7-day all-cause hospital medicine readmissions: a retrospective cohort study

Background

The prevalence and aetiology of diagnostic error among hospitalised adults is unknown, though likely contributes to patient morbidity and mortality. We aim to identify and characterise the prevalence and types of diagnostic error among patients readmitted within 7 days of hospital discharge.

Methods

Retrospective cohort study at a single urban academic hospital examining adult patients discharged from the medical service and readmitted to the same hospital within 7 days between January and December 2018. The primary outcome was diagnostic error presence, identified through two-physician adjudication using validated tools. Secondary outcomes included severity of error impact and characterisation of diagnostic process failures contributing to error.

Results

There were 391 cases of unplanned 7-day readmission (5.2% of 7507 discharges), of which 376 (96.2%) were reviewed. Twenty-one (5.6%) admissions were found to contain at least one diagnostic error during the index admission. The most common problem areas in the diagnostic process included failure to order needed test(s) (n=11, 52.4%), erroneous clinician interpretation of test(s) (n=10, 47.6%) and failure to consider the correct diagnosis (n=8, 38.1%). Nineteen (90.5%) of the diagnostic errors resulted in moderate clinical impact, primarily due to short-term morbidity or contribution to the readmission.

Conclusion

The prevalence of diagnostic error among 7-day medical readmissions was 5.6%. The most common drivers of diagnostic error were related to clinician diagnostic reasoning. Efforts to reduce diagnostic error should include strategies to augment diagnostic reasoning and improve clinician decision-making around diagnostic studies.

Operational measurement of diagnostic safety: state of the science

Reducing the incidence of diagnostic errors is increasingly a priority for government, professional, and philanthropic organizations. Several obstacles to measurement of diagnostic safety have hampered progress toward this goal. Although a coordinated national strategy to measure diagnostic safety remains an aspirational goal, recent research has yielded practical guidance for healthcare organizations to start using measurement to enhance diagnostic safety. This paper, concurrently published as an Issue Brief by the Agency for Healthcare Research and Quality, issues a "call to action" for healthcare organizations to begin measurement efforts using data sources currently available to them. Our aims are to outline the state of the science and provide practical recommendations for organizations to start identifying and learning from diagnostic errors. Whether by strategically leveraging current resources or building additional capacity for data gathering, nearly all organizations can begin their journeys to measure and reduce preventable diagnostic harm.

Identifying psychiatric diagnostic errors with the Safer Dx Instrument

OBJECTIVE

Diagnostic errors in psychiatry are understudied partly because they are difficult to measure. The current study aimed to adapt and test the Safer Dx Instrument, a structured tool to review electronic health records (EHR) for errors in medical diagnoses, to evaluate errors in anxiety diagnoses to improve measurement of psychiatric diagnostic errors.

DESIGN

The iterative adaptation process included a review of the revised Safer Dx-Mental Health Instrument by mental health providers to ensure content and face validity and review by a psychometrician to ensure methodologic validity and pilot testing of the revised instrument.

SETTINGS

None.

PARTICIPANTS

Pilot testing was conducted on 128 records of patients diagnosed with anxiety in integrated primary care mental health clinics. Cases with anxiety diagnoses documented in progress notes but not included as a diagnosis for the encounter (n = 25) were excluded.

INTERVENTION(S)

None.

MAIN OUTCOME MEASURE(S)

None.

RESULTS

Of 103 records meeting the inclusion criteria, 62 likely involved a diagnostic error (42 from use of unspecified anxiety diagnosis when a specific anxiety diagnosis was warranted; 20 from use of unspecified anxiety diagnosis when anxiety symptoms were either undocumented or documented but not severe enough to warrant diagnosis). Reviewer agreement on presence/absence of errors was 88% (κ = 0.71).

CONCLUSION

The revised Safer Dx-Mental Health Instrument has a high reliability for detecting anxiety-related diagnostic errors and deserves testing in additional psychiatric populations and clinical settings.

Frequency, Risk Factors, Causes, and Consequences of Diagnostic Errors in Critically Ill Medical Patients: A Retrospective Cohort Study

OBJECTIVE

Diagnostic errors are a source of significant morbidity and mortality but understudied in the critically ill. We sought to characterize the frequency, causes, consequences, and risk factors of diagnostic errors among unplanned ICU admissions.

DESIGN

We conducted a retrospective cohort study of randomly selected nonsurgical ICU admissions between July 2015 and June 2016.

SETTING

Medical ICU at a tertiary academic medical center.

SUBJECTS

Critically ill adults with unplanned admission to the medical ICU.

MEASUREMENTS AND MAIN RESULTS

The primary investigator reviewed patient records using a modified version of the Safer Dx instrument, a validated instrument for detecting diagnostic error. Two intensivists performed secondary reviews of possible errors, and reviewers met periodically to adjudicate errors by consensus. For each confirmed error, we judged harm on a 1-6 rating scale. We also collected detailed demographic and clinical data for each patient. We analyzed 256 unplanned ICU admissions and identified 18 diagnostic errors (7% of admissions). All errors were associated with harm, and only six errors (33%) were recognized by the ICU team within the first 24 hours. More women than men experienced a diagnostic error (11.7% vs 2.7%; p = 0.015, χ test). On multivariable logistic regression analysis, female sex remained independently associated with risk of diagnostic error both at admission (odds ratio, 5.18; 95% CI, 1.34-20.08) and at 24 hours (odds ratio, 11.6; 95% CI, 1.37-98.6). Similarly, Quick Sequential Organ Failure Assessment score greater than or equal to 2 at admission was independently associated with diagnostic error (odds ratio, 5.73; 95% CI, 1.72-19.01).

CONCLUSIONS

Diagnostic errors may be an underappreciated source of ICU-related harm. Women and higher acuity patients appear to be at increased risk for such errors. Further research is merited to define the scope of error-associated harm and to clarify risk factors for diagnostic errors among the critically ill.

Factors Associated With Diagnostic Error on Admission to a PICU: A Pilot Study

OBJECTIVES

Diagnostic errors can harm critically ill children. However, we know little about their prevalence in PICUs and factors associated with error. The objective of this pilot study was to determine feasibility of record review to identify patient, provider, and work system factors associated with diagnostic errors during the first 12 hours after PICU admission.

DESIGN

Pilot retrospective cohort study with structured record review using a structured tool (Safer Dx instrument) to identify diagnostic error.

SETTING

Academic tertiary referral PICU.

PATIENTS

Patients 0-17 years old admitted nonelectively to the PICU.

INTERVENTIONS

None.

MEASUREMENTS AND MAIN RESULTS

Four of 50 patients (8%) had diagnostic errors in the first 12 hours after admission. The Safer Dx instrument helped identify delayed diagnoses of chronic ear infection, increased intracranial pressure (two cases), and Bartonella encephalitis. We calculated that 610 PICU admissions are needed to achieve 80% power (α = 0.05) to detect significant associations with error.

CONCLUSIONS

Our pilot study found four patients with diagnostic error out of 50 children admitted nonelectively to a PICU. Retrospective record review using a structured tool to identify diagnostic errors is feasible in this population. Pilot data are being used to inform a larger and more definitive multicenter study.

Assessing diagnostic error in cerebral venous thrombosis via detailed chart review

Background Diagnostic error in cerebral venous thrombosis (CVT) has been understudied despite the harm associated with misdiagnosis of other cerebrovascular diseases as well as the known challenges of evaluating non-specific neurological symptoms in clinical practice. Methods We conducted a retrospective cohort study of CVT patients hospitalized at a single center. Two independent reviewers used a medical record review tool, the Safer Dx Instrument, to identify diagnostic errors. Demographic and clinical factors were abstracted. We compared subjects with and without a diagnostic error using the t-test for continuous variables and the chi-square (χ2) test or Fisher's exact test for categorical variables; an alpha of 0.05 was the cutoff for significance. Results A total of 72 CVT patients initially met study inclusion criteria; 19 were excluded due to incomplete medical records. Of the 53 patients included in the final analysis, the mean age was 48 years and 32 (60.4%) were women. Diagnostic error occurred in 11 cases [20.8%; 95% confidence interval (CI) 11.8-33.6%]. Subjects with diagnostic errors were younger (42 vs. 49 years, p = 0.13), more often women (81.8% vs. 54.8%, p = 0.17), and were significantly more likely to have a past medical history of a headache disorder prior to the index CVT visit (7.1% vs. 36.4%, p = 0.03). Conclusions Nearly one in five patients with complete medical records experienced a diagnostic error. Prior history of headache was the only evaluated clinical factor that was more common among those with an error in diagnosis. Future work on distinguishing primary from secondary headaches to improve diagnostic accuracy in acute neurological disease is warranted.

Pediatric Clinician Comfort Discussing Diagnostic Errors for Improving Patient Safety: A Survey

INTRODUCTION

Meaningful conversations about diagnostic errors require safety cultures where clinicians are comfortable discussing errors openly. However, clinician comfort discussing diagnostic errors publicly and barriers to these discussions remain unexplored. We compared clinicians' comfort discussing diagnostic errors to other medical errors and identified barriers to open discussion.

METHODS

Pediatric clinicians at 4 hospitals were surveyed between May and June 2018. The survey assessed respondents' comfort discussing medical errors (with varying degrees of system versus individual clinician responsibility) during morbidity and mortality conferences and privately with peers. Respondents reported the most significant barriers to discussing diagnostic errors publicly. Poststratification weighting accounted for nonresponse bias; the Benjamini-Hochberg adjustment was applied to control for false discovery (significance set at P < 0.018).

RESULTS

Clinicians (n = 838; response rate 22.6%) were significantly less comfortable discussing all error types during morbidity and mortality conferences than privately (P < 0.004) and significantly less comfortable discussing diagnostic errors compared with other medical errors (P < 0.018). Comfort did not differ by clinician type or years in practice; clinicians at one institution were significantly less comfortable discussing diagnostic errors compared with peers at other institutions. The most frequently cited barriers to discussing diagnostic errors publicly included feeling like a bad clinician, loss of reputation, and peer judgment of knowledge base and decision-making.

CONCLUSIONS

Clinicians are more uncomfortable discussing diagnostic errors than other types of medical errors. The most frequent barriers involve the public perception of clinical performance. Addressing this aspect of safety culture may improve clinician participation in efforts to reduce harm from diagnostic errors.

Structured case reviews for organizational learning about diagnostic vulnerabilities: initial experiences from two medical centers

Background

An organization’s ability to identify and learn from opportunities for improvement (OFI) is key to increasing diagnostic safety. Many lack effective processes required to capitalize on these learning opportunities. We describe two parallel attempts at creating such a process and identifying generalizable lessons and learn from them.

Methods

Triggered case review programs were created independently at two organizations, Site 1 (Regions Hospital, HealthPartners, Saint Paul, MN, USA) and site 2 (University of California, San Diego). Both used a five-step process to create the review system and provide feedback: (1) identify trigger criteria; (2) establish a review panel; (3) develop a system to conduct reviews; (4) perform reviews; and (5) provide feedback.

Results

Site 1 identified 112 OFI in 184 case reviews (61%), with 66 (59%) provider OFI and 46 (41%) system OFI. Site 2 focused mainly on systems OFI identifying 105 OFI in 346 cases (30%). Opportunities at both sites were variable; common themes included test result management and communication across teams in peri-procedural care and with consultants. Of provider-initiated reviews, 67% of cases had an OFI at site 1 and 87% at site 2.

Conclusions

Lessons learned include the following: (1) peer review of cases provides opportunities to learn and calibrate diagnostic and management decisions at an organizational level; (2) sharing cases in review groups supports a culture of open discussion of OFIs; (3) reviews focused on diagnostic safety identify opportunities that may complement other organization-wide review opportunities.

Recommendations for using the Revised Safer Dx Instrument to help measure and improve diagnostic safety

The medical record continues to be one of the most useful and accessible sources of information to examine the diagnostic process. However, medical record review studies of diagnostic errors have often used subjective judgments and found low inter-rater agreement among reviewers when determining the presence or absence of diagnostic error. In our previous work, we developed a structured data-collection instrument, called the Safer Dx Instrument, consisting of objective criteria to improve the accuracy of assessing diagnostic errors in primary care. This paper proposes recommendations on how clinicians and health care organizations could use the Revised Safer Dx Instrument in identifying and understanding missed opportunities to make correct and timely diagnoses. The instrument revisions addressed both methodological and implementation issues identified during initial use and included refinements to the instrument to allow broader application across all health care settings. In addition to leveraging knowledge from piloting the instrument in several health care settings, we gained insights from multiple researchers who had used the instrument in studies involving emergency care, inpatient care and intensive care unit settings. This allowed us to enhance and extend the scope of this previously validated data collection instrument. In this paper, we describe the refinement process and provide recommendations for application and use of the Revised Safer Dx Instrument across a broad range of health care settings. The instrument can help users identify potential diagnostic errors in a standardized way for further analysis and safety improvement efforts as well as provide data for clinician feedback and reflection. With wider adoption and use by clinicians and health systems, the Revised Safer Dx Instrument could help propel the science of measuring and reducing diagnostic errors forward.

Missed Diagnosis of New-Onset Systolic Heart Failure at First Presentation in Children with No Known Heart Disease

OBJECTIVE

To determine frequency of missed heart failure diagnosis at first presentation among children with no known heart disease admitted with new-onset heart failure.

STUDY DESIGN

Using a retrospective design, we reviewed electronic medical records of all patients aged <21 years with no known heart disease, hospitalized with new-onset heart failure during 2003-2015 at a tertiary-quaternary care institution. We assessed records for missed diagnosis of heart failure (primary outcome), associated process breakdowns, and clinical outcomes using a structured data collection instrument.

RESULTS

Of 191 patients meeting inclusion criteria, 49% (94/191) were missed on first presentation. Most common incorrect diagnostic labels given to "missed" patients were bacterial infection (29%; 27/94), followed by viral illness (22%; 21/94) and gastroenteritis/hepatitis (21%; 20/94). On multivariable analysis, presentation to primary care provider (PCP), longer duration of symptoms (median 7 days), more than 2 symptoms of heart failure, and nausea/emesis were associated with missed diagnosis. On examining process breakdowns, 49% had errors in history-taking and 50% had no documentation of differential diagnoses. There was no difference in hospital mortality, length of stay, or mechanical circulatory support in missed vs not-missed cohorts. Unnecessary noninvasive and invasive tests were performed in 18% and 4% of patients, respectively.

CONCLUSIONS

Nearly one-half of children with no known heart disease hospitalized with systolic heart failure were missed at first presentation and underwent significant nonrelevant treatment and testing. Initial presentation to the PCP, longer duration of symptoms before presentation, and nausea/emesis were associated with missed diagnosis.

Tracking Progress in Improving Diagnosis: A Framework for Defining Undesirable Diagnostic Events

Diagnostic error is a prevalent, harmful, and costly phenomenon. Multiple national health care and governmental organizations have recently identified the need to improve diagnostic safety as a high priority. A major barrier, however, is the lack of standardized, reliable methods for measuring diagnostic safety. Given the absence of reliable and valid measures for diagnostic errors, we need methods to help establish some type of baseline diagnostic performance across health systems, as well as to enable researchers and health systems to determine the impact of interventions for improving the diagnostic process. Multiple approaches have been suggested but none widely adopted. We propose a new framework for identifying "undesirable diagnostic events" (UDEs) that health systems, professional organizations, and researchers could further define and develop to enable standardized measurement and reporting related to diagnostic safety. We propose an outline for UDEs that identifies both conditions prone to diagnostic error and the contexts of care in which these errors are likely to occur. Refinement and adoption of this framework across health systems can facilitate standardized measurement and reporting of diagnostic safety.

Understanding diagnostic safety in emergency medicine: A case-by-case review of closed ED malpractice claims

The report Improving Diagnosis in Health Care calls for collaboration between professional liability insurance carriers and health care providers to identify opportunities to improve diagnostic performance. We used this collaborative approach and involved risk management/patient safety professionals and emergency medicine physician reviewers to analyze diagnosis-related emergency medicine closed claims from a large malpractice insurer. Our aim was to identify opportunities for risk reduction and to develop an approach for improving at-risk processes. Analysis of these cases revealed several missed opportunities in the diagnostic process. A collaborative approach offered greater insight into diagnosis process failures that may not have been evident if cases were reviewed in silos. Focused review findings led to a multidisciplinary improvement collaborative to develop clinical guidelines for improving at-risk practices and informed a simulation-based training initiative.

Symptom-Disease Pair Analysis of Diagnostic Error (SPADE): a conceptual framework and methodological approach for unearthing misdiagnosis-related harms using big data

BACKGROUND

The public health burden associated with diagnostic errors is likely enormous, with some estimates suggesting millions of individuals are harmed each year in the USA, and presumably many more worldwide. According to the US National Academy of Medicine, improving diagnosis in healthcare is now considered 'a moral, professional, and public health imperative.' Unfortunately, well-established, valid and readily available operational measures of diagnostic performance and misdiagnosis-related harms are lacking, hampering progress. Existing methods often rely on judging errors through labour-intensive human reviews of medical records that are constrained by poor clinical documentation, low reliability and hindsight bias.

METHODS

Key gaps in operational measurement might be filled via thoughtful statistical analysis of existing large clinical, billing, administrative claims or similar data sets. In this manuscript, we describe a method to quantify and monitor diagnostic errors using an approach we call 'Symptom-Disease Pair Analysis of Diagnostic Error' (SPADE).

RESULTS

We first offer a conceptual framework for establishing valid symptom-disease pairs illustrated using the well-known diagnostic error dyad of dizziness-stroke. We then describe analytical methods for both look-back (case-control) and look-forward (cohort) measures of diagnostic error and misdiagnosis-related harms using 'big data'. After discussing the strengths and limitations of the SPADE approach by comparing it to other strategies for detecting diagnostic errors, we identify the sources of validity and reliability that undergird our approach.

CONCLUSION

SPADE-derived metrics could eventually be used for operational diagnostic performance dashboards and national benchmarking. This approach has the potential to transform diagnostic quality and safety across a broad range of clinical problems and settings.

An electronic trigger based on care escalation to identify preventable adverse events in hospitalised patients

Background

Methods to identify preventable adverse events typically have low yield and efficiency. We refined the methods of Institute of Healthcare Improvement’s Global Trigger Tool (GTT) application and leveraged electronic health record (EHR) data to improve detection of preventable adverse events, including diagnostic errors.

Methods

We queried the EHR data repository of a large health system to identify an ‘index hospitalization’ associated with care escalation (defined as transfer to the intensive care unit (ICU) or initiation of rapid response team (RRT) within 15 days of admission) between March 2010 and August 2015. To enrich the record review sample with unexpected events, we used EHR clinical data to modify the GTT algorithm and limited eligible patients to those at lower risk for care escalation based on younger age and presence of minimal comorbid conditions. We modified the GTT review methodology; two physicians independently reviewed eligible ‘e-trigger’ positive records to identify preventable diagnostic and care management events.

Results

Of 88 428 hospitalisations, 887 were associated with care escalation (712 ICU transfers and 175 RRTs), of which 92 were flagged as trigger-positive and reviewed. Preventable adverse events were detected in 41 cases, yielding a trigger positive predictive value of 44.6% (reviewer agreement 79.35%; Cohen’s kappa 0.573). We identified 7 (7.6%) diagnostic errors and 34 (37.0%) care management-related events: 24 (26.1%) adverse drug events, 4 (4.3%) patient falls, 4 (4.3%) procedure-related complications and 2 (2.2%) hospital-associated infections. In most events (73.1%), there was potential for temporary harm.

Conclusion

We developed an approach using an EHR data-based trigger and modified review process to efficiently identify hospitalised patients with preventable adverse events, including diagnostic errors. Such e-triggers can help overcome limitations of currently available methods to detect preventable harm in hospitalised patients.

The global burden of diagnostic errors in primary care

Diagnosis is one of the most important tasks performed by primary care physicians. The World Health Organization (WHO) recently prioritized patient safety areas in primary care, and included diagnostic errors as a high-priority problem. In addition, a recent report from the Institute of Medicine in the USA, 'Improving Diagnosis in Health Care', concluded that most people will likely experience a diagnostic error in their lifetime. In this narrative review, we discuss the global significance, burden and contributory factors related to diagnostic errors in primary care. We synthesize available literature to discuss the types of presenting symptoms and conditions most commonly affected. We then summarize interventions based on available data and suggest next steps to reduce the global burden of diagnostic errors. Research suggests that we are unlikely to find a 'magic bullet' and confirms the need for a multifaceted approach to understand and address the many systems and cognitive issues involved in diagnostic error. Because errors involve many common conditions and are prevalent across all countries, the WHO's leadership at a global level will be instrumental to address the problem. Based on our review, we recommend that the WHO consider bringing together primary care leaders, practicing frontline clinicians, safety experts, policymakers, the health IT community, medical education and accreditation organizations, researchers from multiple disciplines, patient advocates, and funding bodies among others, to address the many common challenges and opportunities to reduce diagnostic error. This could lead to prioritization of practice changes needed to improve primary care as well as setting research priorities for intervention development to reduce diagnostic error.

Finding Diagnostic Errors in Children Admitted to the PICU

OBJECTIVES

To determine whether the Safer Dx Instrument, a structured tool for finding diagnostic errors in primary care, can be used to reliably detect diagnostic errors in patients admitted to a PICU.

DESIGN AND SETTING

The Safer Dx Instrument consists of 11 questions to evaluate the diagnostic process and a final question to determine if diagnostic error occurred. We used the instrument to analyze four "high-risk" patient cohorts admitted to the PICU between June 2013 and December 2013.

PATIENTS

High-risk cohorts were defined as cohort 1: patients who were autopsied; cohort 2: patients seen as outpatients within 2 weeks prior to PICU admission; cohort 3: patients transferred to PICU unexpectedly from an acute care floor after a rapid response and requiring vasoactive medications and/or endotracheal intubation due to decompensation within 24 hours; and cohort 4: patients transferred to PICU unexpectedly from an acute care floor after a rapid response without subsequent decompensation in 24 hours.

INTERVENTIONS

Two clinicians used the instrument to independently review records in each cohort for diagnostic errors, defined as missed opportunities to make a correct or timely diagnosis. Errors were confirmed by senior expert clinicians.

MEASUREMENTS AND MAIN RESULTS

Diagnostic errors were present in 26 of 214 high-risk patient records (12.1%; 95% CI, 8.2-17.5%) with the following frequency distribution: cohort 1: two of 16 (12.5%); cohort 2: one of 41 (2.4%); cohort 3: 13 of 44 (29.5%); and cohort 4: 10 of 113 (8.8%). Overall initial reviewer agreement was 93.6% (κ, 0.72). Infections and neurologic conditions were the most commonly missed diagnoses across all high-risk cohorts (16/26).

CONCLUSIONS

The Safer Dx Instrument has high reliability and validity for diagnostic error detection when used in high-risk pediatric care settings. With further validation in additional clinical settings, it could be useful to enhance learning and feedback about diagnostic safety in children.