Finding Diagnostic Errors in Children Admitted to the PICU

Effect of contextual factors on the prevalence of diagnostic errors among patients managed by physicians of the same specialty: a single-centre retrospective observational study

BACKGROUND

There has been growing recognition that contextual factors influence the physician's cognitive processes. However, given that cognitive processes may depend on the physicians' specialties, the effects of contextual factors on diagnostic errors reported in previous studies could be confounded by difference in physicians.

OBJECTIVE

This study aimed to clarify whether contextual factors such as location and consultation type affect diagnostic accuracy.

METHODS

We reviewed the medical records of 1992 consecutive outpatients consulted by physicians from the Department of Diagnostic and Generalist Medicine in a university hospital between 1 January and 31 December 2019. Diagnostic processes were assessed using the Revised Safer Dx Instrument. Patients were categorised into three groups according to contextual factors (location and consultation type): (1) referred patients with scheduled visit to the outpatient department; (2) patients with urgent visit to the outpatient department; and (3) patients with emergency visit to the emergency room. The effect of the contextual factors on the prevalence of diagnostic errors was investigated using logistic regression analysis.

RESULTS

Diagnostic errors were observed in 12 of 534 referred patients with scheduled visit to the outpatient department (2.2%), 3 of 599 patients with urgent visit to the outpatient department (0.5%) and 13 of 859 patients with emergency visit to the emergency room (1.5%). Multivariable logistic regression analysis showed a significantly higher prevalence of diagnostic errors in referred patients with scheduled visit to the outpatient department than in patients with urgent visit to the outpatient department (OR 4.08, p=0.03), but no difference between patients with emergency and urgent visit to the emergency room and outpatient department, respectively.

CONCLUSION

Contextual factors such as consultation type may affect diagnostic errors; however, since the differences in the prevalence of diagnostic errors were small, the effect of contextual factors on diagnostic accuracy may be small in physicians working in different care settings.

Prevalence and Characteristics of Diagnostic Error in Pediatric Critical Care: A Multicenter Study

OBJECTIVES

Effective interventions to prevent diagnostic error among critically ill children should be informed by diagnostic error prevalence and etiologies. We aimed to determine the prevalence and characteristics of diagnostic errors and identify factors associated with error in patients admitted to the PICU.

DESIGN

Multicenter retrospective cohort study using structured medical record review by trained clinicians using the Revised Safer Dx instrument to identify diagnostic error (defined as missed opportunities in diagnosis). Cases with potential errors were further reviewed by four pediatric intensivists who made final consensus determinations of diagnostic error occurrence. Demographic, clinical, clinician, and encounter data were also collected.

SETTING

Four academic tertiary-referral PICUs.

PATIENTS

Eight hundred eighty-two randomly selected patients 0-18 years old who were nonelectively admitted to participating PICUs.

INTERVENTIONS

None.

MEASUREMENTS AND MAIN RESULTS

Of 882 patient admissions, 13 (1.5%) had a diagnostic error up to 7 days after PICU admission. Infections (46%) and respiratory conditions (23%) were the most common missed diagnoses. One diagnostic error caused harm with a prolonged hospital stay. Common missed diagnostic opportunities included failure to consider the diagnosis despite a suggestive history (69%) and failure to broaden diagnostic testing (69%). Unadjusted analysis identified more diagnostic errors in patients with atypical presentations (23.1% vs 3.6%, p = 0.011), neurologic chief complaints (46.2% vs 18.8%, p = 0.024), admitting intensivists greater than or equal to 45 years old (92.3% vs 65.1%, p = 0.042), admitting intensivists with more service weeks/year (mean 12.8 vs 10.9 wk, p = 0.031), and diagnostic uncertainty on admission (77% vs 25.1%, p < 0.001). Generalized linear mixed models determined that atypical presentation (odds ratio [OR] 4.58; 95% CI, 0.94-17.1) and diagnostic uncertainty on admission (OR 9.67; 95% CI, 2.86-44.0) were significantly associated with diagnostic error.

CONCLUSIONS

Among critically ill children, 1.5% had a diagnostic error up to 7 days after PICU admission. Diagnostic errors were associated with atypical presentations and diagnostic uncertainty on admission, suggesting possible targets for intervention.

Opportunities for Diagnostic Improvement Among Pediatric Hospital Readmissions

OBJECTIVES

Diagnostic errors, termed "missed opportunities for improving diagnosis" (MOIDs), are known sources of harm in children but have not been well characterized in pediatric hospital medicine. Our objectives were to systematically identify and describe MOIDs among general pediatric patients who experienced hospital readmission, outline improvement opportunities, and explore factors associated with increased risk of MOID.

PATIENTS AND METHODS

Our retrospective cohort study included unplanned readmissions within 15 days of discharge from a freestanding children's hospital (October 2018-September 2020). Health records from index admissions and readmissions were independently reviewed and discussed by practicing inpatient physicians to identify MOIDs using an established instrument, SaferDx. MOIDs were evaluated using a diagnostic-specific tool to identify improvement opportunities within the diagnostic process.

RESULTS

MOIDs were identified in 22 (6.3%) of 348 readmissions. Opportunities for improvement included: delay in considering the correct diagnosis (n = 11, 50%) and failure to order needed test(s) (n = 10, 45%). Patients with MOIDs were older (median age: 3.8 [interquartile range 1.5-11.2] vs 1.0 [0.3-4.9] years) than patients without MOIDs but similar in sex, primary language, race, ethnicity, and insurance type. We did not identify conditions associated with higher risk of MOID. Lower respiratory tract infections accounted for 26% of admission diagnoses but only 1 (4.5%) case of MOID.

CONCLUSIONS

Standardized review of pediatric readmissions identified MOIDs and opportunities for improvement within the diagnostic process, particularly in clinician decision-making. We identified conditions with low incidence of MOID. Further work is needed to better understand pediatric populations at highest risk for MOID.

Opportunities to improve diagnosis in emergency transfers to the pediatric intensive care unit

BACKGROUND

Late recognition of in-hospital deterioration is a source of preventable harm. Emergency transfers (ET), when hospitalized patients require intensive care unit (ICU) interventions within 1 h of ICU transfer, are a proximal measure of late recognition associated with increased mortality and length of stay (LOS).

OBJECTIVE

To apply diagnostic process improvement frameworks to identify missed opportunities for improvement in diagnosis (MOID) in ETs and evaluate their association with outcomes.

DESIGN, SETTINGS, AND PARTICIPANTS

A single-center retrospective cohort study of ETs, January 2015 to June 2019. ET criteria include intubation, vasopressor initiation, or≥ $\ge \phantom{\rule{}{0ex}}$ 60 mL/kg fluid resuscitation 1 h before to 1 h after ICU transfer. The primary exposure was the presence of MOID, determined using SaferDx. Cases were screened by an ICU and non-ICU physician. Final determinations were made by an interdisciplinary group. Diagnostic process improvement opportunities were identified.

MAIN OUTCOME AND MEASURES

Primary outcomes were in-hospital mortality and posttransfer LOS, analyzed by multivariable regression adjusting for age, service, deterioration category, and pretransfer LOS.

RESULTS

MOID was identified in 37 of 129 ETs (29%, 95% confidence interval [CI] 21%-37%). Cases with MOID differed in originating service, but not demographically. Recognizing the urgency of an identified condition was the most common diagnostic process opportunity. ET cases with MOID had higher odds of mortality (odds ratio 5.5; 95% CI 1.5-20.6; p = .01) and longer posttransfer LOS (rate ratio 1.7; 95% CI 1.1-2.6; p = .02).

CONCLUSION

MOID are common in ETs and are associated with increased mortality risk and posttransfer LOS. Diagnostic improvement strategies should be leveraged to support earlier recognition of clinical deterioration.

What Can We Learn From In-Depth Analysis of Human Errors Resulting in Diagnostic Errors in the Emergency Department: An Analysis of Serious Adverse Event Reports

INTRODUCTION

Human error plays a vital role in diagnostic errors in the emergency department. A thorough analysis of these human errors, using information-rich reports of serious adverse events (SAEs), could help to better study and understand the causes of these errors and formulate more specific recommendations.

METHODS

We studied 23 SAE reports of diagnostic events in emergency departments of Dutch general hospitals and identified human errors. Two researchers independently applied the Safer Dx Instrument, Diagnostic Error Evaluation and Research Taxonomy, and the Model of Unsafe acts to analyze reports.

RESULTS

Twenty-one reports contained a diagnostic error, in which we identified 73 human errors, which were mainly based on intended actions (n = 69) and could be classified as mistakes (n = 56) or violations (n = 13). Most human errors occurred during the assessment and testing phase of the diagnostic process.

DISCUSSION

The combination of different instruments and information-rich SAE reports allowed for a deeper understanding of the mechanisms underlying diagnostic error. Results indicated that errors occurred most often during the assessment and the testing phase of the diagnostic process. Most often, the errors could be classified as mistakes and violations, both intended actions. These types of errors are in need of different recommendations for improvement, as mistakes are often knowledge based, whereas violations often happen because of work and time pressure. These analyses provided valuable insights for more overarching recommendations to improve diagnostic safety and would be recommended to use in future research and analysis of (serious) adverse events.

Frequency of diagnostic errors in the neonatal intensive care unit: a retrospective cohort study

OBJECTIVE

To determine the frequency and etiology of diagnostic errors during the first 7 days of admission for inborn neonatal intensive care unit (NICU) patients.

STUDY DESIGN

We conducted a retrospective cohort study of 600 consecutive inborn admissions. A physician used the "Safer Dx NICU Instrument" to review the electronic health record for the first 7 days of admission, and categorized cases as "yes," "unclear," or "no" for diagnostic error. A secondary reviewer evaluated all "yes" charts plus a random sample of charts in the other categories. Subsequently, all secondary reviewers reviewed records with discordance between primary and secondary review to arrive at consensus.

RESULTS

We identified 37 diagnostic errors (6.2% of study patients) with "substantial agreement" between reviewers (κ = 0.66). The most common diagnostic process breakdown was missed maternal history (51%).

CONCLUSION

The frequency of diagnostic error in inborn NICU patients during the first 7 days of admission is 6.2%.

Use of e-triggers to identify diagnostic errors in the paediatric ED

BACKGROUND

Diagnostic errors (DxEs) are an understudied source of patient harm in children rarely captured in current adverse event reporting systems. Applying electronic triggers (e-triggers) to electronic health records shows promise in identifying DxEs but has not been used in the emergency department (ED) setting.

OBJECTIVES

To assess the performance of an e-trigger and subsequent manual screening for identifying probable DxEs among children with unplanned admission following a prior ED visit and to compare performance to existing incident reporting systems.

DESIGN/METHODS

Retrospective single-centre cohort study of children ages 0-22 admitted within 14 days of a previous ED visit between 1 January 2018 and 31 December 2019. Subjects were identified by e-trigger, screened to identify cases where index visit and hospital discharge diagnoses were potentially related but pathophysiologically distinct, and then these screened-in cases were reviewed for DxE using the SaferDx Instrument. Cases of DxE identified by e-trigger were cross-referenced against existing institutional incident reporting systems.

RESULTS

An e-trigger identified 1915 unplanned admissions (7.7% of 24 849 total admissions) with a preceding index visit. 453 (23.7%) were screened in and underwent review using SaferDx. 92 cases were classified as likely DxEs, representing 0.4% of all hospital admissions, 4.8% among those selected by e-trigger and 20.3% among those screened in for review. Half of cases were reviewed by two reviewers using SaferDx with substantial inter-rater reliability (Cohen's κ=0.65 (95% CI 0.54 to 0.75)). Six (6.5%) cases had been reported elsewhere: two to the hospital's incident reporting system and five to the ED case review team (one reported to both).

CONCLUSION

An e-trigger coupled with manual screening enriched a cohort of patients at risk for DxEs. Fewer than 10% of DxEs were identified through existing surveillance systems, suggesting that they miss a large proportion of DxEs. Further study is required to identify specific clinical presentations at risk of DxEs.

Inviting patients to identify diagnostic concerns through structured evaluation of their online visit notes

BACKGROUND

The 21st Century Cures Act mandates patients' access to their electronic health record (EHR) notes. To our knowledge, no previous work has systematically invited patients to proactively report diagnostic concerns while documenting and tracking their diagnostic experiences through EHR-based clinician note review.

OBJECTIVE

To test if patients can identify concerns about their diagnosis through structured evaluation of their online visit notes.

METHODS

In a large integrated health system, patients aged 18-85 years actively using the patient portal and seen between October 2019 and February 2020 were invited to respond to an online questionnaire if an EHR algorithm detected any recent unexpected return visit following an initial primary care consultation ("at-risk" visit). We developed and tested an instrument (Safer Dx Patient Instrument) to help patients identify concerns related to several dimensions of the diagnostic process based on notes review and recall of recent "at-risk" visits. Additional questions assessed patients' trust in their providers and their general feelings about the visit. The primary outcome was a self-reported diagnostic concern. Multivariate logistic regression tested whether the primary outcome was predicted by instrument variables.

RESULTS

Of 293 566 visits, the algorithm identified 1282 eligible patients, of whom 486 responded. After applying exclusion criteria, 418 patients were included in the analysis. Fifty-one patients (12.2%) identified a diagnostic concern. Patients were more likely to report a concern if they disagreed with statements "the care plan the provider developed for me addressed all my medical concerns" [odds ratio (OR), 2.65; 95% confidence interval [CI], 1.45-4.87) and "I trust the provider that I saw during my visit" (OR, 2.10; 95% CI, 1.19-3.71) and agreed with the statement "I did not have a good feeling about my visit" (OR, 1.48; 95% CI, 1.09-2.01).

CONCLUSION

Patients can identify diagnostic concerns based on a proactive online structured evaluation of visit notes. This surveillance strategy could potentially improve transparency in the diagnostic process.

Bedside Clinicians' Perceptions on the Contributing Role of Diagnostic Errors in Acutely Ill Patient Presentation: A Survey of Academic and Community Practice

OBJECTIVES

This study aimed to explore clinicians' perceptions of the occurrence of and factors associated with diagnostic errors in patients evaluated during a rapid response team (RRT) activation or unplanned admission to the intensive care unit (ICU).

METHODS

A multicenter prospective survey study was conducted among multiprofessional clinicians involved in the care of patients with RRT activations and/or unplanned ICU admissions (UIAs) at 2 academic hospitals and 1 community-based hospital between April 2019 and March 2020. A study investigator screened eligible patients every day. Within 24 hours of the event, a research coordinator administered the survey to clinicians, who were asked the following: whether diagnostic errors contributed to the reason for RRT/UIA, whether any new diagnosis was made after RRT/UIA, if there were any failures to communicate the diagnosis, and if involvement of specialists earlier would have benefited that patient. Patient clinical data were extracted from the electronic health record.

RESULTS

A total of 1815 patients experienced RRT activations, and 1024 patients experienced UIA. Clinicians reported that 18.2% (95/522) of patients experienced diagnostic errors, 8.0% (42/522) experienced a failure of communication, and 16.7% (87/522) may have benefitted from earlier involvement of specialists. Compared with academic settings, clinicians in the community hospital were less likely to report diagnostic errors (7.0% versus 22.8%, P = 0.002).

CONCLUSIONS

Clinicians report a high rate of diagnostic errors in patients they evaluate during RRT or UIAs.

Diagnostic Error in Pediatrics: A Narrative Review

A priority topic for patient safety research is diagnostic errors. However, despite the significant growth in awareness of their unacceptably high incidence and associated harm, a relative paucity of large, high-quality studies of diagnostic error in pediatrics exists. In this narrative review, we present what is known about the incidence and epidemiology of diagnostic error in pediatrics as well as the established research methods for identifying, evaluating, and reducing diagnostic errors, including their strengths and weaknesses. Additionally, we highlight that pediatric diagnostic error remains an area in need of both innovative research and quality improvement efforts to apply learnings from a rapidly growing evidence base. We propose several key research questions aimed at addressing persistent gaps in the pediatric diagnostic error literature that focus on the foundational knowledge needed to inform effective interventions to reduce the incidence of diagnostic errors and their associated harm. Additional research is needed to better establish the epidemiology of diagnostic error in pediatrics, including identifying high-risk clinical scenarios, patient populations, and groups of diagnoses. A critical need exists for validated measures of both diagnostic errors and diagnostic processes that can be adapted for different clinical settings and standardized for use across varying institutions. Pediatric researchers will need to work collaboratively on large-scale, high-quality studies to accomplish the ultimate goal of reducing diagnostic errors and their associated harm in children by addressing these fundamental gaps in knowledge.

A Research Agenda for Diagnostic Excellence in Critical Care Medicine

Diagnosing critically ill patients in the intensive care unit is difficult. As a result, diagnostic errors in the intensive care unit are common and have been shown to cause harm. Research to improve diagnosis in critical care medicine has accelerated in past years. However, much work remains to fully elucidate the diagnostic process in critical care. To achieve diagnostic excellence, interdisciplinary research is needed, adopting a balanced strategy of continued biomedical discovery while addressing the complex care delivery systems underpinning the diagnosis of critical illness.

Incidence of diagnostic errors in unplanned hospitalized patients using an automated medical history-taking system with differential diagnosis generator: retrospective observational study (Preprint)

Background

Automated medical history–taking systems that generate differential diagnosis lists have been suggested to contribute to improved diagnostic accuracy. However, the effect of these systems on diagnostic errors in clinical practice remains unknown.

Objective

This study aimed to assess the incidence of diagnostic errors in an outpatient department, where an artificial intelligence (AI)–driven automated medical history–taking system that generates differential diagnosis lists was implemented in clinical practice.

Methods

We conducted a retrospective observational study using data from a community hospital in Japan. We included patients aged 20 years and older who used an AI-driven, automated medical history–taking system that generates differential diagnosis lists in the outpatient department of internal medicine for whom the index visit was between July 1, 2019, and June 30, 2020, followed by unplanned hospitalization within 14 days. The primary endpoint was the incidence of diagnostic errors, which were detected using the Revised Safer Dx Instrument by at least two independent reviewers. To evaluate the effect of differential diagnosis lists from the AI system on the incidence of diagnostic errors, we compared the incidence of these errors between a group where the AI system generated the final diagnosis in the differential diagnosis list and a group where the AI system did not generate the final diagnosis in the list; the Fisher exact test was used for comparison between these groups. For cases with confirmed diagnostic errors, further review was conducted to identify the contributing factors of these errors via discussion among three reviewers, using the Safer Dx Process Breakdown Supplement as a reference.

Results

A total of 146 patients were analyzed. A final diagnosis was confirmed for 138 patients and was observed in the differential diagnosis list from the AI system for 69 patients. Diagnostic errors occurred in 16 out of 146 patients (11.0%, 95% CI 6.4%-17.2%). Although statistically insignificant, the incidence of diagnostic errors was lower in cases where the final diagnosis was included in the differential diagnosis list from the AI system than in cases where the final diagnosis was not included in the list (7.2% vs 15.9%, P=.18).

Conclusions

The incidence of diagnostic errors among patients in the outpatient department of internal medicine who used an automated medical history–taking system that generates differential diagnosis lists seemed to be lower than the previously reported incidence of diagnostic errors. This result suggests that the implementation of an automated medical history–taking system that generates differential diagnosis lists could be beneficial for diagnostic safety in the outpatient department of internal medicine.

Diagnostic error in the pediatric hospital: a narrative review

INTRODUCTION

Diagnostic error is a prevalent type of medical error that is associated with considerable patient harm and increased medical costs. The majority of literature guiding the current understanding of diagnostic error in the hospital setting is from adult studies. However, there is research to suggest this type of error is also prevalent in the pediatric specialty.

OBJECTIVES

The primary objective of this study was to define the current understanding of diagnostic error in the pediatric hospital through a structured literature review.

METHODS

We searched PubMed and identified studies focusing on three aspects of diagnostic error in pediatric hospitals: the incidence or prevalence, contributing factors, and related interventions. We used a tiered review, and a standardized electronic form to extract data from included articles.

RESULTS

Fifty-nine abstracts were screened and 23 full-text studies were included in the final review. Seventeen of the 23 studies focused on the incidence or prevalence, with only 3 studies investigating the utility of interventions. Most studies took place in an intensive care unit or emergency department with very few studies including only patients on the general wards. Overall, the prevalence of diagnostic error in pediatric hospitals varied greatly and depended on the measurement technique and specific hospital setting. Both healthcare system factors and individual cognitive factors were found to contribute to diagnostic error, with there being limited evidence to guide how best to mitigate the influence of these factors on the diagnostic process.

CONCLUSION

The general knowledge of diagnostic error in pediatric hospital settings is limited. Future work should incorporate structured frameworks to measure diagnostic errors and examine clinicians' diagnostic processes in real-time to help guide effective hospital-wide interventions.

Diagnostic Error in the Critically Ill: A Hidden Epidemic?

Epidemiologic studies of diagnostic error in the intensive care unit (ICU) consist mostly of descriptive autopsy series. In these studies, rates of diagnostic errors are approximately 5% to 10%. Recently validated methods for retrospectively measuring error have expanded our understanding of the scope of the problem. These alternative measurement strategies have yielded similar estimates for the frequency of diagnostic error in the ICU. Although there is a fair understanding of the frequency of errors, further research is needed to better define the risk factors for diagnostic error in the ICU.

Learning from Missed Opportunities Through Reflective Practice

Identification of diagnostic errors is difficult but is not alone sufficient for performance improvement. Instead, cases must be reflected on to identify ways to improve decision-making in the future. There are many tools and modalities to retrospectively reflect on action to study medical decisions and outcomes and improve future performance. Reflection in action-in which diagnostic decisions are considered in real-time-may also improve medical decision-making especially through strategies such as structured reflection. Ongoing regular feedback can normalize the discussion about improving decision-making, enable reflective practice, and improve decision making.

nalyzing diagnostic errors in the acute setting: a process-driven approach

OBJECTIVES

We describe an approach for analyzing failures in diagnostic processes in a small, enriched cohort of general medicine patients who expired during hospitalization and experienced medical error. Our objective was to delineate a systematic strategy for identifying frequent and significant failures in the diagnostic process to inform strategies for preventing adverse events due to diagnostic error.

METHODS

Two clinicians independently reviewed detailed records of purposively sampled cases identified from established institutional case review forums and assessed the likelihood of diagnostic error using the Safer Dx instrument. Each reviewer used the modified Diagnostic Error Evaluation and Research (DEER) taxonomy, revised for acute care (41 possible failure points across six process dimensions), to characterize the frequency of failure points (FPs) and significant FPs in the diagnostic process.

RESULTS

Of 166 cases with medical error, 16 were sampled: 13 (81.3%) had one or more diagnostic error(s), and a total of 113 FPs and 30 significant FPs were identified. A majority of significant FPs (63.3%) occurred in "Diagnostic Information and Patient Follow-up" and "Patient and Provider Encounter and Initial Assessment" process dimensions. Fourteen (87.5%) cases had a significant FP in at least one of these dimensions.

CONCLUSIONS

Failures in the diagnostic process occurred across multiple dimensions in our purposively sampled cohort. A systematic analytic approach incorporating the modified DEER taxonomy, revised for acute care, offered critical insights into key failures in the diagnostic process that could serve as potential targets for preventative interventions.

Diagnostic Errors in Pediatric Critical Care: A Systematic Review

OBJECTIVES

To summarize the literature on prevalence, impact, and contributing factors related to diagnostic error in the PICU.

DATA SOURCES

Search of PubMed, EMBASE, and the Cochrane Library up to December 2019.

STUDY SELECTION

Studies on diagnostic error and the diagnostic process in pediatric critical care were included. Non-English studies with no translation, case reports/series, studies providing no information on diagnostic error, studies focused on non-PICU populations, and studies focused on a single condition/disease or a single diagnostic test/tool were excluded.

DATA EXTRACTION

Data on research design, objectives, study sample, and results pertaining to the prevalence, impact, and factors associated with diagnostic error were abstracted from each study.

DATA SYNTHESIS

Using independent tiered review, 396 abstracts were screened, and 17 studies (14 full-text, 3 abstracts) were ultimately included. Fifteen of 17 studies (88%) had an observational research design. Autopsy studies (autopsy rates were 20-47%) showed a 10-23% rate of missed major diagnoses; 5-16% of autopsy-discovered diagnostic errors had a potential adverse impact on survival and would have changed management. Retrospective record reviews reported varying rates of diagnostic error from 8% in a general PICU population to 12% among unexpected critical admissions and 21-25% of patients discussed at PICU morbidity and mortality conferences. Cardiovascular, infectious, congenital, and neurologic conditions were most commonly misdiagnosed. Systems factors (40-67%), cognitive factors (20-3%), and both systems and cognitive factors (40%) were associated with diagnostic error. Limited information was available on the impact of misdiagnosis.

CONCLUSIONS

Knowledge of diagnostic errors in the PICU is limited. Future work to understand diagnostic errors should involve a balanced focus between studying the diagnosis of individual diseases and uncovering common system- and process-related determinants of diagnostic error.

Safer Stroke-Dx Instrument: Identifying Stroke Misdiagnosis in the Emergency Department

BACKGROUND

Missed or delayed diagnosis of acute stroke, or false-negative stroke (FNS), at initial emergency department (ED) presentation occurs in ≈9% of confirmed stroke patients. Failure to rapidly diagnose stroke can preclude time-sensitive treatments, resulting in higher risks of severe sequelae and disability. In this study, we developed and tested a modified version of a structured medical record review tool, the Safer Dx Instrument, to identify FNS in a subgroup of hospitalized patients with stroke to gain insight into sources of ED stroke misdiagnosis.

METHODS

We conducted a retrospective cohort study at 2 unaffiliated comprehensive stroke centers. In the development and confirmatory cohorts, we applied the Safer Stroke-Dx Instrument to report the prevalence and documented sources of ED diagnostic error in FNS cases among confirmed stroke patients upon whom an acute stroke was suspected by the inpatient team, as evidenced by stroke code activation or urgent neurological consultation, but not by the ED team. Inter-rater reliability and agreement were assessed using interclass coefficient and kappa values (κ).

RESULTS

Among 183 cases in the development cohort, the prevalence of FNS was 20.2% (95% CI, 15.0-26.7). Too narrow a differential diagnosis and limited neurological examination were common potential sources of error. The interclass coefficient for the Safer Stroke-Dx Instrument items ranged from 0.42 to 0.91, and items were highly correlated with each other. The κ for diagnostic error identification was 0.90 (95% CI, 0.821-0.978) using the Safer Stroke-Dx Instrument. In the confirmatory cohort of 99 cases, the prevalence of FNS was 21.2% (95% CI, 14.2-30.3) with similar sources of diagnostic error identified.

CONCLUSIONS

Hospitalized patients identified by stroke codes and requests for urgent neurological consultation represent an enriched population for the study of diagnostic error in the ED. The Safer Stroke-Dx Instrument is a reliable tool for identifying FNS and sources of diagnostic error.

Increasing Physician Reporting of Diagnostic Learning Opportunities

BACKGROUND

An estimated 10% of Americans experience a diagnostic error annually, yet little is known about pediatric diagnostic errors. Physician reporting is a promising method for identifying diagnostic errors. However, our pediatric hospital medicine (PHM) division had only 1 diagnostic-related safety report in the preceding 4 years. We aimed to improve attending physician reporting of suspected diagnostic errors from 0 to 2 per 100 PHM patient admissions within 6 months.

METHODS

Our improvement team used the Model for Improvement, targeting the PHM service. To promote a safe reporting culture, we used the term diagnostic learning opportunity (DLO) rather than diagnostic error, defined as a "potential opportunity to make a better or more timely diagnosis." We developed an electronic reporting form and encouraged its use through reminders, scheduled reflection time, and monthly progress reports. The outcome measure, the number of DLO reports per 100 patient admissions, was tracked on an annotated control chart to assess the effect of our interventions over time. We evaluated DLOs using a formal 2-reviewer process.

RESULTS

Over the course of 13 weeks, there was an increase in the number of reports filed from 0 to 1.6 per 100 patient admissions, which met special cause variation, and was subsequently sustained. Most events (66%) were true diagnostic errors and were found to be multifactorial after formal review.

CONCLUSIONS

We used quality improvement methodology, focusing on psychological safety, to increase physician reporting of DLOs. This growing data set has generated nuanced learnings that will guide future improvement work.

Handshake antimicrobial stewardship as a model to recognize and prevent diagnostic errors

OBJECTIVES

Few studies describe the impact of antimicrobial stewardship programs (ASPs) on recognizing and preventing diagnostic errors. Handshake stewardship (HS-ASP) is a novel ASP model that prospectively reviews hospital-wide antimicrobial usage with recommendations made in person to treatment teams. The purpose of this study was to determine if HS-ASP could identify and intervene on potential diagnostic errors for children hospitalized at a quaternary care children's hospital.

METHODS

Previously self-identified "Great Catch" (GC) interventions by the Children's Hospital Colorado HS-ASP team from 10/2014 through 5/2018 were retrospectively reviewed. Each GC was categorized based on the types of recommendations from HS-ASP, including if any diagnostic recommendations were made to the treatment team. Each GC was independently scored using the "Safer Dx Instrument" to determine presence of diagnostic error based on a previously determined cut-off score of ≤1.50. Interrater reliability for the instrument was measured using a randomized subset of one third of GCs.

RESULTS

During the study period, there were 162 GC interventions. Of these, 65 (40%) included diagnostic recommendations by HS-ASP and 19 (12%) had a Safer Dx Score of ≤1.50, (Κ=0.44; moderate agreement). Of those GCs associated with diagnostic errors, the HS-ASP team made a diagnostic recommendation to the primary treatment team 95% of the time.

CONCLUSIONS

Handshake stewardship has the potential to identify and intervene on diagnostic errors for hospitalized children.

Operational measurement of diagnostic safety: state of the science

Reducing the incidence of diagnostic errors is increasingly a priority for government, professional, and philanthropic organizations. Several obstacles to measurement of diagnostic safety have hampered progress toward this goal. Although a coordinated national strategy to measure diagnostic safety remains an aspirational goal, recent research has yielded practical guidance for healthcare organizations to start using measurement to enhance diagnostic safety. This paper, concurrently published as an Issue Brief by the Agency for Healthcare Research and Quality, issues a "call to action" for healthcare organizations to begin measurement efforts using data sources currently available to them. Our aims are to outline the state of the science and provide practical recommendations for organizations to start identifying and learning from diagnostic errors. Whether by strategically leveraging current resources or building additional capacity for data gathering, nearly all organizations can begin their journeys to measure and reduce preventable diagnostic harm.

Identifying psychiatric diagnostic errors with the Safer Dx Instrument

OBJECTIVE

Diagnostic errors in psychiatry are understudied partly because they are difficult to measure. The current study aimed to adapt and test the Safer Dx Instrument, a structured tool to review electronic health records (EHR) for errors in medical diagnoses, to evaluate errors in anxiety diagnoses to improve measurement of psychiatric diagnostic errors.

DESIGN

The iterative adaptation process included a review of the revised Safer Dx-Mental Health Instrument by mental health providers to ensure content and face validity and review by a psychometrician to ensure methodologic validity and pilot testing of the revised instrument.

SETTINGS

None.

PARTICIPANTS

Pilot testing was conducted on 128 records of patients diagnosed with anxiety in integrated primary care mental health clinics. Cases with anxiety diagnoses documented in progress notes but not included as a diagnosis for the encounter (n = 25) were excluded.

INTERVENTION(S)

None.

MAIN OUTCOME MEASURE(S)

None.

RESULTS

Of 103 records meeting the inclusion criteria, 62 likely involved a diagnostic error (42 from use of unspecified anxiety diagnosis when a specific anxiety diagnosis was warranted; 20 from use of unspecified anxiety diagnosis when anxiety symptoms were either undocumented or documented but not severe enough to warrant diagnosis). Reviewer agreement on presence/absence of errors was 88% (κ = 0.71).

CONCLUSION

The revised Safer Dx-Mental Health Instrument has a high reliability for detecting anxiety-related diagnostic errors and deserves testing in additional psychiatric populations and clinical settings.

Frequency, Risk Factors, Causes, and Consequences of Diagnostic Errors in Critically Ill Medical Patients: A Retrospective Cohort Study

OBJECTIVE

Diagnostic errors are a source of significant morbidity and mortality but understudied in the critically ill. We sought to characterize the frequency, causes, consequences, and risk factors of diagnostic errors among unplanned ICU admissions.

DESIGN

We conducted a retrospective cohort study of randomly selected nonsurgical ICU admissions between July 2015 and June 2016.

SETTING

Medical ICU at a tertiary academic medical center.

SUBJECTS

Critically ill adults with unplanned admission to the medical ICU.

MEASUREMENTS AND MAIN RESULTS

The primary investigator reviewed patient records using a modified version of the Safer Dx instrument, a validated instrument for detecting diagnostic error. Two intensivists performed secondary reviews of possible errors, and reviewers met periodically to adjudicate errors by consensus. For each confirmed error, we judged harm on a 1-6 rating scale. We also collected detailed demographic and clinical data for each patient. We analyzed 256 unplanned ICU admissions and identified 18 diagnostic errors (7% of admissions). All errors were associated with harm, and only six errors (33%) were recognized by the ICU team within the first 24 hours. More women than men experienced a diagnostic error (11.7% vs 2.7%; p = 0.015, χ test). On multivariable logistic regression analysis, female sex remained independently associated with risk of diagnostic error both at admission (odds ratio, 5.18; 95% CI, 1.34-20.08) and at 24 hours (odds ratio, 11.6; 95% CI, 1.37-98.6). Similarly, Quick Sequential Organ Failure Assessment score greater than or equal to 2 at admission was independently associated with diagnostic error (odds ratio, 5.73; 95% CI, 1.72-19.01).

CONCLUSIONS

Diagnostic errors may be an underappreciated source of ICU-related harm. Women and higher acuity patients appear to be at increased risk for such errors. Further research is merited to define the scope of error-associated harm and to clarify risk factors for diagnostic errors among the critically ill.

Factors Associated With Diagnostic Error on Admission to a PICU: A Pilot Study

OBJECTIVES

Diagnostic errors can harm critically ill children. However, we know little about their prevalence in PICUs and factors associated with error. The objective of this pilot study was to determine feasibility of record review to identify patient, provider, and work system factors associated with diagnostic errors during the first 12 hours after PICU admission.

DESIGN

Pilot retrospective cohort study with structured record review using a structured tool (Safer Dx instrument) to identify diagnostic error.

SETTING

Academic tertiary referral PICU.

PATIENTS

Patients 0-17 years old admitted nonelectively to the PICU.

INTERVENTIONS

None.

MEASUREMENTS AND MAIN RESULTS

Four of 50 patients (8%) had diagnostic errors in the first 12 hours after admission. The Safer Dx instrument helped identify delayed diagnoses of chronic ear infection, increased intracranial pressure (two cases), and Bartonella encephalitis. We calculated that 610 PICU admissions are needed to achieve 80% power (α = 0.05) to detect significant associations with error.

CONCLUSIONS

Our pilot study found four patients with diagnostic error out of 50 children admitted nonelectively to a PICU. Retrospective record review using a structured tool to identify diagnostic errors is feasible in this population. Pilot data are being used to inform a larger and more definitive multicenter study.

Assessing diagnostic error in cerebral venous thrombosis via detailed chart review

Background Diagnostic error in cerebral venous thrombosis (CVT) has been understudied despite the harm associated with misdiagnosis of other cerebrovascular diseases as well as the known challenges of evaluating non-specific neurological symptoms in clinical practice. Methods We conducted a retrospective cohort study of CVT patients hospitalized at a single center. Two independent reviewers used a medical record review tool, the Safer Dx Instrument, to identify diagnostic errors. Demographic and clinical factors were abstracted. We compared subjects with and without a diagnostic error using the t-test for continuous variables and the chi-square (χ2) test or Fisher's exact test for categorical variables; an alpha of 0.05 was the cutoff for significance. Results A total of 72 CVT patients initially met study inclusion criteria; 19 were excluded due to incomplete medical records. Of the 53 patients included in the final analysis, the mean age was 48 years and 32 (60.4%) were women. Diagnostic error occurred in 11 cases [20.8%; 95% confidence interval (CI) 11.8-33.6%]. Subjects with diagnostic errors were younger (42 vs. 49 years, p = 0.13), more often women (81.8% vs. 54.8%, p = 0.17), and were significantly more likely to have a past medical history of a headache disorder prior to the index CVT visit (7.1% vs. 36.4%, p = 0.03). Conclusions Nearly one in five patients with complete medical records experienced a diagnostic error. Prior history of headache was the only evaluated clinical factor that was more common among those with an error in diagnosis. Future work on distinguishing primary from secondary headaches to improve diagnostic accuracy in acute neurological disease is warranted.

Development and Initial Validation of a Frontline Health Worker mHealth Assessment Platform (MEDSINC®) for Children 2-60 Months of Age

Approximately 3 million children younger than 5 years living in low- and middle-income countries (LMICs) die each year from treatable clinical conditions such as pneumonia, dehydration secondary to diarrhea, and malaria. A majority of these deaths could be prevented with early clinical assessments and appropriate therapeutic intervention. In this study, we describe the development and initial validation testing of a mobile health (mHealth) platform, MEDSINC^®, designed for frontline health workers (FLWs) to perform clinical risk assessments of children aged 2–60 months. MEDSINC is a web browser–based clinical severity assessment, triage, treatment, and follow-up recommendation platform developed with physician-based Bayesian pattern recognition logic. Initial validation, usability, and acceptability testing were performed on 861 children aged between 2 and 60 months by 49 FLWs in Burkina Faso, Ecuador, and Bangladesh. MEDSINC-based clinical assessments by FLWs were independently and blindly correlated with clinical assessments by 22 local health-care professionals (LHPs). Results demonstrate that clinical assessments by FLWs using MEDSINC had a specificity correlation between 84% and 99% to LHPs, except for two outlier assessments (63% and 75%) at one study site, in which local survey prevalence data indicated that MEDSINC outperformed LHPs. In addition, MEDSINC triage recommendation distributions were highly correlated with those of LHPs, whereas usability and feasibility responses from LHP/FLW were collectively positive for ease of use, learning, and job performance. These results indicate that the MEDSINC platform could significantly increase pediatric health-care capacity in LMICs by improving FLWs’ ability to accurately assess health status and triage of children, facilitating early life-saving therapeutic interventions.

Diagnostic errors in the neonatal intensive care unit: State of the science and new directions

Diagnostic errors remain understudied in neonatology. The limited available evidence, however, suggests that diagnostic errors in the neonatal intensive care unit (NICU) result in significant and long-term consequences. In this narrative review, we discuss how the concept of diagnostic errors framed as missed opportunities can be applied to the non-linear nature of diagnosis in a critical care environment such as the NICU. We then explore how the etiology of an error in diagnosis can be related to both individual cognitive factors as well as organizational and systemic factors - all of which often contribute to the error. This multifactorial causation has limited the development of methodology to measure diagnostic errors as well as strategies to mitigate and prevent their adverse effects. We recommend research focused on the frequency and etiology of diagnostic error in the NICU as well as potential mitigation strategies to advance this important field in neonatal intensive care.

Recommendations for using the Revised Safer Dx Instrument to help measure and improve diagnostic safety

The medical record continues to be one of the most useful and accessible sources of information to examine the diagnostic process. However, medical record review studies of diagnostic errors have often used subjective judgments and found low inter-rater agreement among reviewers when determining the presence or absence of diagnostic error. In our previous work, we developed a structured data-collection instrument, called the Safer Dx Instrument, consisting of objective criteria to improve the accuracy of assessing diagnostic errors in primary care. This paper proposes recommendations on how clinicians and health care organizations could use the Revised Safer Dx Instrument in identifying and understanding missed opportunities to make correct and timely diagnoses. The instrument revisions addressed both methodological and implementation issues identified during initial use and included refinements to the instrument to allow broader application across all health care settings. In addition to leveraging knowledge from piloting the instrument in several health care settings, we gained insights from multiple researchers who had used the instrument in studies involving emergency care, inpatient care and intensive care unit settings. This allowed us to enhance and extend the scope of this previously validated data collection instrument. In this paper, we describe the refinement process and provide recommendations for application and use of the Revised Safer Dx Instrument across a broad range of health care settings. The instrument can help users identify potential diagnostic errors in a standardized way for further analysis and safety improvement efforts as well as provide data for clinician feedback and reflection. With wider adoption and use by clinicians and health systems, the Revised Safer Dx Instrument could help propel the science of measuring and reducing diagnostic errors forward.

Missed Diagnosis of New-Onset Systolic Heart Failure at First Presentation in Children with No Known Heart Disease

OBJECTIVE

To determine frequency of missed heart failure diagnosis at first presentation among children with no known heart disease admitted with new-onset heart failure.

STUDY DESIGN

Using a retrospective design, we reviewed electronic medical records of all patients aged <21 years with no known heart disease, hospitalized with new-onset heart failure during 2003-2015 at a tertiary-quaternary care institution. We assessed records for missed diagnosis of heart failure (primary outcome), associated process breakdowns, and clinical outcomes using a structured data collection instrument.

RESULTS

Of 191 patients meeting inclusion criteria, 49% (94/191) were missed on first presentation. Most common incorrect diagnostic labels given to "missed" patients were bacterial infection (29%; 27/94), followed by viral illness (22%; 21/94) and gastroenteritis/hepatitis (21%; 20/94). On multivariable analysis, presentation to primary care provider (PCP), longer duration of symptoms (median 7 days), more than 2 symptoms of heart failure, and nausea/emesis were associated with missed diagnosis. On examining process breakdowns, 49% had errors in history-taking and 50% had no documentation of differential diagnoses. There was no difference in hospital mortality, length of stay, or mechanical circulatory support in missed vs not-missed cohorts. Unnecessary noninvasive and invasive tests were performed in 18% and 4% of patients, respectively.

CONCLUSIONS

Nearly one-half of children with no known heart disease hospitalized with systolic heart failure were missed at first presentation and underwent significant nonrelevant treatment and testing. Initial presentation to the PCP, longer duration of symptoms before presentation, and nausea/emesis were associated with missed diagnosis.

Tracking Progress in Improving Diagnosis: A Framework for Defining Undesirable Diagnostic Events

Diagnostic error is a prevalent, harmful, and costly phenomenon. Multiple national health care and governmental organizations have recently identified the need to improve diagnostic safety as a high priority. A major barrier, however, is the lack of standardized, reliable methods for measuring diagnostic safety. Given the absence of reliable and valid measures for diagnostic errors, we need methods to help establish some type of baseline diagnostic performance across health systems, as well as to enable researchers and health systems to determine the impact of interventions for improving the diagnostic process. Multiple approaches have been suggested but none widely adopted. We propose a new framework for identifying "undesirable diagnostic events" (UDEs) that health systems, professional organizations, and researchers could further define and develop to enable standardized measurement and reporting related to diagnostic safety. We propose an outline for UDEs that identifies both conditions prone to diagnostic error and the contexts of care in which these errors are likely to occur. Refinement and adoption of this framework across health systems can facilitate standardized measurement and reporting of diagnostic safety.

Diagnostic errors in paediatric cardiac intensive care

IntroductionDiagnostic errors cause significant patient harm and increase costs. Data characterising such errors in the paediatric cardiac intensive care population are limited. We sought to understand the perceived frequency and types of diagnostic errors in the paediatric cardiac ICU.

METHODS

Paediatric cardiac ICU practitioners including attending and trainee physicians, nurse practitioners, physician assistants, and registered nurses at three North American tertiary cardiac centres were surveyed between October 2014 and January 2015.

RESULTS

The response rate was 46% (N=200). Most respondents (81%) perceived that diagnostic errors harm patients more than five times per year. More than half (65%) reported that errors permanently harm patients, and up to 18% perceived that diagnostic errors contributed to death or severe permanent harm more than five times per year. Medication side effects and psychiatric conditions were thought to be most commonly misdiagnosed. Physician groups also ranked pulmonary overcirculation and viral illness to be commonly misdiagnosed as bacterial illness. Inadequate care coordination, data assessment, and high clinician workload were cited as contributory factors. Delayed diagnostic studies and interventions related to the severity of the patient's condition were thought to be the most commonly reported process breakdowns. All surveyed groups ranked improving teamwork and feedback pathways as strategies to explore for preventing future diagnostic errors.

CONCLUSIONS

Paediatric cardiac intensive care practitioners perceive that diagnostic errors causing permanent harm are common and associated more with systematic and process breakdowns than with cognitive limitations.

Implementing paediatric early warning scores systems in the Netherlands: future implications

Background

Paediatric Early Warning Scores (PEWS) are increasingly being used for early identification and management of clinical deterioration in paediatric patients. A PEWS system includes scores, cut-off points and appropriate early intervention. In 2011, The Dutch Ministry of Health advised hospitals to implement a PEWS system in order to improve patient safety in paediatric wards. The objective of this study was to examine the results of implementation of PEWS systems and to gain insight into the attitudes of professionals towards using a PEWS system in Dutch non-university hospitals.

Methods

Quantitative data were gathered at start, midway and at the end of the implementation period through retrospective patient record review (n = 554). Semi-structured interviews with professionals (n = 8) were used to gain insight in the implementation process and experiences. The interviews were transcribed and analysed using an inductive approach.

Results

Looking at PEWS systems of the five participating hospitals, different parameters and policies were found. While all hospitals included heart rate and respiratory rate, other variables differed among hospitals. At baseline, none of the hospitals used a PEWS system. After 1 year, PEWS were recorded in 69.2% of the patient records and elevated PEWS resulted in appropriate action in 49.1%. Three themes emerged from the interviews: 1) while the importance of using a PEWS system was acknowledged, professionals voiced some doubts about the effectiveness and validity of their PEWS system 2) registering PEWS required little extra effort and was facilitated by PEWS being integrated into the electronic patient record 3) Without a national PEWS system or guidelines, hospitals found it difficult to identify a suitable PEWS system for their setting. Existing systems were not always considered applicable in a non-university setting.

Conclusions

After 1 year, hospitals showed improvements in the use of their PEWS system, although some were decidedly more successful than others. Doubts among staff about validity, effectiveness and communication with other hospitals during transfer to higher level care hospital might hinder sustainable implementation. For these purposes the development of a national PEWS system is recommended, consisting of a “core set” of PEWS, cut-off points and associated early intervention.