Diagnostic error in mental health: a review

Diagnostic errors are associated with patient harm and suboptimal outcomes. Despite national scientific efforts to advance definition, measurement and interventions for diagnostic error, diagnosis in mental health is not well represented in this ongoing work. We aimed to summarise the current state of research on diagnostic errors in mental health and identify opportunities to align future research with the emerging science of diagnostic safety. We review conceptual considerations for defining and measuring diagnostic error, the application of these concepts to mental health settings, and the methods and subject matter focus of recent studies of diagnostic error in mental health. We found that diagnostic error is well understood to be a problem in mental healthcare. Although few studies used clear definitions or frameworks for understanding diagnostic error in mental health, several studies of missed, wrong, delayed and disparate diagnosis of common mental disorders have identified various avenues for future research and development. Nevertheless, a lack of clear consensus on how to conceptualise, define and measure errors in diagnosis will pose a barrier to advancement. Further research should focus on identifying preventable missed opportunities in the diagnosis of mental disorders, which may uncover generalisable opportunities for improvement.

Medical Home Implementation and Follow-Up of Cancer-Related Abnormal Test Results in the Veterans Health Administration

Importance

Lack of timely follow-up of cancer-related abnormal test results can lead to delayed or missed diagnoses, adverse cancer outcomes, and substantial cost burden for patients. Care delivery models, such as the Veterans Affairs' (VA) Patient-Aligned Care Team (PACT), which aim to improve patient-centered care coordination, could potentially also improve timely follow-up of abnormal test results. PACT was implemented nationally in the VA between 2010 and 2012.

Objective

To evaluate the long-term association between PACT implementation and timely follow-up of abnormal test results related to the diagnosis of 5 different cancers.

Design, Setting, and Participants

This multiyear retrospective cohort study used 14 years of VA data (2006-2019), which were analyzed using panel data-based random-effects linear regressions. The setting included all VA clinics and facilities. The participants were adult patients who underwent diagnostic testing related to 5 different cancers and had abnormal test results. Data extraction and statistical analyses were performed from September 2021 to December 2023.

Exposure

Calendar years denoting preperiods and postperiods of PACT implementation, and the PACT Implementation Progress Index Score denoting the extent of implementation in each VA clinic and facility.

Main Outcome and Measure

Percentage of potentially missed timely follow-ups of abnormal test results.

Results

This study analyzed 6 data sets representing 5 different types of cancers. During the initial years of PACT implementation (2010 to 2013), percentage of potentially missed timely follow-ups decreased between 3 to 7 percentage points for urinalysis suggestive of bladder cancer, 12 to 14 percentage points for mammograms suggestive of breast cancer, 19 to 22 percentage points for fecal tests suggestive of colorectal cancer, and 6 to 13 percentage points for iron deficiency anemia laboratory tests suggestive of colorectal cancer, with no statistically significant changes for α-fetoprotien tests and lung cancer imaging. However, these beneficial reductions were not sustained over time. Better PACT implementation scores were associated with a decrease in potentially missed timely follow-up percentages for urinalysis (0.3-percentage point reduction [95% CI, -0.6 to -0.1] with 1-point increase in the score), and laboratory tests suggestive of iron deficiency anemia (0.5-percentage point reduction [95% CI,-0.8 to -0.2] with 1-point increase in the score).

Conclusions and Relevance

This cohort study found that implementation of PACT in the VA was associated with a potential short-term improvement in the quality of follow-up for certain test results. Additional multifaceted sustained interventions to reduce missed test results are required to prevent care delays.

Managing Interruptions to Improve Diagnostic Decision-Making: Strategies and Recommended Research Agenda

Interruptions are an inevitable occurrence in health care. Interruptions in diagnostic decision-making are no exception and can have negative consequences on both the decision-making process and well-being of the decision-maker. This may result in inaccurate or delayed diagnoses. To date, research specific to interruptions on diagnostic decision-making has been limited, but strategies to help manage the negative impacts of interruptions need to be developed and implemented. In this perspective, we first present a modified model of interruptions to visualize the interruption process and illustrate where potential interventions can be implemented. We then consider several empirically tested strategies from the fields of health care and cognitive psychology that can lay the groundwork for additional research to mitigate effects of interruptions during diagnostic decision-making. We highlight strategies to minimize the negative impacts of interruptions as well as strategies to prevent interruptions altogether. Additionally, we build upon these strategies to propose specific research priorities within the field of diagnostic safety. Identifying effective interventions to help clinicians better manage interruptions has the potential to minimize diagnostic errors and improve patient outcomes.

Electronic Health Records' Support for Primary Care Physicians' Situation Awareness: A Metanarrative Review

OBJECTIVE

Situation awareness (SA) refers to people's perception and understanding of their dynamic environment. In primary care, reduced SA among physicians increases errors in clinical decision-making and, correspondingly, patients' risk of experiencing adverse outcomes. Our objective was to understand the extent to which electronic health records (EHRs) support primary care physicians (PCPs)' SA during clinical decision-making.

METHOD

We conducted a metanarrative review of papers in selected academic databases, including CINAHL and MEDLINE. Eligible studies included original peer-reviewed research published between January 2012 and August 2020 on PCP-EHR interactions. We iteratively queried, screened, and summarized literature focused on EHRs supporting PCPs' clinical decision-making and care management for adults. Then, we mapped findings to an established SA framework to classify external factors (individual, task, and system) affecting PCPs' levels of SA (1-Perception, 2-Comprehension, and 3-Projection) and identified SA barriers.

RESULTS

From 1504 articles identified, we included and synthesized 19 studies. Study designs were largely noninterventional. Studies described EHR workflow misalignments, usability issues, and communication challenges. EHR information, including lab results and care plans, was characterized as incomplete, untimely, or irrelevant. Unmet information needs made it difficult for PCPs to obtain even basic SA, Level 1 SA. Prevalent barriers to PCPs developing SA with EHRs were errant mental models, attentional tunneling, and data overload.

CONCLUSION

Based on our review, EHRs do not support the development of higher levels of SA among PCPs. Review findings suggest SA-oriented design processes for health information technology could improve PCPs' SA, satisfaction, and decision-making.

Cancer Evaluations During the COVID-19 Pandemic: An Observational Study Using National Veterans Affairs Data

INTRODUCTION

Fewer cancer diagnoses have been made during the COVID-19 pandemic. Pandemic-related delays in cancer diagnosis could occur from limited access to care or patient evaluation delays (e.g., delayed testing after abnormal results). Follow-up of abnormal test results warranting evaluation for cancer was examined before and during the pandemic.

METHODS

Electronic trigger algorithms were applied to the Department of Veterans Affairs electronic health record data to assess follow-up of abnormal test results before (March 10, 2019-March 7, 2020) and during (March 8, 2020-March 6, 2021) the pandemic.

RESULTS

Electronic triggers were applied to 8,021,406 veterans' electronic health records to identify follow-up delays for abnormal results warranting evaluation for 5 cancers: bladder (urinalysis with high-grade hematuria), breast (abnormal mammograms), colorectal (positive fecal occult blood tests/fecal immunochemical tests or results consistent with iron deficiency anemia), liver (elevated alpha-fetoprotein), and lung (chest imaging suggestive of malignancy) cancers. Between prepandemic and pandemic periods, test quantities decreased by 12.6%-27.8%, and proportions of abnormal results lacking follow-up decreased for urinalyses (-0.8%), increased for fecal occult blood tests/fecal immunochemical test (+2.3%) and chest imaging (+1.8%), and remained constant for others. Follow-up times decreased for most tests; however, control charts suggested increased delays at 2 stages: early (pandemic beginning) for urinalyses, mammograms, fecal occult blood tests/fecal immunochemical test, iron deficiency anemia, and chest imaging and late (30-45 weeks into pandemic) for mammograms, fecal occult blood tests/fecal immunochemical test, and iron deficiency anemia.

CONCLUSIONS

Although early pandemic delays in follow-up may have led to reduced cancer rates, the significant decrease in tests performed is likely a large driver of these reductions. Future emergency preparedness efforts should bolster essential follow-up and testing procedures to facilitate timely cancer diagnosis.

Electronic Co-design (ECO-design) Workshop for Increasing Clinician Participation in the Design of Health Services Interventions: Participatory Design Approach

BACKGROUND

Participation from clinician stakeholders can improve the design and implementation of health care interventions. Participatory design methods, especially co-design methods, comprise stakeholder-led design activities that are time-consuming. Competing work demands and increasing workloads make clinicians' commitments to typical participatory methods even harder. The COVID-19 pandemic further exacerbated barriers to clinician participation in such interventions.

OBJECTIVE

The aim of this study was to explore a web-based participatory design approach to conduct economical, electronic co-design (ECO-design) workshops with primary care clinicians.

METHODS

We adapted traditional in-person co-design workshops to web-based delivery and adapted co-design workshop series to fit within a single 1-hour session. We applied the ECO-design workshop approach to codevelop feedback interventions regarding abnormal test result follow-up in primary care. We conducted ECO-design workshops with primary care clinicians at a medical center in Southern Texas, using videoconferencing software. Each workshop focused on one of three types of feedback interventions: conversation guide, email template, and dashboard prototype. We paired electronic materials and software features to facilitate participant interactions, prototyping, and data collection. The workshop protocol included four main activities: problem identification, solution generation, prototyping, and debriefing. Two facilitators were assigned to each workshop and one researcher resolved technical problems. After the workshops, our research team met to debrief and evaluate workshops.

RESULTS

A total of 28 primary care clinicians participated in our ECO-design workshops. We completed 4 parallel workshops, each with 5-10 participants. We conducted traditional analyses and generated a clinician persona (ie, representative description) and user interface prototypes. We also formulated recommendations for future ECO-design workshop recruitment, technology, facilitation, and data collection. Overall, our adapted workshops successfully enabled primary care clinicians to participate without increasing their workload, even during a pandemic.

CONCLUSIONS

ECO-design workshops are viable, economical alternatives to traditional approaches. This approach fills a need for efficient methods to involve busy clinicians in the design of health care interventions.

Adherence to National Guidelines for Timeliness of Test Results Communication to Patients in the Veterans Affairs Health Care System

This cross-sectional study assesses policy adherence to national guidelines for timeliness of test results communication to patients in the Department of Veteran Affairs health care system.

Patient and clinician experiences of uncertainty in the diagnostic process: Current understanding and future directions

OBJECTIVE

Uncertainty occurs throughout the diagnostic process and must be managed to facilitate accurate and timely diagnoses and treatments. Better characterization of uncertainty can inform strategies to manage it more effectively in clinical practice. We provide a comprehensive overview of current literature on diagnosis-related uncertainty describing (1) where patients and clinicians experience uncertainty within the diagnostic process, (2) how uncertainty affects the diagnostic process, (3) roots of uncertainty related to probability/risk, ambiguity, or complexity, and (4) strategies to manage uncertainty.

DISCUSSION

Each diagnostic process step involves uncertainty, including patient engagement with the healthcare system; information gathering, interpretation, and integration; formulating working diagnoses; and communicating diagnoses to patients. General management strategies include acknowledging uncertainty, obtaining more contextual information from patients (e.g., gathering occupations and family histories), creating diagnostic safety nets (e.g., informing patients what red flags to look for), engaging in worst case/best case scenario planning, and communicating diagnostic uncertainty to patients, families, and colleagues. Potential strategies tailored to various aspects of diagnostic uncertainty are also outlined.

CONCLUSION

Scientific knowledge on diagnostic uncertainty, while previously elusive, is now becoming more clearly defined. Next steps include research to evaluate relationships between management and communication of diagnostic uncertainty and improved patient outcomes.

A Program to Provide Clinicians with Feedback on Their Diagnostic Performance in a Learning Health System

PROBLEM

Reducing diagnostic errors requires improving both systems and individual clinical reasoning. One strategy to achieve diagnostic excellence is learning from feedback. However, clinicians remain uncomfortable receiving feedback on their diagnostic performance. Thus, a team of researchers and clinical leaders aimed to develop and implement a diagnostic performance feedback program for learning that mitigates potential clinician discomfort.

APPROACH

The program was developed as part of a larger project to create a learning health system around diagnostic safety at Geisinger, a large, integrated health care system in rural Pennsylvania. Steps included identifying potential missed opportunities in diagnosis (MODs) from various sources (for example, risk management, clinician reports, patient complaints); confirming MODs through chart review; and having trained facilitators provide feedback to clinicians about MODs as learning opportunities. The team developed a guide for facilitators to conduct effective diagnostic feedback sessions and surveyed facilitators and recipients about their experiences and perceptions of the feedback sessions.

OUTCOMES

28 feedback sessions occurred from January 2019 to June 2020, involving MODs from emergency medicine, primary care, and hospital medicine. Most facilitators (90.6% [29/32]) reported that recipients were receptive to learning and discussing MODs. Most recipients reported that conversations were constructive and nonpunitive (83.3% [25/30]) and allowed them to take concrete steps toward improving diagnosis (76.7% [23/30]). Both groups believed discussions would improve future diagnostic safety (93.8% [30/32] and 70.0% [21/30], respectively).

KEY INSIGHTS AND NEXT STEPS

An institutional program was developed and implemented to deliver diagnostic performance feedback. Such a program may facilitate learning and improvement to reduce MODs. Future efforts should assess long-term effects on diagnostic performance and patient outcomes.

Patient Perspectives on the Usefulness of an Artificial Intelligence-Assisted Symptom Checker: Cross-Sectional Survey Study

Background

Patients are increasingly seeking Web-based symptom checkers to obtain diagnoses. However, little is known about the characteristics of the patients who use these resources, their rationale for use, and whether they find them accurate and useful.

Objective

The study aimed to examine patients’ experiences using an artificial intelligence (AI)–assisted online symptom checker.

Methods

An online survey was administered between March 2, 2018, through March 15, 2018, to US users of the Isabel Symptom Checker within 6 months of their use. User characteristics, experiences of symptom checker use, experiences discussing results with physicians, and prior personal history of experiencing a diagnostic error were collected.

Results

A total of 329 usable responses was obtained. The mean respondent age was 48.0 (SD 16.7) years; most were women (230/304, 75.7%) and white (271/304, 89.1%). Patients most commonly used the symptom checker to better understand the causes of their symptoms (232/304, 76.3%), followed by for deciding whether to seek care (101/304, 33.2%) or where (eg, primary or urgent care: 63/304, 20.7%), obtaining medical advice without going to a doctor (48/304, 15.8%), and understanding their diagnoses better (39/304, 12.8%). Most patients reported receiving useful information for their health problems (274/304, 90.1%), with half reporting positive health effects (154/302, 51.0%). Most patients perceived it to be useful as a diagnostic tool (253/301, 84.1%), as a tool providing insights leading them closer to correct diagnoses (231/303, 76.2%), and reported they would use it again (278/304, 91.4%). Patients who discussed findings with their physicians (103/213, 48.4%) more often felt physicians were interested (42/103, 40.8%) than not interested in learning about the tool’s results (24/103, 23.3%) and more often felt physicians were open (62/103, 60.2%) than not open (21/103, 20.4%) to discussing the results. Compared with patients who had not previously experienced diagnostic errors (missed or delayed diagnoses: 123/304, 40.5%), patients who had previously experienced diagnostic errors (181/304, 59.5%) were more likely to use the symptom checker to determine where they should seek care (15/123, 12.2% vs 48/181, 26.5%; P=.002), but they less often felt that physicians were interested in discussing the tool’s results (20/34, 59% vs 22/69, 32%; P=.04).

Conclusions

Despite ongoing concerns about symptom checker accuracy, a large patient-user group perceived an AI-assisted symptom checker as useful for diagnosis. Formal validation studies evaluating symptom checker accuracy and effectiveness in real-world practice could provide additional useful information about their benefit.

Pediatric clinician perspectives on communicating diagnostic uncertainty

OBJECTIVE

Diagnosis often evolves over time, involves uncertainty, and is vulnerable to errors. We examined pediatric clinicians' perspectives on communicating diagnostic uncertainty to patients' parents and how this occurs.

DESIGN

We conducted semi-structured interviews, which were audiotaped, transcribed, and analyzed using content analysis. Two researchers independently coded transcripts and then discussed discrepancies to reach consensus.

SETTING

A purposive sample of pediatric clinicians at two large academic medical institutions in Texas.

PARTICIPANTS

Twenty pediatric clinicians participated: 18 physicians, 2 nurse practitioners; 7 males, 13 females; 7 inpatient, 11 outpatient, and 2 practicing in mixed settings; with 0-16 years' experience post-residency.

INTERVENTION(S)

None.

MAIN OUTCOME MEASURE(S)

Pediatric clinician perspectives on communication of diagnostic uncertainty.

RESULTS

Pediatric clinicians commonly experienced diagnostic uncertainty and most were comfortable seeking help and discussing with colleagues. However, when communicating uncertainty to parents, clinicians used multiple considerations to adjust the degree to which they communicated. Considerations included parent characteristics (education, socioeconomic status, emotional response, and culture) and strength of parent-clinician relationships. Communication content included setting expectations, explaining the diagnostic process, discussing most relevant differentials, and providing reassurance. Responses to certain parent characteristics, however, were variable. For example, some clinicians were more open to discussing diagnostic uncertainty with more educated parents- others were less.

CONCLUSIONS

While pediatric clinicians are comfortable discussing diagnostic uncertainty with colleagues, how they communicate uncertainty to parents appears variable. Parent characteristics and parent-clinician relationships affect extent of communication and content discussed. Development and implementation of optimal strategies for managing and communicating diagnostic uncertainty can improve the diagnostic process.

Evaluating a mobile application for improving clinical laboratory test ordering and diagnosis

Objective

Mobile applications for improving diagnostic decision making often lack clinical evaluation. We evaluated if a mobile application improves generalist physicians' appropriate laboratory test ordering and diagnosis decisions and assessed if physicians perceive it as useful for learning.

Methods

In an experimental, vignette study, physicians diagnosed 8 patient vignettes with normal prothrombin times (PT) and abnormal partial thromboplastin times (PTT). Physicians made test ordering and diagnosis decisions for 4 vignettes using each resource: a mobile app, PTT Advisor, developed by the Centers for Disease Control and Prevention (CDC)'s Clinical Laboratory Integration into Healthcare Collaborative (CLIHC); and usual clinical decision support. Then, physicians answered questions regarding their perceptions of the app's usefulness for diagnostic decision making and learning using a modified Kirkpatrick Training Evaluation Framework.

Results

Data from 368 vignettes solved by 46 physicians at 7 US health care institutions show advantages for using PTT Advisor over usual clinical decision support on test ordering and diagnostic decision accuracy (82.6 vs 70.2% correct; P < .001), confidence in decisions (7.5 vs 6.3 out of 10; P < .001), and vignette completion time (3:02 vs 3:53 min.; P = .06). Physicians reported positive perceptions of the app's potential for improved clinical decision making, and recommended it be used to address broader diagnostic challenges.

Conclusions

A mobile app, PTT Advisor, may contribute to better test ordering and diagnosis, serve as a learning tool for diagnostic evaluation of certain clinical disorders, and improve patient outcomes. Similar methods could be useful for evaluating apps aimed at improving testing and diagnosis for other conditions.

Impact of a national QI programme on reducing electronic health record notifications to clinicians

BACKGROUND

Emerging evidence suggests electronic health record (EHR)-related information overload is a risk to patient safety. In the US Department of Veterans Affairs (VA), EHR-based 'inbox' notifications originally intended for communicating important clinical information are now cited by 70% of primary care practitioners (PCPs) to be of unmanageable volume. We evaluated the impact of a national, multicomponent, quality improvement (QI) programme to reduce low-value EHR notifications.

METHODS

The programme involved three steps: (1) accessing daily PCP notification load data at all 148 facilities operated nationally by the VA; (2) standardising and restricting mandatory notification types at all facilities to a recommended list; and (3) hands-on training for all PCPs on customising and processing notifications more effectively. Designated leaders at each of VA's 18 regional networks led programme implementation using a nationally developed toolkit. Each network supervised technical requirements and data collection, ensuring consistency. Coaching calls and emails allowed the national team to address implementation challenges and monitor effects. We analysed notification load and mandatory notifications preintervention (March 2017) and immediately postintervention (June-July 2017) to assess programme impact.

RESULTS

Median number of mandatory notification types at each facility decreased significantly from 15 (IQR: 13-19) to 10 (IQR: 10-11) preintervention to postintervention, respectively (P<0.001). Mean daily notifications per PCP decreased significantly from 128 (SEM=4) to 116 (SEM=4; P<0.001). Heterogeneity in implementation across sites led to differences in observed programme impact, including potentially beneficial carryover effects.

CONCLUSIONS

Based on prior estimates on time to process notifications, a national QI programme potentially saved 1.5 hours per week per PCP to enable higher value work. The number of daily notifications remained high, suggesting the need for additional multifaceted interventions and protected clinical time to help manage them. Nevertheless, our project suggests feasibility of using large-scale 'de-implementation' interventions to reduce unintended safety or efficiency consequences of well-intended electronic communication systems.

Patient perspectives on how physicians communicate diagnostic uncertainty: An experimental vignette study

OBJECTIVE

We evaluated the effects of three different strategies for communicating diagnostic uncertainty on patient perceptions of physician competence and visit satisfaction.

DESIGN/SETTING

Experimental vignette-based study design involving pediatric cases presented to a convenience sample of parents living in a large US city.

PARTICIPANTS/INTERVENTION(S)

Three vignettes were developed, each describing one of three different ways physicians communicated diagnostic uncertainty to parents-(i) explicit expression of uncertainty ('not sure' about diagnosis), (ii) implicit expression of uncertainty using broad differential diagnoses and (iii) implicit expression of uncertainty using 'most likely' diagnoses. Participants were randomly assigned to one of the three vignettes and then answered a 37-item web-based questionnaire.

MAIN OUTCOME MEASURE(S)

Outcome variables included parent-perceived technical competence of physician, trust and confidence, visit satisfaction and adherence to physician instructions. Differences between the three groups were compared using analysis of variance, followed by individual post hoc analyses with Bonferroni correction.

RESULTS

Seventy-one participants completed the vignette questions. Demographic characteristics and scores on activation (parent activation measure [PAM]) and intolerance to uncertainty were similar across the three groups. Explicit expression of uncertainty was associated with lower perceived technical competence, less trust and confidence, and lower patient adherence as compared to the two groups with implicit communication. These latter two groups had comparable outcomes.

CONCLUSION

Parents may react less negatively in terms of perceived competence, physician confidence and trust, and intention to adhere when diagnostic uncertainty is communicated using implicit strategies, such as using broad differential diagnoses or most likely diagnoses. Evidence-based strategies to communicate diagnostic uncertainty to patients need further development.

Development and Validation of Trigger Algorithms to Identify Delays in Diagnostic Evaluation of Gastroenterological Cancer

BACKGROUND & AIMS

Colorectal cancer (CRC) and hepatocellular cancer (HCC) are common causes of death and morbidity, and patients benefit from early detection. However, delays in follow-up of suspicious findings are common, and methods to efficiently detect such delays are needed. We developed, refined, and tested trigger algorithms that identify patients with delayed follow-up evaluation of findings suspicious of CRC or HCC.

METHODS

We developed and validated two trigger algorithms that detect delays in diagnostic evaluation of CRC and HCC using laboratory, diagnosis, procedure, and referral codes from the Department of Veteran Affairs National Corporate Data Warehouse. The algorithm initially identified patients with positive test results for iron deficiency anemia or fecal immunochemical test (for CRC) and elevated α-fetoprotein results (for HCC). Our algorithm then excluded patients for whom follow-up evaluation was unnecessary, such as patients with a terminal illness or those who had already completed a follow-up evaluation within 60 days. Clinicians reviewed samples of both delayed and nondelayed records, and review data were used to calculate trigger performance.

RESULTS

We applied the algorithm for CRC to 245,158 patients seen from January 1, 2013, through December 31, 2013 and identified 1073 patients with delayed follow up. In a review of 400 randomly selected records, we found that our algorithm identified patients with delayed follow-up with a positive predictive value of 56.0% (95% CI, 51.0%-61.0%). We applied the algorithm for HCC to 333,828 patients seen from January 1, 2011 through December 31, 2014, and identified 130 patients with delayed follow-up. During manual review of all 130 records, we found that our algorithm identified patients with delayed follow-up with a positive predictive value of 82.3% (95% CI, 74.4%-88.2%). When we extrapolated the findings to all patients with abnormal results, the algorithm identified patients with delayed follow-up evaluation for CRC with 68.6% sensitivity (95% CI, 65.4%-71.6%) and 81.1% specificity (95% CI, 79.5%-82.6%); it identified patients with delayed follow-up evaluation for HCC with 89.1% sensitivity (95% CI, 81.8%-93.8%) and 96.5% specificity (95% CI, 94.8%-97.7%). Compared to nonselective methods, use of the algorithm reduced the number of records required for review to identify a delay by more than 99%.

CONCLUSIONS

Using data from the Veterans Affairs electronic health record database, we developed an algorithm that greatly reduces the number of record reviews necessary to identify delays in follow-up evaluations for patients with suspected CRC or HCC. This approach offers a more efficient method to identify delayed diagnostic evaluation of gastroenterological cancers.

Errors in Diagnosis of Spinal Epidural Abscesses in the Era of Electronic Health Records

PURPOSE

With this study, we set out to identify missed opportunities in diagnosis of spinal epidural abscesses to outline areas for process improvement.

METHODS

Using a large national clinical data repository, we identified all patients with a new diagnosis of spinal epidural abscess in the Department of Veterans Affairs (VA) during 2013. Two physicians independently conducted retrospective chart reviews on 250 randomly selected patients and evaluated their records for red flags (eg, unexplained weight loss, neurological deficits, and fever) 90 days prior to diagnosis. Diagnostic errors were defined as missed opportunities to evaluate red flags in a timely or appropriate manner. Reviewers gathered information about process breakdowns related to patient factors, the patient-provider encounter, test performance and interpretation, test follow-up and tracking, and the referral process. Reviewers also determined harm and time lag between red flags and definitive diagnoses.

RESULTS

Of 250 patients, 119 had a new diagnosis of spinal epidural abscess, 66 (55.5%) of which experienced diagnostic error. Median time to diagnosis in error cases was 12 days, compared with 4 days in cases without error (P <.01). Red flags that were frequently not evaluated in error cases included unexplained fever (n = 57; 86.4%), focal neurological deficits with progressive or disabling symptoms (n = 54; 81.8%), and active infection (n = 54; 81.8%). Most errors involved breakdowns during the patient-provider encounter (n = 60; 90.1%), including failures in information gathering/integration, and were associated with temporary harm (n = 43; 65.2%).

CONCLUSION

Despite wide availability of clinical data, errors in diagnosis of spinal epidural abscesses are common and involve inadequate history, physical examination, and test ordering. Solutions should include renewed attention to basic clinical skills.

Electronic Detection of Delayed Test Result Follow-Up in Patients with Hypothyroidism

BACKGROUND

Delays in following up abnormal test results are a common problem in outpatient settings. Surveillance systems that use trigger tools to identify delayed follow-up can help reduce missed opportunities in care.

OBJECTIVE

To develop and test an electronic health record (EHR)-based trigger algorithm to identify instances of delayed follow-up of abnormal thyroid-stimulating hormone (TSH) results in patients being treated for hypothyroidism.

DESIGN

We developed an algorithm using structured EHR data to identify patients with hypothyroidism who had delayed follow-up (>60 days) after an abnormal TSH. We then retrospectively applied the algorithm to a large EHR data warehouse within the Department of Veterans Affairs (VA), on patient records from two large VA networks for the period from January 1, 2011, to December 31, 2011. Identified records were reviewed to confirm the presence of delays in follow-up.

KEY RESULTS

During the study period, 645,555 patients were seen in the outpatient setting within the two networks. Of 293,554 patients with at least one TSH test result, the trigger identified 1250 patients on treatment for hypothyroidism with elevated TSH. Of these patients, 271 were flagged as potentially having delayed follow-up of their test result. Chart reviews confirmed delays in 163 of the 271 flagged patients (PPV = 60.1%).

CONCLUSIONS

An automated trigger algorithm applied to records in a large EHR data warehouse identified patients with hypothyroidism with potential delays in thyroid function test results follow-up. Future prospective application of the TSH trigger algorithm can be used by clinical teams as a surveillance and quality improvement technique to monitor and improve follow-up.

Application of Electronic Algorithms to Improve Diagnostic Evaluation for Bladder Cancer

BACKGROUND

Strategies to ensure timely diagnostic evaluation of hematuria are needed to reduce delays in bladder cancer diagnosis.

OBJECTIVE

To evaluate the performance of electronic trigger algorithms to detect delays in hematuria follow-up.

METHODS

We developed a computerized trigger to detect delayed follow-up action on a urinalysis result with high-grade hematuria (>50 red blood cells/high powered field). The trigger scanned clinical data within a Department of Veterans Affairs (VA) national data repository to identify all patient records with hematuria, then excluded those where follow-up was unnecessary (e.g., terminal illness) or where typical follow-up action was detected (e.g., cystoscopy). We manually reviewed a randomly-selected sample of flagged records to confirm delays. We performed a similar analysis of records with hematuria that were marked as not delayed (non-triggered). We used review findings to calculate trigger performance.

RESULTS

Of 310,331 patients seen between 1/1/2012-12/31/2014, the trigger identified 5,857 patients who experienced high-grade hematuria, of which 495 experienced a delay. On manual review of 400 randomly-selected triggered records and 100 non-triggered records, the trigger achieved positive and negative predictive values of 58% and 97%, respectively.

CONCLUSIONS

Triggers offer a promising method to detect delays in care of patients with high-grade hematuria and warrant further evaluation in clinical practice as a means to reduce delays in bladder cancer diagnosis.

Electronic health record-related safety concerns: a cross-sectional survey

Federal electronic health record (EHR)-related initiatives are leading to rapid increases in their adoption. Despite their benefits, EHRs also introduce new risks that can lead to serious safety events. We conducted a Web-based survey of the American Society for Healthcare Risk Management and the American Health Lawyers Association to elicit perceptions regarding the frequency and types of EHR-related serious safety events. We received 369 responses. The majority (66%) worked for large hospitals and health systems with varying degrees of EHR adoption. More than half (53%) of respondents reported at least one EHR-related serious safety event in the previous 5 years, and 10% reported more than 20 events. EHR workflow (63%), user familiarity with the EHR system (63%), and integration with existing systems (59%) were most frequently endorsed as variables associated with EHR-related serious safety events. Because EHR-related safety concerns are underreported, organizations should consider implementing robust measures of EHR safety within their institution as a key step for mitigating these concerns.

Online public reactions to frequency of diagnostic errors in US outpatient care

BACKGROUND

Diagnostic errors pose a significant threat to patient safety but little is known about public perceptions of diagnostic errors. A study published in BMJ Quality & Safety in 2014 estimated that diagnostic errors affect at least 5% of US adults (or 12 million) per year. We sought to explore online public reactions to media reports on the reported frequency of diagnostic errors in the US adult population.

METHODS

We searched the World Wide Web for any news article reporting findings from the study. We then gathered all the online comments made in response to the news articles to evaluate public reaction to the newly reported diagnostic error frequency (n=241). Two coders conducted content analyses of the comments and an experienced qualitative researcher resolved differences.

RESULTS

Overall, there were few comments made regarding the frequency of diagnostic errors. However, in response to the media coverage, 44 commenters shared personal experiences of diagnostic errors. Additionally, commentary centered on diagnosis-related quality of care as affected by two emergent categories: (1) US health care providers (n=79; 63 commenters) and (2) US health care reform-related policies, most commonly the Affordable Care Act (ACA) and insurance/reimbursement issues (n=62; 47 commenters).

CONCLUSION

The public appears to have substantial concerns about the impact of the ACA and other reform initiatives on the diagnosis-related quality of care. However, policy discussions on diagnostic errors are largely absent from the current national conversation on improving quality and safety. Because outpatient diagnostic errors have emerged as a major safety concern, researchers and policymakers should consider evaluating the effects of policy and practice changes on diagnostic accuracy.

Workarounds and Test Results Follow-up in Electronic Health Record-Based Primary Care

BACKGROUND

Electronic health records (EHRs) have potential to facilitate reliable communication and follow-up of test results. However, limitations in EHR functionality remain, leading practitioners to use workarounds while managing test results. Workarounds can lead to patient safety concerns and signify indications as to how to build better EHR systems that meet provider needs.

OBJECTIVE

To understand why primary care practitioners (PCPs) use workarounds to manage test results by analyzing data from a previously conducted national cross-sectional survey on test result management.

METHODS

We conducted a secondary data analysis of quantitative and qualitative data from a national survey of PCPs practicing in the Department of Veterans Affairs (VA) and explored the use of workarounds in test results management. We used multivariate logistic regression analysis to examine the association between key sociotechnical factors that could affect test results follow-up (e.g., both technology-related and those unrelated to technology, such as organizational support for patient notification) and workaround use. We conducted a qualitative content analysis of free text survey data to examine reasons for use of workarounds.

RESULTS

Of 2554 survey respondents, 1104 (43%) reported using workarounds related to test results management. Of these 1028 (93%) described the type of workaround they were using; 719 (70%) reported paper-based methods, while 230 (22%) used a combination of paper- and computer-based workarounds. Primary care practitioners who self-reported limited administrative support to help them notify patients of test results or described an instance where they personally (or a colleague) missed results, were more likely to use workarounds (p=0.02 and p=0.001, respectively). Qualitative analysis identified three main reasons for workaround use: 1) as a memory aid, 2) for improved efficiency and 3) for facilitating internal and external care coordination.

CONCLUSION

Workarounds to manage EHR-based test results are common, and their use results from unmet provider information management needs. Future EHRs and the respective work systems around them need to evolve to meet these needs.

Accuracy of the Safer Dx Instrument to Identify Diagnostic Errors in Primary Care

IMPORTANCE

Diagnostic errors are common and harmful, but difficult to define and measure. Measurement of diagnostic errors often depends on retrospective medical record reviews, frequently resulting in reviewer disagreement.

OBJECTIVES

We aimed to test the accuracy of an instrument to help detect presence or absence of diagnostic error through record reviews.

DESIGN

We gathered questions from several previously used instruments for diagnostic error measurement, then developed and refined our instrument. We tested the accuracy of the instrument against a sample of patient records (n = 389), with and without previously identified diagnostic errors (n = 129 and n = 260, respectively).

RESULTS

The final version of our instrument (titled Safer Dx Instrument) consisted of 11 questions assessing diagnostic processes in the patient-provider encounter and a main outcome question to determine diagnostic error. In comparison with the previous sample, the instrument yielded an overall accuracy of 84 %, sensitivity of 71 %, specificity of 90 %, negative predictive value of 86 %, and positive predictive value of 78 %. All 11 items correlated significantly with the instrument's error outcome question (all p values ≤ 0.01). Using factor analysis, the 11 questions clustered into two domains with high internal consistency (initial diagnostic assessment, and performance and interpretation of diagnostic tests) and a patient factor domain with low internal consistency (Cronbach's alpha coefficients 0.93, 0.92, and 0.38, respectively).

CONCLUSIONS

The Safer Dx Instrument helps quantify the likelihood of diagnostic error in primary care visits, achieving a high degree of accuracy for measuring their presence or absence. This instrument could be useful to identify high-risk cases for further study and quality improvement.

Crowdsourcing Diagnosis for Patients With Undiagnosed Illnesses: An Evaluation of CrowdMed

Background

Despite visits to multiple physicians, many patients remain undiagnosed. A new online program, CrowdMed, aims to leverage the “wisdom of the crowd” by giving patients an opportunity to submit their cases and interact with case solvers to obtain diagnostic possibilities.

Objective

To describe CrowdMed and provide an independent assessment of its impact.

Methods

Patients submit their cases online to CrowdMed and case solvers sign up to help diagnose patients. Case solvers attempt to solve patients’ diagnostic dilemmas and often have an interactive online discussion with patients, including an exchange of additional diagnostic details. At the end, patients receive detailed reports containing diagnostic suggestions to discuss with their physicians and fill out surveys about their outcomes. We independently analyzed data collected from cases between May 2013 and April 2015 to determine patient and case solver characteristics and case outcomes.

Results

During the study period, 397 cases were completed. These patients previously visited a median of 5 physicians, incurred a median of US $10,000 in medical expenses, spent a median of 50 hours researching their illnesses online, and had symptoms for a median of 2.6 years. During this period, 357 active case solvers participated, of which 37.9% (132/348) were male and 58.3% (208/357) worked or studied in the medical industry. About half (50.9%, 202/397) of patients were likely to recommend CrowdMed to a friend, 59.6% (233/391) reported that the process gave insights that led them closer to the correct diagnoses, 57% (52/92) reported estimated decreases in medical expenses, and 38% (29/77) reported estimated improvement in school or work productivity.

Conclusions

Some patients with undiagnosed illnesses reported receiving helpful guidance from crowdsourcing their diagnoses during their difficult diagnostic journeys. However, further development and use of crowdsourcing methods to facilitate diagnosis requires long-term evaluation as well as validation to account for patients’ ultimate correct diagnoses.

Diagnostic errors related to acute abdominal pain in the emergency department

OBJECTIVE

Diagnostic errors in the emergency department (ED) are harmful and costly. We reviewed a selected high-risk cohort of patients presenting to the ED with abdominal pain to evaluate for possible diagnostic errors and associated process breakdowns.

DESIGN

We conducted a retrospective chart review of ED patients >18 years at an urban academic hospital. A computerised 'trigger' algorithm identified patients possibly at high risk for diagnostic errors to facilitate selective record reviews. The trigger determined patients to be at high risk because they: (1) presented to the ED with abdominal pain, and were discharged home and (2) had a return ED visit within 10 days that led to a hospitalisation. Diagnostic errors were defined as missed opportunities to make a correct or timely diagnosis based on the evidence available during the first ED visit, regardless of patient harm, and included errors that involved both ED and non-ED providers. Errors were determined by two independent record reviewers followed by team consensus in cases of disagreement.

RESULTS

Diagnostic errors occurred in 35 of 100 high-risk cases. Over two-thirds had breakdowns involving the patient-provider encounter (most commonly history-taking or ordering additional tests) and/or follow-up and tracking of diagnostic information (most commonly follow-up of abnormal test results). The most frequently missed diagnoses were gallbladder pathology (n=10) and urinary infections (n=5).

CONCLUSIONS

Diagnostic process breakdowns in ED patients with abdominal pain most commonly involved history-taking, ordering insufficient tests in the patient-provider encounter and problems with follow-up of abnormal test results.

Electronic Trigger-Based Intervention to Reduce Delays in Diagnostic Evaluation for Cancer: A Cluster Randomized Controlled Trial

PURPOSE

We tested whether prospective use of electronic health record-based trigger algorithms to identify patients at risk of diagnostic delays could prevent delays in diagnostic evaluation for cancer.

METHODS

We performed a cluster randomized controlled trial of primary care providers (PCPs) at two sites to test whether triggers that prospectively identify patients with potential delays in diagnostic evaluation for lung, colorectal, or prostate cancer can reduce time to follow-up diagnostic evaluation. Intervention steps included queries of the electronic health record repository for patients with abnormal findings and lack of associated follow-up actions, manual review of triggered records, and communication of this information to PCPs via secure e-mail and, if needed, phone calls to ensure message receipt. We compared times to diagnostic evaluation and proportions of patients followed up between intervention and control cohorts based on final review at 7 months.

RESULTS

We recruited 72 PCPs (36 in the intervention group and 36 in the control group) and applied the trigger to all patients under their care from April 20, 2011, to July 19, 2012. Of 10,673 patients with abnormal findings, the trigger flagged 1,256 patients (11.8%) as high risk for delayed diagnostic evaluation. Times to diagnostic evaluation were significantly lower in intervention patients compared with control patients flagged by the colorectal trigger (median, 104 v 200 days, respectively; n = 557; P < .001) and prostate trigger (40% received evaluation at 144 v 192 days, respectively; n = 157; P < .001) but not the lung trigger (median, 65 v 93 days, respectively; n = 19; P = .59). More intervention patients than control patients received diagnostic evaluation by final review (73.4% v 52.2%, respectively; relative risk, 1.41; 95% CI, 1.25 to 1.58).

CONCLUSION

Electronic trigger-based interventions seem to be effective in reducing time to diagnostic evaluation of colorectal and prostate cancer as well as improving the proportion of patients who receive follow-up. Similar interventions could improve timeliness of diagnosis of other serious conditions.

Evaluation of outcomes from a national patient-initiated second-opinion program

BACKGROUND

We examined outcomes of patient-initiated second opinions provided by a national second-opinion program.

METHODS

We independently examined data collected from January 1, 2011 to December 31, 2012 from a second-opinion program (Best Doctors, Inc.) that allows employee-beneficiaries to request free second opinions. Clinical intake included ascertaining why patients sought second opinions and acquiring patients' complete medical records. Trained physicians summarized the cases; identified key, unresolved clinical questions; and forwarded the cases to expert specialists who provided independent assessments and recommendations. Second opinions were discussed with and returned to patients for review with their physicians. Nurses determined whether second opinions confirmed, clarified, or changed initial diagnoses and treatments, and physicians estimated their clinical impact. Patient satisfaction also was surveyed.

RESULTS

A total of 6791 patient-initiated second opinions were completed across medical specialties. Patients primarily sought second opinions for help choosing treatment options (41.3%) and for diagnostic concerns (34.8%). Second opinions often resulted in changes in diagnosis (14.8%), treatment (37.4%), or changes in both (10.6%). Clinical impact was estimated as moderate/major in 20.9% of cases for diagnosis and 30.7% of cases for treatment. Changes in diagnoses and/or treatments and clinical impact varied across medical specialties. In patients surveyed (n = 2683), most (94.7%) were satisfied with the experience, but fewer (61.2%) planned to follow the recommendations.

CONCLUSIONS

Patient-initiated second opinions led to recommended changes in diagnosis for about 15% and in treatment for about 37% of participants. Further evaluation is needed to determine whether this impacts clinical outcomes, such as the reduction of diagnosis and treatment errors.

Development and Validation of Electronic Health Record-based Triggers to Detect Delays in Follow-up of Abnormal Lung Imaging Findings

Purpose To develop an electronic health record (EHR)-based trigger algorithm to identify delays in follow-up of patients with imaging results that are suggestive of lung cancer and to validate this trigger on retrospective data. Materials and Methods The local institutional review board approved the study. A "trigger" algorithm was developed to automate the detection of delays in diagnostic evaluation of chest computed tomographic (CT) images and conventional radiographs that were electronically flagged by reviewing radiologists as being "suspicious for malignancy." The trigger algorithm was developed through literature review and expert input. It included patients who were alive and 40-70 years old, and it excluded instances in which appropriate timely follow-up (defined as occurring within 30 days) was detected (eg, pulmonary visit) or when follow-up was unnecessary (eg, in patients with a terminal illness). The algorithm was iteratively applied to a retrospective test cohort in an EHR data warehouse at a large Veterans Affairs facility, and manual record reviews were used to validate each individual criterion. The final algorithm aimed at detecting an absence of timely follow-up was retrospectively applied to an independent validation cohort to determine the positive predictive value (PPV). Trigger performance, time to follow-up, reasons for lack of follow-up, and cancer outcomes were analyzed and reported by using descriptive statistics. Results The trigger algorithm was retrospectively applied to the records of 89 168 patients seen between January 1, 2009, and December 31, 2009. Of 538 records with an imaging report that was flagged as suspicious for malignancy, 131 were identified by the trigger as being high risk for delayed diagnostic evaluation. Manual chart reviews confirmed a true absence of follow-up in 75 cases (trigger PPV of 57.3% for detecting evaluation delays), of which four received a diagnosis of primary lung cancer within the subsequent 2 years. Conclusion EHR-based triggers can be used to identify patients with suspicious imaging findings in whom follow-up diagnostic evaluation was delayed. (©) RSNA, 2015.

Lack of timely follow-up of abnormal imaging results and radiologists' recommendations

PURPOSE

Abnormal imaging results may not always lead to timely follow-up. We tested whether certain aspects of communication in radiology reports influence the response of the referring providers, and hence follow-up on abnormal findings.

METHODS

We focused on 2 communication-related items that we hypothesized could affect follow-up: expressions of doubt in the radiology report, and recommendations for further imaging. After institutional review board approval, we conducted a retrospective review of 250 outpatient radiology reports from a multispecialty ambulatory clinic of a tertiary-care Veterans Affairs facility. The selected studies included 92 cases confirmed to lack timely follow-up (ie, further tests or consultations, treatment, and/or communication to the patient within 4 weeks), as determined in a previous study. An additional 158 cases with documented timely follow-up served as controls. Doubt in the narrative was measured by the presence of key phrases (eg, "unable to exclude," "cannot exclude," "cannot rule out," "possibly," and "unlikely"), in the absence of which we used reviewer interpretation. A physician blinded to follow-up outcomes collected the data.

RESULTS

Patients whose reports contained recommendations for further imaging were more likely to have been lost to follow-up at 4 weeks compared with patients without such recommendations (P = .01). Language in the report suggestive of doubt did not affect the timeliness of follow-up (P = .59).

CONCLUSIONS

Abnormal imaging results with recommendations for additional imaging may be more vulnerable to lack of timely follow-up. Additional safeguards, such as tracking systems, should be developed to prevent failure to follow up on such results.

Developing checklists to prevent diagnostic error in Emergency Room settings

BACKGROUND

Checklists have been shown to improve performance of complex, error-prone processes. To develop a checklist with potential to reduce the likelihood of diagnostic error for patients presenting to the Emergency Room (ER) with undiagnosed conditions.

METHODS

Participants included 15 staff ER physicians working in two large academic centers. A rapid cycle design and evaluation process was used to develop a general checklist for high-risk situations vulnerable to diagnostic error. Physicians used the general checklists and a set of symptom-specific checklists for a period of 2 months. We conducted a mixed methods evaluation that included interviews regarding user perceptions and quantitative assessment of resource utilization before and after checklist use.

RESULTS

A general checklist was developed iteratively by obtaining feedback from users and subject matter experts, and was trialed along with a set of specific checklists in the ER. Both the general and the symptom-specific checklists were judged to be helpful, with a slight preference for using symptom-specific lists. Checklist use commonly prompted consideration of additional diagnostic possibilities, changed the working diagnosis in approximately 10% of cases, and anecdotally was thought to be helpful in avoiding diagnostic errors. Checklist use was prompted by a variety of different factors, not just diagnostic uncertainty. None of the physicians used the checklists in collaboration with the patient, despite being encouraged to do so. Checklist use did not prompt large changes in test ordering or consultation.

CONCLUSIONS

In the ER setting, checklists for diagnosis are helpful in considering additional diagnostic possibilities, thus having potential to prevent diagnostic errors. Inconsistent usage and using the checklists privately, instead of with the patient, are factors that may detract from obtaining maximum benefit. Further research is needed to optimize checklists for use in the ER, determine how to increase usage, to evaluate the impact of checklist utilization on error rates and patient outcomes, to determine how checklist usage affects test ordering and consultation, and to compare checklists generally with other approaches to reduce diagnostic error.

Patient-initiated second opinions: systematic review of characteristics and impact on diagnosis, treatment, and satisfaction

The impact of second opinions on diagnosis in radiology and pathology is well documented; however, the value of patient-initiated second opinions for diagnosis and treatment in general medical practice is unknown. We conducted a systematic review of patient-initiated second opinions to assess their impact on clinical outcomes and patient satisfaction and to determine characteristics and motivating factors of patients who seek a second opinion. We searched PubMed, EMBASE, Cochrane, and Academic OneFile databases using Medical Subject Headings (MeSH) indexes and keyword searches. Search terms included referral and consultation, patient-initiated, patient preference, patient participation, second opinion, second review, and diagnosis. Multiple reviewers screened abstracts and articles to determine eligibility and extract data. We assessed risk of bias using the Cochrane Risk of Bias Tool and rated study quality using Cochrane's GRADE (Grades of Recommendation, Assessment, Development and Evaluation) approach. We screened 1342 abstracts and reviewed full text of 41 articles, identifying 7 articles that reported clinical agreement data and 10 that discussed patient characteristics, motivation, and satisfaction. We found that a second opinion typically confirms the original diagnosis or treatment regimen but that 90% of patients with poorly defined conditions remain undiagnosed. However, 10% to 62% of second opinions yield a major change in the diagnosis, treatment, or prognosis. A larger fraction of patients receive different advice on treatment than on diagnosis. Factors motivating a second opinion include diagnosis or treatment confirmation, dissatisfaction with a consultation, desire for more information, persistent symptoms, or treatment complications. Patients generally believed that second opinions were valuable. Second opinions can result in diagnostic and treatment differences. The literature on patient-initiated second opinions is limited, and the accuracy of the second opinion through follow-up is generally unknown. Standardized methods and outcome measures are needed to determine the value of second opinions, and the potential of second opinions to reduce diagnostic errors merits more rigorous evaluation.

The frequency of diagnostic errors in outpatient care: estimations from three large observational studies involving US adult populations

BACKGROUND

The frequency of outpatient diagnostic errors is challenging to determine due to varying error definitions and the need to review data across multiple providers and care settings over time. We estimated the frequency of diagnostic errors in the US adult population by synthesising data from three previous studies of clinic-based populations that used conceptually similar definitions of diagnostic error.

METHODS

Data sources included two previous studies that used electronic triggers, or algorithms, to detect unusual patterns of return visits after an initial primary care visit or lack of follow-up of abnormal clinical findings related to colorectal cancer, both suggestive of diagnostic errors. A third study examined consecutive cases of lung cancer. In all three studies, diagnostic errors were confirmed through chart review and defined as missed opportunities to make a timely or correct diagnosis based on available evidence. We extrapolated the frequency of diagnostic error obtained from our studies to the US adult population, using the primary care study to estimate rates of diagnostic error for acute conditions (and exacerbations of existing conditions) and the two cancer studies to conservatively estimate rates of missed diagnosis of colorectal and lung cancer (as proxies for other serious chronic conditions).

RESULTS

Combining estimates from the three studies yielded a rate of outpatient diagnostic errors of 5.08%, or approximately 12 million US adults every year. Based upon previous work, we estimate that about half of these errors could potentially be harmful.

CONCLUSIONS

Our population-based estimate suggests that diagnostic errors affect at least 1 in 20 US adults. This foundational evidence should encourage policymakers, healthcare organisations and researchers to start measuring and reducing diagnostic errors.

Physicians' diagnostic accuracy, confidence, and resource requests: a vignette study

IMPORTANCE

Little is known about the relationship between physicians' diagnostic accuracy and their confidence in that accuracy.

OBJECTIVE

To evaluate how physicians' diagnostic calibration, defined as the relationship between diagnostic accuracy and confidence in that accuracy, changes with evolution of the diagnostic process and with increasing diagnostic difficulty of clinical case vignettes.

DESIGN, SETTING, AND PARTICIPANTS

We recruited general internists from an online physician community and asked them to diagnose 4 previously validated case vignettes of variable difficulty (2 easier; 2 more difficult). Cases were presented in a web-based format and divided into 4 sequential phases simulating diagnosis evolution: history, physical examination, general diagnostic testing data, and definitive diagnostic testing. After each phase, physicians recorded 1 to 3 differential diagnoses and corresponding judgments of confidence. Before being presented with definitive diagnostic data, physicians were asked to identify additional resources they would require to diagnose each case (ie, additional tests, second opinions, curbside consultations, referrals, and reference materials).

MAIN OUTCOMES AND MEASURES

Diagnostic accuracy (scored as 0 or 1), confidence in diagnostic accuracy (on a scale of 0-10), diagnostic calibration, and whether additional resources were requested (no or yes).

RESULTS

A total of 118 physicians with broad geographical representation within the United States correctly diagnosed 55.3% of easier and 5.8% of more difficult cases (P < .001). Despite a large difference in diagnostic accuracy between easier and more difficult cases, the difference in confidence was relatively small (7.2 vs 6.4 out of 10, for easier and more difficult cases, respectively) (P < .001) and likely clinically insignificant. Overall, diagnostic calibration was worse for more difficult cases (P < .001) and characterized by overconfidence in accuracy. Higher confidence was related to decreased requests for additional diagnostic tests (P = .01); higher case difficulty was related to more requests for additional reference materials (P = .01).

CONCLUSIONS AND RELEVANCE

Our study suggests that physicians' level of confidence may be relatively insensitive to both diagnostic accuracy and case difficulty. This mismatch might prevent physicians from reexamining difficult cases where their diagnosis may be incorrect.

Types and origins of diagnostic errors in primary care settings

IMPORTANCE

Diagnostic errors are an understudied aspect of ambulatory patient safety.

OBJECTIVES

To determine the types of diseases missed and the diagnostic processes involved in cases of confirmed diagnostic errors in primary care settings and to determine whether record reviews could shed light on potential contributory factors to inform future interventions.

DESIGN

We reviewed medical records of diagnostic errors detected at 2 sites through electronic health record-based triggers. Triggers were based on patterns of patients' unexpected return visits after an initial primary care index visit.

SETTING

A large urban Veterans Affairs facility and a large integrated private health care system.

PARTICIPANTS

Our study focused on 190 unique instances of diagnostic errors detected in primary care visits between October 1, 2006, and September 30, 2007.

MAIN OUTCOME MEASURES

Through medical record reviews, we collected data on presenting symptoms at the index visit, types of diagnoses missed, process breakdowns, potential contributory factors, and potential for harm from errors.

RESULTS

In 190 cases, a total of 68 unique diagnoses were missed. Most missed diagnoses were common conditions in primary care, with pneumonia (6.7%), decompensated congestive heart failure (5.7%), acute renal failure (5.3%), cancer (primary) (5.3%), and urinary tract infection or pyelonephritis (4.8%) being most common. Process breakdowns most frequently involved the patient-practitioner clinical encounter (78.9%) but were also related to referrals (19.5%), patient-related factors (16.3%), follow-up and tracking of diagnostic information (14.7%), and performance and interpretation of diagnostic tests (13.7%). A total of 43.7% of cases involved more than one of these processes. Patient-practitioner encounter breakdowns were primarily related to problems with history-taking (56.3%), examination (47.4%), and/or ordering diagnostic tests for further workup (57.4%). Most errors were associated with potential for moderate to severe harm.

CONCLUSIONS AND RELEVANCE

Diagnostic errors identified in our study involved a large variety of common diseases and had significant potential for harm. Most errors were related to process breakdowns in the patient-practitioner clinical encounter. Preventive interventions should target common contributory factors across diagnoses, especially those that involve data gathering and synthesis in the patient-practitioner encounter.

Taking the testing effect beyond the college freshman: benefits for lifelong learning

Formal learning is a lifelong pursuit that does not occur exclusively within universities. Accordingly, methods for improving long-term learning, including the well-established use of testing, should be examined for various ages of learners outside typical university settings to properly assess their usefulness. This study examined testing effects in 60 younger university students aged 18-25, 60 younger community adults aged 18-25, and 60 middle-aged to older community adults aged 55-65 at immediate and longer delays (2-day). All groups similarly benefited from testing at both delays, implying that testing can be a beneficial lifelong learning tool for a diversity of learners.

Challenges of making a diagnosis in the outpatient setting: a multi-site survey of primary care physicians

BACKGROUND

Although misdiagnosis in the outpatient setting leads to significant patient harm and wasted resources, it is not well studied. The authors surveyed primary care physicians (PCPs) about barriers to timely diagnosis in the outpatient setting and assessed their perceptions of diagnostic difficulty.

METHODS

Surveys of PCPs practicing in an integrated health system across 10 geographically dispersed states in 2005. The survey elicited information on key cognitive failures (including in clinical knowledge or judgement) for a specific case, and solicited strategies for reducing diagnostic delays. Content analysis was used to categorise cognitive failures and strategies for improvement. The authors examined the extent and predictors of diagnostic difficulty, defined as reporting >5% patients difficult to diagnose.

RESULTS

Of 1817 physicians surveyed, 1054 (58%) responded; 848 (80%) respondents primarily practiced in outpatient settings and had an assigned patient panel (inclusion sample). Inadequate knowledge (19.9%) was the most commonly reported cognitive factor. Half reported >5% of their patients were difficult to diagnose; more experienced physicians reported less diagnostic difficulty. In adjusted analyses, problems with information processing (information availability and time to review it) and the referral process were associated with greater diagnostic difficulty. Strategies for improvement most commonly involved workload issues (panel size, non-visit tasks).

CONCLUSIONS

PCPs report a variety of reasons for diagnostic difficulties in primary care practice. In this study, knowledge gaps appear to be a prominent concern. Interventions that address these gaps as well as practice level issues such as time to process diagnostic information and better subspecialty input may reduce diagnostic difficulties in primary care.

Cognitive interventions to reduce diagnostic error: a narrative review

BACKGROUND

Errors in clinical reasoning occur in most cases in which the diagnosis is missed, delayed or wrong. The goal of this review was to identify interventions that might reduce the likelihood of these cognitive errors.

DESIGN

We searched PubMed and other medical and non-medical databases and identified additional literature through references from the initial data set and suggestions from subject matter experts. Articles were included if they either suggested a possible intervention or formally evaluated an intervention and excluded if they focused solely on improving diagnostic tests or provider satisfaction.

RESULTS

We identified 141 articles for full review, 42 reporting tested interventions to reduce the likelihood of cognitive errors, 100 containing suggestions, and one article with both suggested and tested interventions. Articles were classified into three categories: (1) Interventions to improve knowledge and experience, such as simulation-based training, improved feedback and education focused on a single disease; (2) Interventions to improve clinical reasoning and decision-making skills, such as reflective practice and active metacognitive review; and (3) Interventions that provide cognitive 'help' that included use of electronic records and integrated decision support, informaticians and facilitating access to information, second opinions and specialists.

CONCLUSIONS

We identified a wide range of possible approaches to reduce cognitive errors in diagnosis. Not all the suggestions have been tested, and of those that have, the evaluations typically involved trainees in artificial settings, making it difficult to extrapolate the results to actual practice. Future progress in this area will require methodological refinements in outcome evaluation and rigorously evaluating interventions already suggested, many of which are well conceptualised and widely endorsed.