Types and origins of diagnostic errors in primary care settings

Adoption, acceptance, and use of a decision support tool to promote timely investigations for cancer in primary care

BACKGROUND

The complexities of diagnosing cancer in general practice has driven the development of quality improvement (QI) interventions, including clinical decision support (CDS) and auditing tools. Future Health Today (FHT) is a novel QI tool, consisting of CDS at the point-of-care, practice population-level auditing, recall, and the monitoring of QI activities.

OBJECTIVES

Explore the acceptability and usability of the FHT cancer module, which flags patients with abnormal test results that may be indicative of undiagnosed cancer.

METHODS

Interviews were conducted with general practitioners (GPs) and general practice nurses (GPNs), from practices participating in a randomized trial evaluating the appropriate follow-up of patients. Clinical Performance Feedback Intervention Theory (CP-FIT) was used to analyse and interpret the data.

RESULTS

The majority of practices reported not using the auditing and QI components of the tool, only the CDS which was delivered at the point-of-care. The tool was used primarily by GPs; GPNs did not perceive the clinical recommendations to be within their role. For the CDS, facilitators for use included a good workflow fit, ease of use, low time cost, importance, and perceived knowledge gain. Barriers for use of the CDS included accuracy, competing priorities, and the patient population.

CONCLUSIONS

The CDS aligned with the clinical workflow of GPs, was considered non-disruptive to the consultation and easy to implement into usual care. By applying the CP-FIT theory, we were able to demonstrate the key drivers for GPs using the tool, and what limited the use by GPNs.

Large Language Model Influence on Diagnostic Reasoning: A Randomized Clinical Trial

Importance

Large language models (LLMs) have shown promise in their performance on both multiple-choice and open-ended medical reasoning examinations, but it remains unknown whether the use of such tools improves physician diagnostic reasoning.

Objective

To assess the effect of an LLM on physicians' diagnostic reasoning compared with conventional resources.

Design, Setting, and Participants

A single-blind randomized clinical trial was conducted from November 29 to December 29, 2023. Using remote video conferencing and in-person participation across multiple academic medical institutions, physicians with training in family medicine, internal medicine, or emergency medicine were recruited.

Intervention

Participants were randomized to either access the LLM in addition to conventional diagnostic resources or conventional resources only, stratified by career stage. Participants were allocated 60 minutes to review up to 6 clinical vignettes.

Main Outcomes and Measures

The primary outcome was performance on a standardized rubric of diagnostic performance based on differential diagnosis accuracy, appropriateness of supporting and opposing factors, and next diagnostic evaluation steps, validated and graded via blinded expert consensus. Secondary outcomes included time spent per case (in seconds) and final diagnosis accuracy. All analyses followed the intention-to-treat principle. A secondary exploratory analysis evaluated the standalone performance of the LLM by comparing the primary outcomes between the LLM alone group and the conventional resource group.

Results

Fifty physicians (26 attendings, 24 residents; median years in practice, 3 [IQR, 2-8]) participated virtually as well as at 1 in-person site. The median diagnostic reasoning score per case was 76% (IQR, 66%-87%) for the LLM group and 74% (IQR, 63%-84%) for the conventional resources-only group, with an adjusted difference of 2 percentage points (95% CI, -4 to 8 percentage points; P = .60). The median time spent per case for the LLM group was 519 (IQR, 371-668) seconds, compared with 565 (IQR, 456-788) seconds for the conventional resources group, with a time difference of -82 (95% CI, -195 to 31; P = .20) seconds. The LLM alone scored 16 percentage points (95% CI, 2-30 percentage points; P = .03) higher than the conventional resources group.

Conclusions and Relevance

In this trial, the availability of an LLM to physicians as a diagnostic aid did not significantly improve clinical reasoning compared with conventional resources. The LLM alone demonstrated higher performance than both physician groups, indicating the need for technology and workforce development to realize the potential of physician-artificial intelligence collaboration in clinical practice.

Trial Registration

ClinicalTrials.gov Identifier: NCT06157944.

Adverse Events in Patients Transitioning From the Emergency Department to the Inpatient Setting

OBJECTIVES

The objective of this study was to determine the incidence and types of adverse events (AEs), including preventable and ameliorable AEs, in patients transitioning from the emergency department (ED) to the inpatient setting. A second objective was to examine the risk factors for patients with AEs.

METHODS

This was a prospective cohort study of patients at risk for AEs in 2 urban academic hospitals from August 2020 to January 2022. Eighty-one eligible patients who were being admitted to any internal medicine or hospitalist service were recruited from the ED of these hospitals by a trained nurse. The nurse conducted a structured interview during admission and referred possible AEs for adjudication. Two blinded trained physicians using a previously established methodology adjudicated AEs.

RESULTS

Over 22% of 81 patients experienced AEs from the ED to the inpatient setting. The most common AEs were adverse drug events (42%), followed by management (38%), and diagnostic errors (21%). Of these AEs, 75% were considered preventable. Patients who stayed in the ED longer were more likely to experience an AE (adjusted odds ratio = 1.99, 95% confidence interval = 1.19-3.32, P = 0.01).

CONCLUSIONS

AEs were common for patients transitioning from the ED to the inpatient setting. Further research is needed to understand the underlying causes of AEs that occur when patients transition from the ED to the inpatient setting. Understanding the contribution of factors such as length of stay in the ED will significantly help efforts to develop targeted interventions to improve this crucial transition of care.

"Do you smoke?" - content and linguistic analysis of students' substance histories in simulated patient interviews

Background

The use of tobacco, alcohol and other drugs has considerable health consequences. Substance histories are often only incompletely taken in everyday clinical practice. When learning to take a medical history in medical school, one of the learning objectives is to inquire about consumption behavior. The aim of this exploratory study was therefore to examine the content and language of substance histories taken by medical students.

Methods

From a simulation training of a first working day in hospital, 91 video films of medical histories were available, which advanced medical students had conducted with six patients with different consumer behavior. These interviews were verbatim transcribed and analyzed using content-structuring qualitative content analysis according to Kuckartz. For all substances, the reasons for the questions and the depth of the respective substance use were categorized and errors in the questions were examined. In addition, a linguistic analysis of the verbal ways in which the substances were inquired about was carried out.

Results

The students most frequently asked about smoking (73.3%). In only 15.4% of the interviews were all substances asked about, in none were all substances asked about completely. A total of 112 protocol questions and 21 occasion-related questions were identified. Logical errors and double questions were found. Most of the questions were asked in a factual manner. However, questions in the categories "evasive" and "stigmatizing" were also found.

Conclusion

The content-related and linguistic deficits of medical students in the collection of substance histories identified in this study should be addressed in communication courses at an early stage of undergraduate medical studies.

Addressing diagnostic uncertainty and excellence in emergency care-from multicountry policy analysis to communication practice in Australian emergency departments: a multimethod study protocol

INTRODUCTION

Communication failings may compromise the diagnostic process and pose a risk to quality of care and patient safety. With a focus on emergency care settings, this project aims to examine the critical role and impact of communication in the diagnostic process, including in diagnosis-related health and research policy, and diagnostic patient-clinician interactions in emergency departments (EDs).

METHODS AND ANALYSIS

This project uses a qualitatively driven multimethod design integrating findings from two research studies to gain a comprehensive understanding of the impact of context and communication on diagnostic excellence from diverse perspectives. Study 1 will map the diagnostic policy and practice landscape in Australia, New Zealand and the USA through qualitative expert interviews and policy analysis. Study 2 will investigate the communication of uncertainty in diagnostic interactions through a qualitative ethnography of two metropolitan Australian ED sites incorporating observations, field notes, video-recorded interactions, semistructured interviews and written medical documentation, including linguistic analysis of recorded diagnostic interactions and written documentation. This study will also feature a description of clinician, patient and carer perspectives on, and involvement in, interpersonal diagnostic interactions and will provide crucial new insights into the impact of communicating diagnostic uncertainty for these groups. Project-spanning patient and stakeholder involvement strategies will build research capacity among healthcare consumers via educational workshops, engage with community stakeholders in analysis and build consensus among stakeholders.

ETHICS AND DISSEMINATION

The project has received ethical approvals from the Human Research Ethics Committee at ACT Health, Northern Sydney Local Health District and the Australian National University. Findings will be disseminated to academic peers, clinicians and healthcare consumers, health policy-makers and the general public, using local and international academic and consumer channels (journals, evidence briefs and conferences) and outreach activities (workshops and seminars).

"What Else Could It Be?" A Scoping Review of Questions for Patients to Ask Throughout the Diagnostic Process

BACKGROUND

Diagnostic errors are a global patient safety challenge. Over 75% of diagnostic errors in ambulatory care result from breakdowns in patient-clinician communication. Encouraging patients to speak up and ask questions has been recommended as one strategy to mitigate these failures.

OBJECTIVES

The goal of the scoping review was to identify, summarize, and thematically map questions patients are recommended to ask during ambulatory encounters along the diagnostic process. This is the first step in a larger study to co-design a patient-facing question prompt list for patients to use throughout the diagnostic process.

METHODS

Medline and Google Scholar were searched to identify question lists in the peer-reviewed literature. Organizational websites and a search engine were searched to identify question lists in the gray literature. Articles and resources were screened for eligibility and data were abstracted. Interrater reliability (K = 0.875) was achieved.

RESULTS

A total of 5509 questions from 235 resources met inclusion criteria. Most questions (n = 4243, 77.02%) were found in the gray literature. Question lists included an average of 23.44 questions. Questions were most commonly coded along the diagnostic process categories of treatment (2434 questions from 250 resources), communication of the diagnosis (1160 questions, 204 resources), and outcomes (766 questions, 172 resources).

CONCLUSIONS

Despite recommendations for patients to ask questions, most question prompt lists focus on later stages of the diagnostic process such as communication of the diagnosis, treatment, and outcomes. Further research is needed to identify and prioritize diagnostic-related questions from the patient perspective and to develop simple, usable guidance on question-asking to improve patient safety across the diagnostic continuum.

The Use of Deep Learning and Machine Learning on Longitudinal Electronic Health Records for the Early Detection and Prevention of Diseases: Scoping Review

BACKGROUND

Electronic health records (EHRs) contain patients' health information over time, including possible early indicators of disease. However, the increasing amount of data hinders clinicians from using them. There is accumulating evidence suggesting that machine learning (ML) and deep learning (DL) can assist clinicians in analyzing these large-scale EHRs, as algorithms thrive on high volumes of data. Although ML has become well developed, studies mainly focus on engineering but lack medical outcomes.

OBJECTIVE

This study aims for a scoping review of the evidence on how the use of ML on longitudinal EHRs can support the early detection and prevention of disease. The medical insights and clinical benefits that have been generated were investigated by reviewing applications in a variety of diseases.

METHODS

This study was conducted according to the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines. A literature search was performed in 2022 in collaboration with a medical information specialist in the following databases: PubMed, Embase, Web of Science Core Collection (Clarivate Analytics), and IEEE Xplore Digital Library and computer science bibliography. Studies were eligible when longitudinal EHRs were used that aimed for the early detection of disease via ML in a prevention context. Studies with a technical focus or using imaging or hospital admission data were beyond the scope of this review. Study screening and selection and data extraction were performed independently by 2 researchers.

RESULTS

In total, 20 studies were included, mainly published between 2018 and 2022. They showed that a variety of diseases could be detected or predicted, particularly diabetes; kidney diseases; diseases of the circulatory system; and mental, behavioral, and neurodevelopmental disorders. Demographics, symptoms, procedures, laboratory test results, diagnoses, medications, and BMI were frequently used EHR data in basic recurrent neural network or long short-term memory techniques. By developing and comparing ML and DL models, medical insights such as a high diagnostic performance, an earlier detection, the most important predictors, and additional health indicators were obtained. A clinical benefit that has been evaluated positively was preliminary screening. If these models are applied in practice, patients might also benefit from personalized health care and prevention, with practical benefits such as workload reduction and policy insights.

CONCLUSIONS

Longitudinal EHRs proved to be helpful for support in health care. Current ML models on EHRs can support the detection of diseases in terms of accuracy and offer preliminary screening benefits. Regarding the prevention of diseases, ML and specifically DL models can accurately predict or detect diseases earlier than current clinical diagnoses. Adding personally responsible factors allows targeted prevention interventions. While ML models based on textual EHRs are still in the developmental stage, they have high potential to support clinicians and the health care system and improve patient outcomes.

Machine learning in diagnostic support in medical emergency departments

Diagnosing patients in the medical emergency department is complex and this is expected to increase in many countries due to an ageing population. In this study we investigate the feasibility of training machine learning algorithms to assist physicians handling the complex situation in the medical emergency departments. This is expected to reduce diagnostic errors and improve patient logistics and outcome. We included a total of 9,190 consecutive patient admissions diagnosed and treated in two hospitals in this cohort study. Patients had a biochemical workup including blood and urine analyses on clinical decision totaling 260 analyses. After adding nurse-registered data we trained 19 machine learning algorithms on a random 80% sample of the patients and validated the results on the remaining 20%. We trained algorithms for 19 different patient outcomes including the main outcomes death in 7 (Area under the Curve (AUC) 91.4%) and 30 days (AUC 91.3%) and safe-discharge(AUC 87.3%). The various algorithms obtained areas under the Receiver Operating Characteristics -curves in the range of 71.8-96.3% in the holdout cohort (68.3-98.2% in the training cohort). Performing this list of biochemical analyses at admission also reduced the number of subsequent venipunctures within 24 h from patient admittance by 22%. We have shown that it is possible to develop a list of machine-learning algorithms with high AUC for use in medical emergency departments. Moreover, the study showed that it is possible to reduce the number of venipunctures in this cohort.

Frequency distribution of health disorders in primary care-its consistency and meaning for diagnostics and nomenclature

RN Braun observed that frequencies of health disorders in general practice are so consistent that he called his discovery "Case Distribution Law". Our study compares morbidity data from methodologically similar surveys in primary care practices over a period of fifty years. Frequency ranks were determined for each observation period and the first 150 ranks were compared with Spearman's correlation coefficients. All correlations were consistently positive. Frequency ranks were strikingly similar for surveys carried out at approximately the same time, especially when nomenclatural matching had been carried out before data collection. Ranks were also very similar where clear disease classifications were possible, but less so for non-specific symptoms.The consistency of the distribution of health disorders helps develop diagnostic strategies (diagnostic protocols) and appropriate labeling for non-specific, diagnostically open symptom classifications. According to Braun's considerations, the regularity of case distribution plays an important role in the professionalization of primary care.

Effects of interacting with a large language model compared with a human coach on the clinical diagnostic process and outcomes among fourth-year medical students: study protocol for a prospective, randomised experiment using patient vignettes

INTRODUCTION

Versatile large language models (LLMs) have the potential to augment diagnostic decision-making by assisting diagnosticians, thanks to their ability to engage in open-ended, natural conversations and their comprehensive knowledge access. Yet the novelty of LLMs in diagnostic decision-making introduces uncertainties regarding their impact. Clinicians unfamiliar with the use of LLMs in their professional context may rely on general attitudes towards LLMs more broadly, potentially hindering thoughtful use and critical evaluation of their input, leading to either over-reliance and lack of critical thinking or an unwillingness to use LLMs as diagnostic aids. To address these concerns, this study examines the influence on the diagnostic process and outcomes of interacting with an LLM compared with a human coach, and of prior training vs no training for interacting with either of these 'coaches'. Our findings aim to illuminate the potential benefits and risks of employing artificial intelligence (AI) in diagnostic decision-making.

METHODS AND ANALYSIS

We are conducting a prospective, randomised experiment with N=158 fourth-year medical students from Charité Medical School, Berlin, Germany. Participants are asked to diagnose patient vignettes after being assigned to either a human coach or ChatGPT and after either training or no training (both between-subject factors). We are specifically collecting data on the effects of using either of these 'coaches' and of additional training on information search, number of hypotheses entertained, diagnostic accuracy and confidence. Statistical methods will include linear mixed effects models. Exploratory analyses of the interaction patterns and attitudes towards AI will also generate more generalisable knowledge about the role of AI in medicine.

ETHICS AND DISSEMINATION

The Bern Cantonal Ethics Committee considered the study exempt from full ethical review (BASEC No: Req-2023-01396). All methods will be conducted in accordance with relevant guidelines and regulations. Participation is voluntary and informed consent will be obtained. Results will be published in peer-reviewed scientific medical journals. Authorship will be determined according to the International Committee of Medical Journal Editors guidelines.

Emergency Department Triage Chief Complaints Among Children Evaluated for Physical Abuse Concerns

OBJECTIVES

The aims of this study were to describe chief complaints provided at emergency department triage for young children ultimately given a diagnosed with injuries concerning for physical abuse and compare chief complaints by hospital child protection team assessment (abuse most likely, accident most likely, undetermined) among children younger than 2 years who were the subject of a report to child protective services.

METHODS

This is a retrospective review of children evaluated by the child protection team at an urban children's hospital over a 5-year period. Children younger than 2 years who were the subject of a report to child protective services for suspected physical abuse were included. Chief complaints noted in emergency department triage notes were categorized as follows: 1, medical sign or symptom; 2, accidental trauma incident; 3, identified injury; 4, concern for abuse; or 5, multiple unrelated complaints. Child protection team assessments were categorized as follows: 1, abuse most likely; 2, accident most likely; or 3, undetermined. We used descriptive statistics and tests of association (χ 2 , Fisher exact, Kruskal-Wallis).

RESULTS

Median age of the 422 children included was 4.9 months. Child protection team assessment was abuse most likely in 44%, accident most likely in 23%, and undetermined in 34%. Chief complaints in the overall sample were 39% medical, 29% trauma incident, 16% injury, 10% abuse concern, and 6% multiple unrelated. When the abuse most likely and accident most likely groups were compared, medical chief complaints were more common in the former (47% vs 19%, P < 0.001), whereas trauma incident chief complaints were more common in the latter (19% vs 64%, P < 0.001). Most common medical complaints in the abuse most likely group were altered mental status, abnormal limb use, swelling, pain, apnea, and vomiting.

CONCLUSION

Many children found to have injuries concerning for abuse (47%) present without mention of trauma, injury, or abuse concern as part of the chief complaint. Our findings suggest important topics to include in training physicians about recognition of abuse.

A Nordic Perspective on Patient Online Record Access and the European Health Data Space

The Nordic countries are, together with the United States, forerunners in online record access (ORA), which has now become widespread. The importance of accessible and structured health data has also been highlighted by policy makers internationally. To ensure the full realization of ORA's potential in the short and long term, there is a pressing need to study ORA from a cross-disciplinary, clinical, humanistic, and social sciences perspective that looks beyond strictly technical aspects. In this viewpoint paper, we explore the policy changes in the European Health Data Space (EHDS) proposal to advance ORA across the European Union, informed by our research in a Nordic-led project that carries out the first of its kind, large-scale international investigation of patients' ORA-NORDeHEALTH (Nordic eHealth for Patients: Benchmarking and Developing for the Future). We argue that the EHDS proposal will pave the way for patients to access and control third-party access to their electronic health records. In our analysis of the proposal, we have identified five key principles for ORA: (1) the right to access, (2) proxy access, (3) patient input of their own data, (4) error and omission rectification, and (5) access control. ORA implementation today is fragmented throughout Europe, and the EHDS proposal aims to ensure all European citizens have equal online access to their health data. However, we argue that in order to implement the EHDS, we need more research evidence on the key ORA principles we have identified in our analysis. Results from the NORDeHEALTH project provide some of that evidence, but we have also identified important knowledge gaps that still need further exploration.

MedT5SQL: a transformers-based large language model for text-to-SQL conversion in the healthcare domain

Introduction

In response to the increasing prevalence of electronic medical records (EMRs) stored in databases, healthcare staff are encountering difficulties retrieving these records due to their limited technical expertise in database operations. As these records are crucial for delivering appropriate medical care, there is a need for an accessible method for healthcare staff to access EMRs.

Methods

To address this, natural language processing (NLP) for Text-to-SQL has emerged as a solution, enabling non-technical users to generate SQL queries using natural language text. This research assesses existing work on Text-to-SQL conversion and proposes the MedT5SQL model specifically designed for EMR retrieval. The proposed model utilizes the Text-to-Text Transfer Transformer (T5) model, a Large Language Model (LLM) commonly used in various text-based NLP tasks. The model is fine-tuned on the MIMICSQL dataset, the first Text-to-SQL dataset for the healthcare domain. Performance evaluation involves benchmarking the MedT5SQL model on two optimizers, varying numbers of training epochs, and using two datasets, MIMICSQL and WikiSQL.

Results

For MIMICSQL dataset, the model demonstrates considerable effectiveness in generating question-SQL pairs achieving accuracy of 80.63%, 98.937%, and 90% for exact match accuracy matrix, approximate string-matching, and manual evaluation, respectively. When testing the performance of the model on WikiSQL dataset, the model demonstrates efficiency in generating SQL queries, with an accuracy of 44.2% on WikiSQL and 94.26% for approximate string-matching.

Discussion

Results indicate improved performance with increased training epochs. This work highlights the potential of fine-tuned T5 model to convert medical-related questions written in natural language to Structured Query Language (SQL) in healthcare domain, providing a foundation for future research in this area.

Correlations and discrepancies between cardiac ultrasound, clinical diagnosis and the autopsy findings in early deceased patients with suspected cardiovascular emergencies

Cardiac ultrasound (CUS), either focused cardiac ultrasound (FoCUS) or emergency echocardiography, is frequently used in cardiovascular (CV) emergencies. We assessed correlations and discrepancies between CUS, clinical diagnosis and the autopsy findings in early deceased patients with suspected CV emergencies. We retrospectively analysed clinical and autopsy data of 131 consecutive patients who died within 24 h of hospital admission. The type of CUS and its findings were analysed in relation to the clinical and autopsy diagnoses. CUS was performed in 58% of patients - FoCUS in 83%, emergency echocardiography in 12%, and both types of CUS in 5% of cases. CUS was performed more frequently in patients without a history of CV disease (64 vs. 40%, p = 0.08) and when the time between admission and death was longer (6 vs. 2 h, p = 0.021). In 7% of patients, CUS was inconclusive. In 10% of patients, the ante-mortem cause of death could not be determined, while discrepancies between the clinical and post-mortem diagnosis were found in 26% of cases. In the multivariate logistic regression model, only conclusive CUS [odds ratio (OR) 2.76, 95% confidence interval (CI) 1.30-7.39, p = 0.044] and chest pain at presentation (OR 30.19, 95%CI 5.65 -161.22, p < 0.001) were independently associated with congruent clinical and autopsy diagnosis. In a tertiary university hospital, FoCUS was used more frequently than emergency echocardiography in critically ill patients with suspected cardiac emergencies. Chest pain at presentation and a conclusive CUS were associated with concordant clinical and autopsy diagnoses.

Exploring Clinical Lessons Learned by Experienced Hospitalists from Diagnostic Errors and Successes

BACKGROUND

Diagnostic errors cause significant patient harm. The clinician's ultimate goal is to achieve diagnostic excellence in order to serve patients safely. This can be accomplished by learning from both errors and successes in patient care. However, the extent to which clinicians grow and navigate diagnostic errors and successes in patient care is poorly understood. Clinically experienced hospitalists, who have cared for numerous acutely ill patients, should have great insights from their successes and mistakes to inform others striving for excellence in patient care.

OBJECTIVE

To identify and characterize clinical lessons learned by experienced hospitalists from diagnostic errors and successes.

DESIGN

A semi-structured interview guide was used to collect qualitative data from hospitalists at five independently administered hospitals in the Mid-Atlantic area from February to June 2022.

PARTICIPANTS

12 academic and 12 community-based hospitalists with ≥ 5 years of clinical experience.

APPROACH

A constructivist qualitative approach was used and "reflexive thematic analysis" of interview transcripts was conducted to identify themes and patterns of meaning across the dataset.

RESULTS

Five themes were generated from the data based on clinical lessons learned by hospitalists from diagnostic errors and successes. The ideas included appreciating excellence in clinical reasoning as a core skill, connecting with patients and other members of the health care team to be able to tap into their insights, reflecting on the diagnostic process, committing to growth, and prioritizing self-care.

CONCLUSIONS

The study identifies key lessons learned from the errors and successes encountered in patient care by clinically experienced hospitalists. These findings may prove helpful for individuals and groups that are authentically committed to moving along the continuum from diagnostic competence towards excellence.

Longitudinal Changes in Diagnostic Accuracy of a Differential Diagnosis List Developed by an AI-Based Symptom Checker: Retrospective Observational Study

BACKGROUND

Artificial intelligence (AI) symptom checker models should be trained using real-world patient data to improve their diagnostic accuracy. Given that AI-based symptom checkers are currently used in clinical practice, their performance should improve over time. However, longitudinal evaluations of the diagnostic accuracy of these symptom checkers are limited.

OBJECTIVE

This study aimed to assess the longitudinal changes in the accuracy of differential diagnosis lists created by an AI-based symptom checker used in the real world.

METHODS

This was a single-center, retrospective, observational study. Patients who visited an outpatient clinic without an appointment between May 1, 2019, and April 30, 2022, and who were admitted to a community hospital in Japan within 30 days of their index visit were considered eligible. We only included patients who underwent an AI-based symptom checkup at the index visit, and the diagnosis was finally confirmed during follow-up. Final diagnoses were categorized as common or uncommon, and all cases were categorized as typical or atypical. The primary outcome measure was the accuracy of the differential diagnosis list created by the AI-based symptom checker, defined as the final diagnosis in a list of 10 differential diagnoses created by the symptom checker. To assess the change in the symptom checker's diagnostic accuracy over 3 years, we used a chi-square test to compare the primary outcome over 3 periods: from May 1, 2019, to April 30, 2020 (first year); from May 1, 2020, to April 30, 2021 (second year); and from May 1, 2021, to April 30, 2022 (third year).

RESULTS

A total of 381 patients were included. Common diseases comprised 257 (67.5%) cases, and typical presentations were observed in 298 (78.2%) cases. Overall, the accuracy of the differential diagnosis list created by the AI-based symptom checker was 172 (45.1%), which did not differ across the 3 years (first year: 97/219, 44.3%; second year: 32/72, 44.4%; and third year: 43/90, 47.7%; P=.85). The accuracy of the differential diagnosis list created by the symptom checker was low in those with uncommon diseases (30/124, 24.2%) and atypical presentations (12/83, 14.5%). In the multivariate logistic regression model, common disease (P<.001; odds ratio 4.13, 95% CI 2.50-6.98) and typical presentation (P<.001; odds ratio 6.92, 95% CI 3.62-14.2) were significantly associated with the accuracy of the differential diagnosis list created by the symptom checker.

CONCLUSIONS

A 3-year longitudinal survey of the diagnostic accuracy of differential diagnosis lists developed by an AI-based symptom checker, which has been implemented in real-world clinical practice settings, showed no improvement over time. Uncommon diseases and atypical presentations were independently associated with a lower diagnostic accuracy. In the future, symptom checkers should be trained to recognize uncommon conditions.

Effect of contextual factors on the prevalence of diagnostic errors among patients managed by physicians of the same specialty: a single-centre retrospective observational study

BACKGROUND

There has been growing recognition that contextual factors influence the physician's cognitive processes. However, given that cognitive processes may depend on the physicians' specialties, the effects of contextual factors on diagnostic errors reported in previous studies could be confounded by difference in physicians.

OBJECTIVE

This study aimed to clarify whether contextual factors such as location and consultation type affect diagnostic accuracy.

METHODS

We reviewed the medical records of 1992 consecutive outpatients consulted by physicians from the Department of Diagnostic and Generalist Medicine in a university hospital between 1 January and 31 December 2019. Diagnostic processes were assessed using the Revised Safer Dx Instrument. Patients were categorised into three groups according to contextual factors (location and consultation type): (1) referred patients with scheduled visit to the outpatient department; (2) patients with urgent visit to the outpatient department; and (3) patients with emergency visit to the emergency room. The effect of the contextual factors on the prevalence of diagnostic errors was investigated using logistic regression analysis.

RESULTS

Diagnostic errors were observed in 12 of 534 referred patients with scheduled visit to the outpatient department (2.2%), 3 of 599 patients with urgent visit to the outpatient department (0.5%) and 13 of 859 patients with emergency visit to the emergency room (1.5%). Multivariable logistic regression analysis showed a significantly higher prevalence of diagnostic errors in referred patients with scheduled visit to the outpatient department than in patients with urgent visit to the outpatient department (OR 4.08, p=0.03), but no difference between patients with emergency and urgent visit to the emergency room and outpatient department, respectively.

CONCLUSION

Contextual factors such as consultation type may affect diagnostic errors; however, since the differences in the prevalence of diagnostic errors were small, the effect of contextual factors on diagnostic accuracy may be small in physicians working in different care settings.

Influence of a Large Language Model on Diagnostic Reasoning: A Randomized Clinical Vignette Study

Importance

Diagnostic errors are common and cause significant morbidity. Large language models (LLMs) have shown promise in their performance on both multiple-choice and open-ended medical reasoning examinations, but it remains unknown whether the use of such tools improves diagnostic reasoning.

Objective

To assess the impact of the GPT-4 LLM on physicians' diagnostic reasoning compared to conventional resources.

Design

Multi-center, randomized clinical vignette study.

Setting

The study was conducted using remote video conferencing with physicians across the country and in-person participation across multiple academic medical institutions.

Participants

Resident and attending physicians with training in family medicine, internal medicine, or emergency medicine.

Interventions

Participants were randomized to access GPT-4 in addition to conventional diagnostic resources or to just conventional resources. They were allocated 60 minutes to review up to six clinical vignettes adapted from established diagnostic reasoning exams.

Main Outcomes and Measures

The primary outcome was diagnostic performance based on differential diagnosis accuracy, appropriateness of supporting and opposing factors, and next diagnostic evaluation steps. Secondary outcomes included time spent per case and final diagnosis.

Results

50 physicians (26 attendings, 24 residents) participated, with an average of 5.2 cases completed per participant. The median diagnostic reasoning score per case was 76.3 percent (IQR 65.8 to 86.8) for the GPT-4 group and 73.7 percent (IQR 63.2 to 84.2) for the conventional resources group, with an adjusted difference of 1.6 percentage points (95% CI -4.4 to 7.6; p=0.60). The median time spent on cases for the GPT-4 group was 519 seconds (IQR 371 to 668 seconds), compared to 565 seconds (IQR 456 to 788 seconds) for the conventional resources group, with a time difference of -82 seconds (95% CI -195 to 31; p=0.20). GPT-4 alone scored 15.5 percentage points (95% CI 1.5 to 29, p=0.03) higher than the conventional resources group.

Conclusions and Relevance

In a clinical vignette-based study, the availability of GPT-4 to physicians as a diagnostic aid did not significantly improve clinical reasoning compared to conventional resources, although it may improve components of clinical reasoning such as efficiency. GPT-4 alone demonstrated higher performance than both physician groups, suggesting opportunities for further improvement in physician-AI collaboration in clinical practice.

Before and after COVID-19: Changes in symptoms and diagnoses in 13,033 adults

BACKGROUND

Most patients with COVID-19 report experiencing one or more symptoms after acute infection subsides, known as post-acute sequelae of SARS-CoV-2 infection (PASC). Though research has examined PASC after acute COVID-19, few studies have examined PASC over a longer follow-up duration or accounted for rates of symptoms and diagnoses before COVID-19 infection, and included those not actively seeking treatment for PASC. To determine what symptoms and diagnoses are occurring at higher rates after acute COVID-19 infection from a more inclusive sample, we extracted electronic hospital records (EHR) data from 13,033 adults with previously known diagnoses and symptoms.

METHODS

The sample was comprised of patients who had a positive PCR test for SARS-CoV-2 between March 1, 2020, and December 31, 2020, and follow-up was conducted through November 29, 2021. All patients in the sample had medical appointments ≥4 weeks before and ≥4 weeks after their positive PCR test. At these appointments, all ICD-10 codes recorded in the EHR were classified into 21 categories based on the literature and expert review. Conditional logistic regression models were used to quantify the odds of these symptoms and diagnostic categories following COVID-19 infection relative to visits occurring before infection. The sample was comprised of 28.0% adults over 65 and was 57.0% female. After the positive PCR test, the most recorded diagnoses and symptoms were dyspnea and respiratory failure, myositis, musculoskeletal pain/stiffness, anxiety, and depression.

RESULTS

Results from regression analyses showed increased odds of diagnosis for 15 of the 21 categories following positive PCR. Relative to pre-COVID, the diagnoses and symptoms with the greatest odds after a positive PCR test were loss of smell or taste [OR (95% CI) = 6.20 (3.18-12.09)], pulmonary fibrosis [3.50 (1.59-7.68)], and dyspnea/respiratory failure [2.14 (1.92-2.40)]. Stratification of these analyses by age, gender, race, and ethnicity showed similar results.

CONCLUSION

The increased symptoms and diagnoses detected in the current study match prior analyses of PASC diagnosis and treatment-seeking patients. The current research expands upon the literature by showing that these symptoms are more frequently detected following acute COVID-19 than before COVID-19. Further, our analyses provide a broad snapshot of the population as we were able to describe PASC among all patients who tested positive for COVID-19.

Navigating Pathways to Diagnosis in Idiopathic Subglottic Stenosis: A Qualitative Study

OBJECTIVE

Idiopathic subglottic stenosis is a rare disease, and time to diagnosis is often prolonged. In the United States, some estimate it takes an average of 9 years for patients with similar rare disease to be diagnosed. Patient experience during this period is termed the diagnostic odyssey. The aim of this study is to use qualitative methods grounded in behavioral-ecological conceptual frameworks to identify drivers of diagnostic odyssey length that can help inform efforts to improve health care for iSGS patients.

METHODS

Qualitative study using semi-structured interviews. Setting consisted of participants who were recruited from those enrolled in a large, prospective multicenter trial. We use directed content analysis to analyze qualitative semi-structured interviews with iSGS patients focusing on their pathways to diagnosis.

RESULTS

Overall, 30 patients with iSGS underwent semi-structured interviews. The patient-reported median time to diagnosis was 21 months. On average, the participants visited four different health care providers. Specialists were most likely to make an appropriate referral to otolaryngology that ended in diagnosis. However, when primary care providers referred to otolaryngology, patients experienced a shorter diagnostic odyssey. The most important behavioral-ecological factors in accelerating diagnosis were strong social support for the patient and providers' willingness to refer.

CONCLUSION

Several factors affected time to diagnosis for iSGS patients. Patient social capital was a catalyst in decreasing time to diagnosis. Patient-reported medical paternalism and gatekeeping limited specialty care referrals extended diagnostic odysseys. Additional research is needed to understand the effect of patient-provider and provider-provider relationships on time to diagnosis for patients with iSGS.

LEVEL OF EVIDENCE

4 Laryngoscope, 134:815-824, 2024.

Burden of serious harms from diagnostic error in the USA

BACKGROUND

Diagnostic errors cause substantial preventable harms worldwide, but rigorous estimates for total burden are lacking. We previously estimated diagnostic error and serious harm rates for key dangerous diseases in major disease categories and validated plausible ranges using clinical experts.

OBJECTIVE

We sought to estimate the annual US burden of serious misdiagnosis-related harms (permanent morbidity, mortality) by combining prior results with rigorous estimates of disease incidence.

METHODS

Cross-sectional analysis of US-based nationally representative observational data. We estimated annual incident vascular events and infections from 21.5 million (M) sampled US hospital discharges (2012-2014). Annual new cancers were taken from US-based registries (2014). Years were selected for coding consistency with prior literature. Disease-specific incidences for 15 major vascular events, infections and cancers ('Big Three' categories) were multiplied by literature-based rates to derive diagnostic errors and serious harms. We calculated uncertainty estimates using Monte Carlo simulations. Validity checks included sensitivity analyses and comparison with prior published estimates.

RESULTS

Annual US incidence was 6.0 M vascular events, 6.2 M infections and 1.5 M cancers. Per 'Big Three' dangerous disease case, weighted mean error and serious harm rates were 11.1% and 4.4%, respectively. Extrapolating to all diseases (including non-'Big Three' dangerous disease categories), we estimated total serious harms annually in the USA to be 795 000 (plausible range 598 000-1 023 000). Sensitivity analyses using more conservative assumptions estimated 549 000 serious harms. Results were compatible with setting-specific serious harm estimates from inpatient, emergency department and ambulatory care. The 15 dangerous diseases accounted for 50.7% of total serious harms and the top 5 (stroke, sepsis, pneumonia, venous thromboembolism and lung cancer) accounted for 38.7%.

CONCLUSION

An estimated 795 000 Americans become permanently disabled or die annually across care settings because dangerous diseases are misdiagnosed. Just 15 diseases account for about half of all serious harms, so the problem may be more tractable than previously imagined.

A Hybrid Model for 30-Day Syncope Prognosis Prediction in the Emergency Department

Syncope is a challenging problem in the emergency department (ED) as the available risk prediction tools have suboptimal predictive performances. Predictive models based on machine learning (ML) are promising tools whose application in the context of syncope remains underexplored. The aim of the present study was to develop and compare the performance of ML-based models in predicting the risk of clinically significant outcomes in patients presenting to the ED for syncope. We enrolled 266 consecutive patients (age 73, IQR 58-83; 52% males) admitted for syncope at three tertiary centers. We collected demographic and clinical information as well as the occurrence of clinically significant outcomes at a 30-day telephone follow-up. We implemented an XGBoost model based on the best-performing candidate predictors. Subsequently, we integrated the XGboost predictors with knowledge-based rules. The obtained hybrid model outperformed the XGboost model (AUC = 0.81 vs. 0.73, p < 0.001) with acceptable calibration. In conclusion, we developed an ML-based model characterized by a commendable capability to predict adverse events within 30 days post-syncope evaluation in the ED. This model relies solely on clinical data routinely collected during a patient's initial syncope evaluation, thus obviating the need for laboratory tests or syncope experienced clinical judgment.

Insights into diagnostic errors in endocrinology: a prospective, case-based, international study

BACKGROUND

Diagnostic errors in internal medicine are common. While cognitive errors have previously been identified to be the most common contributor to errors, very little is known about errors in specific fields of internal medicine such as endocrinology. This prospective, multicenter study focused on better understanding the causes of diagnostic errors made by general practitioners and internal specialists in the area of endocrinology.

METHODS

From August 2019 until January 2020, 24 physicians completed five endocrine cases on an online platform that simulated the diagnostic process. After each case, the participants had to state and explain why they chose their assumed diagnosis. The data gathering process as well as the participants' explanations were quantitatively and qualitatively analyzed to determine the causes of the errors. The diagnostic processes in correctly and incorrectly solved cases were compared.

RESULTS

Seven different causes of diagnostic error were identified, the most frequent being misidentification (mistaking one diagnosis with a related one or with more frequent and similar diseases) in 23% of the cases. Other causes were faulty context generation (21%) and premature closure (17%). The diagnostic confidence did not differ between correctly and incorrectly solved cases (median 8 out of 10, p = 0.24). However, in incorrectly solved cases, physicians spent less time on the technical findings (such as lab results, imaging) (median 250 s versus 199 s, p < 0.049).

CONCLUSIONS

The causes for errors in endocrine case scenarios are similar to the causes in other fields of internal medicine. Spending more time on technical findings might prevent misdiagnoses in everyday clinical practice.

Internal medicine intern performance on the gastrointestinal physical exam

OBJECTIVES

The gastrointestinal (GI) physical exam provides critical information about underlying disease states. However, since assessment of physical examination skills is rarely conducted as part of internal medicine residency training, little is known about resident performance on the GI physical exam.

METHODS

During a clinical skills assessment that took place between November 2019 and February 2020, internal medicine interns examined the same patient with chronic liver disease while being observed by faculty preceptors. We compared the exam maneuvers performed with those expected by the faculty evaluators. We noted which maneuvers were performed incorrectly, whether physical exam technique correlated with identification of physical exam findings, and if performance on the physical exam was associated with building an appropriate differential diagnosis. This four-hour assessment was required for internal medicine interns within two different residency programs in the Baltimore area.

RESULTS

More than half of the 29 participating interns (n=17, 58.6 %) received a "needs improvement" score on their physical exam technique. Technique was highly correlated with identifying the correct physical signs (r=0.88, p<0.0001). The most commonly excluded maneuvers were assessing for splenomegaly and hepatomegaly. The most commonly missed findings were splenomegaly and hepatomegaly. Most interns included chronic liver disease as part of their differential diagnosis even if they received "needs improvement" scores on physical exam technique or identifying physical signs.

CONCLUSIONS

Internal medicine interns would benefit from learning an organized approach to the gastrointestinal exam. This would likely lead to increased identification of important gastrointestinal findings.

Diagnostic errors in uncommon conditions: a systematic review of case reports of diagnostic errors

OBJECTIVES

To assess the usefulness of case reports as sources for research on diagnostic errors in uncommon diseases and atypical presentations.

CONTENT

We reviewed 563 case reports of diagnostic error. The commonality of the final diagnoses was classified based on the description in the articles, Orphanet, or epidemiological data on available references; the typicality of presentation was classified based on the description in the articles and the judgment of the physician researchers. Diagnosis Error Evaluation and Research (DEER), Reliable Diagnosis Challenges (RDC), and Generic Diagnostic Pitfalls (GDP) taxonomies were used to assess the factors contributing to diagnostic errors.

SUMMARY AND OUTLOOK

Excluding three cases in that commonality could not be classified, 560 cases were classified into four categories: typical presentations of common diseases (60, 10.7 %), atypical presentations of common diseases (35, 6.2 %), typical presentations of uncommon diseases (276, 49.3 %), and atypical presentations of uncommon diseases (189, 33.8 %). The most important DEER taxonomy was "Failure/delay in considering the diagnosis" among the four categories, whereas the most important RDC and GDP taxonomies varied with the categories. Case reports can be a useful data source for research on the diagnostic errors of uncommon diseases with or without atypical presentations.

Cognitive bias during clinical decision-making and its influence on patient outcomes in the emergency department: A scoping review

BACKGROUND

An integral part of clinical practice is decision-making. Yet there is widespread acceptance that there is evidence of cognitive bias within clinical practice among nurses and physicians. However, how cognitive bias among emergency nurses and physicians' decision-making influences patient outcomes remains unclear.

AIM

The aim of this review was to systematically synthesise research exploring the emergency nurses' and physicians' cognitive bias in decision-making and its influence on patient outcomes.

METHODS

This scoping review was guided by the PRISMA Extension for Scoping Reviews. The databases searched included CINAHL, MEDLINE, Web of Science and PubMed. No date limits were applied. The Patterns, Advances, Gaps, Evidence for practice and Research recommendation (PAGER) framework was used to guide the discussion.

RESULTS

The review included 18 articles, consisting of 10 primary studies (nine quantitative and one qualitative) and eight literature reviews. Of the 18 articles, nine investigated physicians, five articles examined nurses, and four both physicians and nurses with sample sizes ranging from 13 to 3547. Six primary studies were cross-sectional and five used hypothetical scenarios, and one real-world assessment. Three were experimental studies. Twenty-nine cognitive biases were identified with Implicit bias (n = 12) most frequently explored, followed by outcome bias (n = 4). Results were inconclusive regarding the influence of biases on treatment decisions and patient outcomes. Four key themes were identified; (i) cognitive biases among emergency clinicians; (ii) measurement of cognitive bias; (iii) influence of cognitive bias on clinical decision-making; and (iv) association between emergency clinicians' cognitive bias and patient outcomes.

CONCLUSIONS

This review identified that cognitive biases were present among emergency nurses and physicians during clinical decision-making, but it remains unclear how cognitive bias influences patient outcomes. Further research examining emergency clinicians' cognitive bias is required.

RELEVANCE TO CLINICAL PRACTICE

Awareness of emergency clinicians' own cognitive biases may result to the provision of equity in care.

NO PATIENT OR PUBLIC CONTRIBUTION IN THIS REVIEW

We intend to disseminate the results through publication in a peer-reviewed journals and conference presentations.

Associations Between Hospitalist Shift Busyness, Diagnostic Confidence, and Resource Utilization: A Pilot Study

OBJECTIVES

Hospitalized patients are at risk for diagnostic errors. Hospitalists caring for these patients are often multitasking when overseeing patient care. We aimed to measure hospitalist workload and understand its influences on diagnostic performance in a real-world clinical setting.

METHODS

We conducted a single-center, prospective, pilot observational study of hospitalists admitting new patients to the hospital. Hospitalists completed an abridged Mindful Attention Awareness Tool and a survey about diagnostic confidence at shift completion. Data on differential diagnoses and resource utilization (e.g., laboratory, imaging tests ordered, and consultations) were collected from the medical record. The number of admissions and paging volume per shift were used as separate proxies for shift busyness. Data were analyzed using linear mixed effects models (continuous outcomes) or mixed effects logistic regression (dichotomous outcomes).

RESULTS

Of the 53 hospitalists approached, 47 (89%) agreed to participate; complete data were available for 37 unique hospitalists who admitted 160 unique patients. Increases in admissions (odds ratio, 1.99; 95% confidence interval [CI], 1.04 to 3.82; P = 0.04) and pages (odds ratio, 1.11; 95% CI, 1.02 to 1.21; P = 0.01) were associated with increased odds of hospitalists finding it "difficult to focus on what is happening in the present." Increased pages was associated with a decrease in the number of listed differential diagnoses (coefficient, -0.02; 95% CI, -0.04 to -0.003; P = 0.02).

CONCLUSIONS

Evaluation of hospitalist busyness and its associations with factors that may influence diagnosis in a real-world environment was feasible and demonstrated important implications on physician focus and differential diagnosis.

Diagnostic uncertainty in patients, parents, and physicians: a compensatory control theory perspective

Medical diagnoses offer a structure by which psychological uncertainty can be attenuated, allowing patients to diminish psychological threats and focus on health prognosis. Yet when no diagnosis can be made, patients may experience diagnostic uncertainty - perceiving the medical field as unable to provide an accurate explanation of the cause of their health problems. This review examines the psychological threat that diagnostic uncertainty imposes on individuals' need for control and understanding, and the resulting consequences experienced by patients, parents of pediatric patients, and physicians. Using compensatory control theory as a framework, we propose a taxonomy of behaviors that people may adopt in order to regain control in the face of diagnostic uncertainty and to reaffirm that the world is not random and chaotic. To manage diagnostic uncertainty, people may bolster their personal agency, affiliate with external systems they see as acting in their interest, affirm clear connections between behaviors and outcomes, and affirm nonspecific epistemic structure. Diagnostic uncertainty is approached from the perspectives of patients, parents of pediatric patients, and physicians, demonstrating how each group responds in order to maintain a sense that the world has structure and is not random. Discussion centers on moderators, limitations, and implications for clinical practice.

Comparative Perspectives on Diagnostic Error Discussions Between Inpatient and Outpatient Pediatric Providers

Diagnostic error remains understudied and underaddressed despite causing significant morbidity and mortality. One barrier to addressing this issue remains provider discomfort. Survey studies have shown significantly more discomfort among providers in discussing diagnostic error compared with other forms of error. Whether the comfort in discussing diagnostic error differs depending on practice setting has not been previously studied. The objective of this study was to assess differences in provider willingness to discuss diagnostic error in the inpatient versus outpatient setting. A multicenter survey was sent out to 3881 providers between May and June 2018. This survey was designed to assess comfort level of discussing diagnostic error and looking at barriers to discussing diagnostic error. Forty-three percent versus 22% of inpatient versus outpatient providers (P = 0.004) were comfortable discussing short-term diagnostic error publicly. Similarly, 76% versus 60% of inpatient versus outpatient providers (P = 0.010) were comfortable discussing short-term diagnostic error privately. A higher percentage of inpatient (64%) compared with outpatient providers (46%) (P = 0.043) were comfortable discussing long-term diagnostic error privately. Forty percent versus 24% of inpatient versus outpatient providers (P = 0.018) were comfortable discussing long-term error publicly. No difference in barriers cited depending on practice setting. Inpatient providers are more comfortable discussing diagnostic error than their outpatient counterparts. More study is needed to determine the etiology of this discrepancy and to develop strategies to increase outpatient provider comfort.

Diagnostic Performance, Triage Safety, and Usability of a Clinical Decision Support System Within a University Hospital Emergency Department: Algorithm Performance and Usability Study

Background

Computerized clinical decision support systems (CDSSs) are increasingly adopted in health care to optimize resources and streamline patient flow. However, they often lack scientific validation against standard medical care.

Objective

The purpose of this study was to assess the performance, safety, and usability of a CDSS in a university hospital emergency department setting in Kuopio, Finland.

Methods

Patients entering the emergency department were asked to voluntarily participate in this study. Patients aged 17 years or younger, patients with cognitive impairments, and patients who entered the unit in an ambulance or with the need for immediate care were excluded. Patients completed the CDSS web-based form and usability questionnaire when waiting for the triage nurse's evaluation. The CDSS data were anonymized and did not affect the patients' usual evaluation or treatment. Retrospectively, 2 medical doctors evaluated the urgency of each patient's condition by using the triage nurse's information, and urgent and nonurgent groups were created. The International Statistical Classification of Diseases, Tenth Revision diagnoses were collected from the electronic health records. Usability was assessed by using a positive version of the System Usability Scale questionnaire.

Results

In total, our analyses included 248 patients. Regarding urgency, the mean sensitivities were 85% and 19%, respectively, for urgent and nonurgent cases when assessing the performance of CDSS evaluations in comparison to that of physicians. The mean sensitivities were 85% and 35%, respectively, when comparing the evaluations between the two physicians. Our CDSS did not miss any cases that were evaluated to be emergencies by physicians; thus, all emergency cases evaluated by physicians were evaluated as either urgent cases or emergency cases by the CDSS. In differential diagnosis, the CDSS had an exact match accuracy of 45.5% (97/213). The usability was good, with a mean System Usability Scale score of 78.2 (SD 16.8).

Conclusions

In a university hospital emergency department setting with a large real-world population, our CDSS was found to be equally as sensitive in urgent patient cases as physicians and was found to have an acceptable differential diagnosis accuracy, with good usability. These results suggest that this CDSS can be safely assessed further in a real-world setting. A CDSS could accelerate triage by providing patient-provided data in advance of patients' initial consultations and categorize patient cases as urgent and nonurgent cases upon patients' arrival to the emergency department.

Effects of Combinational Use of Additional Differential Diagnostic Generators on the Diagnostic Accuracy of the Differential Diagnosis List Developed by an Artificial Intelligence-Driven Automated History-Taking System: Pilot Cross-Sectional Study

BACKGROUND

Low diagnostic accuracy is a major concern in automated medical history-taking systems with differential diagnosis (DDx) generators. Extending the concept of collective intelligence to the field of DDx generators such that the accuracy of judgment becomes higher when accepting an integrated diagnosis list from multiple people than when accepting a diagnosis list from a single person may be a possible solution.

OBJECTIVE

The purpose of this study is to assess whether the combined use of several DDx generators improves the diagnostic accuracy of DDx lists.

METHODS

We used medical history data and the top 10 DDx lists (index DDx lists) generated by an artificial intelligence (AI)-driven automated medical history-taking system from 103 patients with confirmed diagnoses. Two research physicians independently created the other top 10 DDx lists (second and third DDx lists) per case by imputing key information into the other 2 DDx generators based on the medical history generated by the automated medical history-taking system without reading the index lists generated by the automated medical history-taking system. We used the McNemar test to assess the improvement in diagnostic accuracy from the index DDx lists to the three types of combined DDx lists: (1) simply combining DDx lists from the index, second, and third lists; (2) creating a new top 10 DDx list using a 1/n weighting rule; and (3) creating new lists with only shared diagnoses among DDx lists from the index, second, and third lists. We treated the data generated by 2 research physicians from the same patient as independent cases. Therefore, the number of cases included in analyses in the case using 2 additional lists was 206 (103 cases × 2 physicians' input).

RESULTS

The diagnostic accuracy of the index lists was 46% (47/103). Diagnostic accuracy was improved by simply combining the other 2 DDx lists (133/206, 65%, P<.001), whereas the other 2 combined DDx lists did not improve the diagnostic accuracy of the DDx lists (106/206, 52%, P=.05 in the collective list with the 1/n weighting rule and 29/206, 14%, P<.001 in the only shared diagnoses among the 3 DDx lists).

CONCLUSIONS

Simply adding each of the top 10 DDx lists from additional DDx generators increased the diagnostic accuracy of the DDx list by approximately 20%, suggesting that the combinational use of DDx generators early in the diagnostic process is beneficial.

Approach to Tendinopathies of the Upper Limb: What Works

Tendinopathies are some of the most common diagnoses treated by hand surgeons. Diagnoses such as trigger digit, de Quervain tenosynovitis, extensor carpi ulnaris tendinitis, and epicondylitis often resolve with nonoperative treatment and/or a single ambulatory procedure. When symptoms persist or worsen after surgery, patients are disappointed and treatment can be challenging. This article reviews practical points in evaluation of such cases, and surgical options that work in revision scenarios.

Lessons in clinical reasoning - pitfalls, myths, and pearls: a case of crushing, substernal chest pain

OBJECTIVES

Diagnostic error is not uncommon and diagnostic accuracy can be improved with the use of problem representation, pre-test probability, and Bayesian analysis for improved clinical reasoning.

CASE PRESENTATION

A 48-year-old female presented as a transfer from another Emergency Department (ED) to our ED with crushing, substernal pain associated with dyspnea, diaphoresis, nausea, and a tingling sensation down both arms with radiation to the back and neck. Troponins were elevated along with an abnormal electrocardiogram. A negative myocardial perfusion scan led to the patient's discharge. The patient presented to the ED 10 days later with an anterior ST-elevation myocardial infarction.

CONCLUSIONS

An overemphasis on a single testing modality led to diagnostic error and a severe event. The use of pre-test probabilities guided by history-taking can lead to improved interpretation of test results, ultimately improving diagnostic accuracy and preventing serious medical errors.

Collaborative Diagnostic Conversations Between Clinicians, Patients, and Their Families: A Way to Avoid Diagnostic Errors

Objective

To identify the components of the collaborative diagnostic conversations between clinicians, patients, and their families and how deficiencies in these conversations can lead to diagnostic errors.

Patients and Methods

We purposively selected 60 video recordings of clinical encounters that included diagnosis conversations. These videos were obtained from the internal medicine, and family medicine services at Mayo Clinic's campus in Rochester, Minnesota. These clinical encounters were recorded between November 2017, and December 2021, during the conduct of studies aiming at developing or testing shared decision-making interventions. We followed a critically reflective approach model for data analysis.

Results

We identified 3 components of diagnostic conversations as follows: (1) recognizing diagnostic situations, (2) setting priorities, and (3) creating and reconciling a diagnostic plan. Deficiencies in diagnostic conversations could lead to framing issues in a way that sets diagnostic activities off in an incorrect or undesirable direction, incorrect prioritization of diagnostic concerns, and diagnostic plans of care that are not feasible, desirable, or productive.

Conclusion

We identified 3 clinician-and-patient diagnostic conversation components and mapped them to potential diagnostic errors. This information may inform additional research to identify areas of intervention to decrease the frequency and harm associated with diagnostic errors in clinical practice.

A New Neurological Screening Approach for Diagnosing Brainstem Infarction Using the Calling Method and Familiar Voices

This report proposes a new approach to assess dysarthria in patients with brainstem infarction by involving familiar individuals. Collaboration provides valuable insights compared to subjective traditional methods. A man in his 70s presented with resolved positional vertigo. Standard neurological tests showed no abnormalities, and inquiries with the patient's friend did not reveal voice changes. While inquiring about voice changes with family, friends, and acquaintances is a common practice in clinical settings, our approach involved the patient calling out to his friend from a distance. Despite the physician detecting no abnormalities, the friend noticed a lower voice. Subsequent magnetic resonance imaging (MRI) confirmed brainstem infarction. Early and subtle symptoms of brainstem infarction pose a detection challenge and can lead to serious outcomes if overlooked. This report provides the first evidence that distance calling can detect subtle voice changes associated with brainstem infarction potentially overlooked by conventional neurological examinations, including inquiries with individuals familiar with the patient's voice. Detecting brainstem infarction in emergency department cases is often missed, but conducting MRIs on every patient is not feasible. This simple method may identify patients overlooked by conventional screening who should undergo neuroimaging such as MRI. Further research is needed, and involving non-professionals in assessments could significantly advance the diagnostic process.

Copy-Pasting in Patients' Electronic Medical Records (EMRs): Use Judiciously and With Caution

An electronic medical record (EMR) is an electronic, comprehensive, and up-to-date compilation of a patient's medical history and information stored in a secure digital format. It provides real-time access to patient data, enabling healthcare providers to make informed decisions quickly and accurately. EMR systems streamline a patient's healthcare journey and enable shared care across the medical practice. By providing a comprehensive view of a patient's medical history, EMRs can be invaluable tools for physicians and healthcare providers, allowing them to collaborate more effectively and provide better care. Additionally, EMRs can help reduce paperwork, improve accuracy, and increase efficiency, ultimately leading to improved patient outcomes. The true potential of EMR systems can be realized when they are used in conjunction with evidence-based medicine methodologies, quality improvement initiatives, and team-based care. This combination of technologies and practices can revolutionize healthcare delivery, improving patient outcomes, greater efficiency, and cost savings. "Copy-pasting" is an essential feature of EMR systems, with physicians relying on it for up to 35.7% of their workflow. By leveraging the copy-pasting feature of their EMR system, physicians can ensure that their data capture is accurate and timely, leading to better patient care. Copy-pasting can be a valuable tool for physicians, saving time and allowing them to focus on practical clinical issues. However, it is essential to note that while most clinicians copy-paste, 25% of them believe it can lead to a high frequency of medical errors, with the potential for a significant number of errors being attributed to this practice. Therefore, physicians must exercise caution when copy-pasting and take the necessary steps to ensure accuracy and reduce the risk of errors. Copy-pasting can cause severe adverse patient events by introducing new inaccuracies, rapidly spreading inaccurate or outdated information, leading to discordant notes, and creating long notes that mask essential clinical information. Despite these risks, copy-pasting has become widely used in EMRs. Additionally, copy-pasting can reduce the time spent on documentation, allowing healthcare providers to focus more on patient care. Inappropriate copy-pasting can have serious consequences, such as compromising data integrity, endangering patient safety, increasing costs, and even leading to fraudulent malpractice claims. In conclusion, copy-pasting can be helpful for healthcare professionals, but it must be used cautiously. Proper education and safeguards should be implemented to ensure accuracy and up-to-date patient data. Additionally, healthcare professionals should be aware of the legal implications of copy-pasting, as it may be considered a form of medical malpractice. With the proper precautions, copy-pasting can be a safe and efficient way to save time and reduce errors in patient records.

Heuristics and biases in medical decision-making under uncertainty: The case of neuropronostication for consciousness disorders

Neuropronostication for consciousness disorders can be very complex and prone to high uncertainty. Despite notable advancements in the development of dedicated scales and physiological markers using innovative paradigms, these technical progressions are often overshadowed by factors intrinsic to the medical environment. Beyond the scarcity of objective data guiding medical decisions, factors like time pressure, fatigue, multitasking, and emotional load can drive clinicians to rely more on heuristic-based clinical reasoning. Such an approach, albeit beneficial under certain circumstances, may lead to systematic error judgments and impair medical decisions, especially in complex and uncertain environments. After a brief review of the main theoretical frameworks, this paper explores the influence of clinicians' cognitive biases on clinical reasoning and decision-making in the challenging context of neuroprognostication for consciousness disorders. The discussion further revolves around developing and implementing various strategies designed to mitigate these biases and their impact, aiming to enhance the quality of care and the patient safety.

Enhancing physicians' radiology diagnostics of COVID-19's effects on lung health by leveraging artificial intelligence

Introduction: This study aimed to develop an individualized artificial intelligence model to help radiologists assess the severity of COVID-19's effects on patients' lung health. Methods: Data was collected from medical records of 1103 patients diagnosed with COVID-19 using RT- qPCR between March and June 2020, in Hospital Madrid-Group (HM-Group, Spain). By using Convolutional Neural Networks, we determine the effects of COVID-19 in terms of lung area, opacities, and pulmonary air density. We then combine these variables with age and sex in a regression model to assess the severity of these conditions with respect to fatality risk (death or ICU). Results: Our model can predict high effect with an AUC of 0.736. Finally, we compare the performance of the model with respect to six physicians' diagnosis, and test for improvements on physicians' performance when using the prediction algorithm. Discussion: We find that the algorithm outperforms physicians (39.5% less error), and thus, physicians can significantly benefit from the information provided by the algorithm by reducing error by almost 30%.

Diagnostic consistency between admission and discharge of pediatric cases in a tertiary teaching hospital in China

BACKGROUND

Patient-centered, high-quality health care relies on accurate and timely diagnosis. Diagnosis is a complex, error-prone process. Prevention of errors involves understanding the cause of errors. This study investigated diagnostic discordance between admission and discharge in pediatric cases.

METHODS

We retrospectively reviewed the electronic medical records of 5381 pediatric inpatients during 2017-2018 in a tertiary teaching hospital. We analyzed diagnostic consistency by comparing the first 4 digits of admission and discharge ICD-10 codes of the cases and classified them as concordant for "complete and partial match" or discordant for "no match".

RESULTS

Diagnostic discordance was observed in 49.2% with the highest prevalence in infections of the nervous and respiratory systems (Ps < 0.001). Multiple (multivariable) logistic regression analysis predicted a lower risk of diagnostic discordance with older children (aOR, 95%CI: 0.94, 0.93-0.96) and a higher risk with infectious diseases (aOR, 95%CI: 1.49, 1.33-1.66) and admission by resident and attending pediatricians (aOR, 95%CI: 1.41, 1.30-1.54). Discordant cases had a higher rate of antibiotic prescription (OR, 95%CI: 2.09, 1.87-2.33), a longer duration of antibiotic use (P = 0.02), a longer length of hospital stay (P < 0.001), and higher medical expenses (P < 0.001).

CONCLUSIONS

This study denotes a considerably high rate of discordance between admission and discharge diagnoses with an associated higher and longer prescription of antibiotics, a longer length of stay, and higher medical expenses among Chinese pediatric inpatient cases. Infectious diseases were identified as high-risk clinical conditions for discordance. Considering potential diagnostic and coding errors, departmental investigation of preventable diagnostic discordance is suggested for quality health care and preventing potential medicolegal consequences.

Effects of a computerised diagnostic decision support tool on diagnostic quality in emergency departments: study protocol of the DDx-BRO multicentre cluster randomised cross-over trial

INTRODUCTION

Computerised diagnostic decision support systems (CDDS) suggesting differential diagnoses to physicians aim to improve clinical reasoning and diagnostic quality. However, controlled clinical trials investigating their effectiveness and safety are absent and the consequences of its use in clinical practice are unknown. We aim to investigate the effect of CDDS use in the emergency department (ED) on diagnostic quality, workflow, resource consumption and patient outcomes.

METHODS AND ANALYSIS

This is a multicentre, outcome assessor and patient-blinded, cluster-randomised, multiperiod crossover superiority trial. A validated differential diagnosis generator will be implemented in four EDs and randomly allocated to a sequence of six alternating intervention and control periods. During intervention periods, the treating ED physician will be asked to consult the CDDS at least once during diagnostic workup. During control periods, physicians will not have access to the CDDS and diagnostic workup will follow usual clinical care. Key inclusion criteria will be patients' presentation to the ED with either fever, abdominal pain, syncope or a non-specific complaint as chief complaint. The primary outcome is a binary diagnostic quality risk score composed of presence of an unscheduled medical care after discharge, change in diagnosis or death during time of follow-up or an unexpected upscale in care within 24 hours after hospital admission. Time of follow-up is 14 days. At least 1184 patients will be included. Secondary outcomes include length of hospital stay, diagnostics and data regarding CDDS usage, physicians' confidence calibration and diagnostic workflow. Statistical analysis will use general linear mixed modelling methods.

ETHICS AND DISSEMINATION

Approved by the cantonal ethics committee of canton Berne (2022-D0002) and Swissmedic, the Swiss national regulatory authority on medical devices. Study results will be disseminated through peer-reviewed journals, open repositories and the network of investigators and the expert and patients advisory board.

TRIAL REGISTRATION NUMBER

NCT05346523.

Partnering with patients and families living with chronic conditions to coproduce diagnostic safety through OurDX: a previsit online engagement tool

OBJECTIVE

Patients and families are key partners in diagnosis, but methods to routinely engage them in diagnostic safety are lacking. Policy mandating patient access to electronic health information presents new opportunities. We tested a new online tool ("OurDX") that was codesigned with patients and families, to determine the types and frequencies of potential safety issues identified by patients/families with chronic health conditions and whether their contributions were integrated into the visit note.

METHODS

Patients/families at 2 US healthcare sites were invited to contribute, through an online previsit survey: (1) visit priorities, (2) recent medical history/symptoms, and (3) potential diagnostic concerns. Two physicians reviewed patient-reported diagnostic concerns to verify and categorize diagnostic safety opportunities (DSOs). We conducted a chart review to determine whether patient contributions were integrated into the note. We used descriptive statistics to report implementation outcomes, verification of DSOs, and chart review findings.

RESULTS

Participants completed OurDX reports in 7075 of 18 129 (39%) eligible pediatric subspecialty visits (site 1), and 460 of 706 (65%) eligible adult primary care visits (site 2). Among patients reporting diagnostic concerns, 63% were verified as probable DSOs. In total, probable DSOs were identified by 7.5% of pediatric and adult patients/families with underlying health conditions, respectively. The most common types of DSOs were patients/families not feeling heard; problems/delays with tests or referrals; and problems/delays with explanation or next steps. In chart review, most clinician notes included all or some patient/family priorities and patient-reported histories.

CONCLUSIONS

OurDX can help engage patients and families living with chronic health conditions in diagnosis. Participating patients/families identified DSOs and most of their OurDX contributions were included in the visit note.

Types of diagnostic errors reported by paediatric emergency providers in a global paediatric emergency care research network

Background

Diagnostic errors, reframed as missed opportunities for improving diagnosis (MOIDs), are poorly understood in the paediatric emergency department (ED) setting. We investigated the clinical experience, harm and contributing factors related to MOIDs reported by physicians working in paediatric EDs.

Methods

We developed a web-based survey in which physicians participating in the international Paediatric Emergency Research Network representing five out of six WHO regions, described examples of MOIDs involving their own or a colleague’s patients. Respondents provided case summaries and answered questions regarding harm and factors contributing to the event.

Results

Of 1594 physicians surveyed, 412 (25.8%) responded (mean age=43 years (SD=9.2), 42.0% female, mean years in practice=12 (SD=9.0)). Patient presentations involving MOIDs had common undifferentiated symptoms at initial presentation, including abdominal pain (21.1%), fever (17.2%) and vomiting (16.5%). Patients were discharged from the ED with commonly reported diagnoses, including acute gastroenteritis (16.7%), viral syndrome (10.2%) and constipation (7.0%). Most reported MOIDs (65%) were detected on ED return visits (46% within 24 hours and 76% within 72 hours). The most common reported MOID was appendicitis (11.4%), followed by brain tumour (4.4%), meningitis (4.4%) and non-accidental trauma (4.1%). More than half (59.1%) of the reported MOIDs involved the patient/parent–provider encounter (eg, misinterpreted/ignored history or an incomplete/inadequate physical examination). Types of MOIDs and contributing factors did not differ significantly between countries. More than half of patients had either moderate (48.7%) or major (10%) harm due to the MOID.

Conclusions

An international cohort of paediatric ED physicians reported several MOIDs, often in children who presented to the ED with common undifferentiated symptoms. Many of these were related to patient/parent–provider interaction factors such as suboptimal history and physical examination. Physicians’ personal experiences offer an underexplored source for investigating and mitigating diagnostic errors in the paediatric ED.

Clinical-GAN: Trajectory forecasting of clinical events using transformer and Generative Adversarial Networks

Predicting the trajectory of a disease at an early stage can aid physicians in offering effective treatment, prompt care to patients, and also avoid misdiagnosis. However, forecasting patient trajectories is challenging due to long-range dependencies, irregular intervals between consecutive admissions, and non-stationarity data. To address these challenges, we propose a novel method called Clinical-GAN, a Transformer-based Generative Adversarial Networks (GAN) to forecast the patients' medical codes for subsequent visits. First, we represent the patients' medical codes as a time-ordered sequence of tokens akin to language models. Then, a Transformer mechanism is used as a Generator to learn from existing patients' medical history and is trained adversarially against a Transformer-based Discriminator. We address the above mentioned challenges based on our data modeling and Transformer-based GAN architecture. Additionally, we enable the local interpretation of the model's prediction using a multi-head attention mechanism. We evaluated our method using a publicly available dataset, Medical Information Mart for Intensive Care IV v1.0 (MIMIC-IV), with more than 500,000 visits completed by around 196,000 adult patients over an 11-year period from 2008-2019. Clinical-GAN significantly outperforms baseline methods and existing works, as demonstrated through various experiments. Source code is at https://github.com/vigi30/Clinical-GAN.

I had not time to make it shorter: an exploratory analysis of how physicians reduce note length and time in notes

OBJECTIVE

We analyze observed reductions in physician note length and documentation time, 2 contributors to electronic health record (EHR) burden and burnout.

MATERIALS AND METHODS

We used EHR metadata from January to May, 2021 for 130 079 ambulatory physician Epic users. We identified cohorts of physicians who decreased note length and/or documentation time and analyzed changes in their note composition.

RESULTS

37 857 physicians decreased either note length (n = 15 647), time in notes (n = 15 417), or both (n = 6793). Note length decreases were primarily attributable to reductions in copy/paste text (average relative change of -18.9%) and templated text (-17.2%). Note time decreases were primarily attributable to reductions in manual text (-27.3%) and increases in note content from other care team members (+21.1%).

DISCUSSION

Organizations must consider priorities and tradeoffs in the distinct approaches needed to address different contributors to EHR burden.

CONCLUSION

Future research should explore scalable burden-reduction initiatives responsive to both note bloat and documentation time.