Now you see me: a pragmatic cohort study comparing first and final radiological diagnoses in the emergency department

Artificial intelligence in commercial fracture detection products: a systematic review and meta-analysis of diagnostic test accuracy

Conventional radiography (CR) is primarily utilized for fracture diagnosis. Artificial intelligence (AI) for CR is a rapidly growing field aimed at enhancing efficiency and increasing diagnostic accuracy. However, the diagnostic performance of commercially available AI fracture detection solutions (CAAI-FDS) for CR in various anatomical regions, their synergy with human assessment, as well as the influence of industry funding on reported accuracy are unknown. Peer-reviewed diagnostic test accuracy (DTA) studies were identified through a systematic review on Pubmed and Embase. Diagnostic performance measures were extracted especially for different subgroups such as product, type of rater (stand-alone AI, human unaided, human aided), funding, and anatomical region. Pooled measures were obtained with a bivariate random effects model. The impact of rater was evaluated with comparative meta-analysis. Seventeen DTA studies of seven CAAI-FDS analyzing 38,978 x-rays with 8,150 fractures were included. Stand-alone AI studies (n = 15) evaluated five CAAI-FDS; four with good sensitivities (> 90%) and moderate specificities (80-90%) and one with very poor sensitivity (< 60%) and excellent specificity (> 95%). Pooled sensitivities were good to excellent, and specificities were moderate to good in all anatomical regions (n = 7) apart from ribs (n = 4; poor sensitivity / moderate specificity) and spine (n = 4; excellent sensitivity / poor specificity). Funded studies (n = 4) had higher sensitivity (+ 5%) and lower specificity (-4%) than non-funded studies (n = 11). Sensitivity did not differ significantly between stand-alone AI and human AI aided ratings (p = 0.316) but specificity was significantly higher the latter group (p < 0.001). Sensitivity was significant lower in human unaided compared to human AI aided respectively stand-alone AI ratings (both p ≤ 0.001); specificity was higher in human unaided ratings compared to stand-alone AI (p < 0.001) and showed no significant differences AI aided ratings (p = 0.316). The study demonstrates good diagnostic accuracy across most CAAI-FDS and anatomical regions, with the highest performance achieved when used in conjunction with human assessment. Diagnostic accuracy appears lower for spine and rib fractures. The impact of industry funding on reported performance is small.

Comparison of tomographic reports by radiologists and non-radiologists in trauma and interferences in management in a trauma reference center

OBJECTIVE

diagnostic errors during the interpretation of an imaging test by the physician can lead to increased mortality and length of hospital stay for patients. The rate of divergence in the report given by a radiologist and an Emergency Physicians (EP) can reach over 20%. The objective of this study was to compare the unofficial tomographic reports issued by EP with the official reports issued by radiologists.

METHODS

a cross-sectional study, in which interpretations of the exams (documented in the medical records by the EP) of all patients undergoing computed tomography (CT) of the chest, abdomen or pelvis performed in the emergency room, at an interval of 8 months, were evaluated. These data were compared with the official reports of the radiologist (gold standard).

RESULTS

508 patients were included. The divergence between EP and the radiologist occurred in 27% of the cases. The most common type of divergence was the one not described by the EP, but described by the radiologist. The chance of having divergence in a case of multiple trauma is 4.93 times greater in relation to the case of only blunt trauma in one segment. A statistically relevant difference was also found in the length of stay of patients who had different interpretations of the CT scans.

CONCLUSION

the study found a relatively high divergence rate between the EP report and the official radiologist report. However, less than 4% of these were considered to be clinically relevant, indicating the ability of the EP to interpret it satisfactorily.

LOw-dose CT Or Lung UltraSonography versus standard of care based-strategies for the diagnosis of pneumonia in the elderly: protocol for a multicentre randomised controlled trial (OCTOPLUS)

INTRODUCTION

Pneumonia is a leading cause of mortality and a common indication for antibiotic in elderly patients. However, its diagnosis is often inaccurate. We aim to compare the diagnostic accuracy, the clinical and cost outcomes and the use of antibiotics associated with three imaging strategies in patients >65 years old with suspected pneumonia in the emergency room (ER): chest X-ray (CXR, standard of care), low-dose CT scan (LDCT) or lung ultrasonography (LUS).

METHODS AND ANALYSIS

This is a multicentre randomised superiority clinical trial with three parallel arms. Patients will be allocated in the ER to a diagnostic strategy based on either CXR, LDCT or LUS. All three imaging modalities will be performed but the results of two of them will be masked during 5 days to the patients, the physicians in charge of the patients and the investigators according to random allocation. The primary objective is to compare the accuracy of LDCT versus CXR-based strategies. As secondary objectives, antibiotics prescription, clinical and cost outcomes will be compared, and the same analyses repeated to compare the LUS and CXR strategies. The reference diagnosis will be established a posteriori by a panel of experts. Based on a previous study, we expect an improvement of 16% of the accuracy of pneumonia diagnosis using LDCT instead of CXR. Under this assumption, and accounting for 10% of drop-out, the enrolment of 495 patients is needed to prove the superiority of LDCT over CRX (alpha error=0.05, beta error=0.10).

ETHICS AND DISSEMINATION

Ethical approval: CER Geneva 2019-01288.

TRIAL REGISTRATION NUMBER

NCT04978116.

Identifying trigger concepts to screen emergency department visits for diagnostic errors

OBJECTIVES

The diagnostic process is a vital component of safe and effective emergency department (ED) care. There are no standardized methods for identifying or reliably monitoring diagnostic errors in the ED, impeding efforts to enhance diagnostic safety. We sought to identify trigger concepts to screen ED records for diagnostic errors and describe how they can be used as a measurement strategy to identify and reduce preventable diagnostic harm.

METHODS

We conducted a literature review and surveyed ED directors to compile a list of potential electronic health record (EHR) trigger (e-triggers) and non-EHR based concepts. We convened a multidisciplinary expert panel to build consensus on trigger concepts to identify and reduce preventable diagnostic harm in the ED.

RESULTS

Six e-trigger and five non-EHR based concepts were selected by the expert panel. E-trigger concepts included: unscheduled ED return to ED resulting in hospital admission, death following ED visit, care escalation, high-risk conditions based on symptom-disease dyads, return visits with new diagnostic/therapeutic interventions, and change of treating service after admission. Non-EHR based signals included: cases from mortality/morbidity conferences, risk management/safety office referrals, ED medical director case referrals, patient complaints, and radiology/laboratory misreads and callbacks. The panel suggested further refinements to aid future research in defining diagnostic error epidemiology in ED settings.

CONCLUSIONS

We identified a set of e-trigger concepts and non-EHR based signals that could be developed further to screen ED visits for diagnostic safety events. With additional evaluation, trigger-based methods can be used as tools to monitor and improve ED diagnostic performance.

Factors Associated With Diagnostic Error on Admission to a PICU: A Pilot Study

OBJECTIVES

Diagnostic errors can harm critically ill children. However, we know little about their prevalence in PICUs and factors associated with error. The objective of this pilot study was to determine feasibility of record review to identify patient, provider, and work system factors associated with diagnostic errors during the first 12 hours after PICU admission.

DESIGN

Pilot retrospective cohort study with structured record review using a structured tool (Safer Dx instrument) to identify diagnostic error.

SETTING

Academic tertiary referral PICU.

PATIENTS

Patients 0-17 years old admitted nonelectively to the PICU.

INTERVENTIONS

None.

MEASUREMENTS AND MAIN RESULTS

Four of 50 patients (8%) had diagnostic errors in the first 12 hours after admission. The Safer Dx instrument helped identify delayed diagnoses of chronic ear infection, increased intracranial pressure (two cases), and Bartonella encephalitis. We calculated that 610 PICU admissions are needed to achieve 80% power (α = 0.05) to detect significant associations with error.

CONCLUSIONS

Our pilot study found four patients with diagnostic error out of 50 children admitted nonelectively to a PICU. Retrospective record review using a structured tool to identify diagnostic errors is feasible in this population. Pilot data are being used to inform a larger and more definitive multicenter study.

Emergent Comprehensive Imaging of the Major Trauma Patient: A New Paradigm for Improved Clinical Decision-Making

Modern advances in the medical imaging layered onto sophisticated trauma resuscitation strategies in highly organized regionalized trauma systems have created a paradigm shift in the management of severely injured patients. Although immediate exploratory surgery to identify and control life-threatening injuries still has its place, accelerated image acquisition and interpretation procedures now make it rare for trauma surgeons in major centers to venture into damage control surgery unaided by computed tomography (CT) or other imaging, particularly in cases of blunt trauma. Indeed, because of the high incidence of clinically occult injuries associated with major mechanism trauma, and even lower energy trauma in frail or elderly patients, CT imaging has become as invaluable as physical examination, if not more so, in critical decision-making in support of optimal outcomes. In particular, whole-body computed tomography (WBCT) completed promptly after initial assessment of a major trauma provides a quick, comprehensive survey of injuries that enables better surgical planning, obviates the need for multiple subsequent studies, and permits specialized reconstructions when needed. For those at risk for problematic occult injury after modest trauma, WBCT facilitates safer discharge planning and simplified follow-up. Through standardized guidelines, streamlined protocols, synoptic reporting, accessible web-based platforms, and active collaboration with clinicians, radiologists dedicated to trauma and emergency imaging enable clearer understanding of complex injuries in high-risk patients which leads to superior clinical decision-making. Whereas dated dogma has long warned that the CT scanner is the last place to take a challenging trauma patient, modern practice suggests that, more often than not, early comprehensive imaging can be done safely and efficiently and is in the patient's best interest. This article outlines how the role of diagnostic imaging for major trauma has evolved considerably in recent years.

Radiograph Interpretation Discrepancies in a Community Hospital Emergency Department

Introduction

In many hospitals, off-hours emergency department (ED) radiographs are not read by a radiologist until the following morning and are instead interpreted by the emergency physician (EP) at the time of service. Studies have found conflicting results regarding the radiographic interpretation discrepancies between EPs and trained radiologists. The aim of this study was to identify the number of radiologic interpretation discrepancies between EPs and radiologists in a community ED setting.

Methods

Using a pre-existing logbook of radiologic discrepancies as well as our institution’s picture archiving and communication system, all off-hours interpretation discrepancies between January 2012 and January 2015 were reviewed and recorded in a de-identified fashion. We recorded the type of radiograph obtained for each patient. Discrepancy grades were recorded based on a pre-existing 1–4 scale defined in the institution’s protocol logbook as Grade 1 (no further action needed); Grade 2 (call to the patient or pharmacy); Grade 3 (return to ED for further treatment, e.g., fracture not splinted); Grade 4 (return to ED for serious risk, e.g., pneumothorax, bowel obstruction). We also recorded the total number of radiographs formally interpreted by EPs during the prescribed time-frame to determine overall agreement between EPs and radiologists.

Results

There were 1044 discrepancies out of 16,111 EP reads, indicating 93.5% agreement. Patients averaged 48.4 ± 25.0 years of age and 53.3% were female; 25.1% were over-calls by EPs. The majority of discrepancies were minor with 75.8% Grade 1 and 22.3% Grade 2. Only 1.7% were Grade 3, which required return to the ED for further treatment. A small number of discrepancies, 0.2%, were Grade 4. Grade 4 discrepancies accounted for two of the 16,111 total reads, equivalent to 0.01%. A slight disagreement in finding between EP and radiologist accounted for 8.3% of discrepancies.

Conclusion

Results suggest that plain radiographic studies can be interpreted by EPs with a very low incidence of clinically significant discrepancies when compared to the radiologist interpretation. Due to rare though significant discrepancies, radiologist interpretation should be performed when available. Further studies are needed to determine the generalizability of this study to EDs with differing volume, patient population, acuity, and physician training.

Diagnostic error increases mortality and length of hospital stay in patients presenting through the emergency room

BACKGROUND

Diagnostic errors occur frequently, especially in the emergency room. Estimates about the consequences of diagnostic error vary widely and little is known about the factors predicting error. Our objectives thus was to determine the rate of discrepancy between diagnoses at hospital admission and discharge in patients presenting through the emergency room, the discrepancies' consequences, and factors predicting them.

METHODS

Prospective observational clinical study combined with a survey in a University-affiliated tertiary care hospital. Patients' hospital discharge diagnosis was compared with the diagnosis at hospital admittance through the emergency room and classified as similar or discrepant according to a predefined scheme by two independent expert raters. Generalized linear mixed-effects models were used to estimate the effect of diagnostic discrepancy on mortality and length of hospital stay and to determine whether characteristics of patients, diagnosing physicians, and context predicted diagnostic discrepancy.

RESULTS

755 consecutive patients (322 [42.7%] female; mean age 65.14 years) were included. The discharge diagnosis differed substantially from the admittance diagnosis in 12.3% of cases. Diagnostic discrepancy was associated with a longer hospital stay (mean 10.29 vs. 6.90 days; Cohen's d 0.47; 95% confidence interval 0.26 to 0.70; P = 0.002) and increased patient mortality (8 (8.60%) vs. 25(3.78%); OR 2.40; 95% CI 1.05 to 5.5 P = 0.038). A factor available at admittance that predicted diagnostic discrepancy was the diagnosing physician's assessment that the patient presented atypically for the diagnosis assigned (OR 3.04; 95% CI 1.33-6.96; P = 0.009).

CONCLUSIONS

Diagnostic discrepancies are a relevant healthcare problem in patients admitted through the emergency room because they occur in every ninth patient and are associated with increased in-hospital mortality. Discrepancies are not readily predictable by fixed patient or physician characteristics; attention should focus on context.

TRIAL REGISTRATION

https://bmjopen.bmj.com/content/6/5/e011585.

Patient safety culture in a university hospital emergency department in Switzerland - a survey study

Aim of the study: Poor safety culture, bad teamwork, non-functional inter-departmental working relationships and increased cognitive demands are associated with higher amounts of adverse events in hospitals. To improve patient safety, one of the first steps is to assess safety culture among health care providers in an institution. Considering the sparsity of studies addressing patient safety culture in Europe and Switzerland, the aim of the present study was to assess patient safety culture in the emergency department of a University Hospital in Switzerland. Methods: We employed the Hospital Survey On Patient Safety Culture, developed by the U.S. Agency for Healthcare Research and Quality. 140 questionnaires were distributed to nurses and physicians. Two weeks after the first questionnaire, we performed a sensitization campaign addressed to health care providers, and then repeated the survey. We calculated composite scores for each question category and percentages of positive responses for each dimension. For group comparisons such as possible differences relating to education and duration of employment and to compare results of the first and second survey we used T-tests. The results were compared to other published surveys outside of Switzerland. Results: Particularly positive assessments were found for the categories "nonpunitive response to errors", "teamwork within units", "supervisor/manager expectations and actions promoting patient safety" and, compared to other hospitals, also "staffing". The lowest average percent positive responses were found in the categories "frequencies of reported event", "teamwork across units" and "handoffs and transitions". Nurses and health care personnel with a longer employment history had an overall more negative assessment of patient safety culture, when compared to physicians and personnel with a shorter duration of employment, respectively. Conclusions: The present study has identified strengths and potential weaknesses in the safety culture of a large university hospital emergency department in Switzerland. The results provide opportunities for improvement of patient safety in particular in the reporting of adverse events, in interaction across units and patient transitions. Furthermore, as we employed a standardized self-assessment tool similar to previously published studies, the work contributes to the establishment of a benchmark for hospital safety culture at the national, European and international level.