Interobserver agreement between radiologists and radiology residents and emergency physicians in the detection of PE using CTPA

Performance and clinical utility of an artificial intelligence-enabled tool for pulmonary embolism detection

PURPOSE

Diagnosing pulmonary embolism (PE) is still challenging due to other conditions that can mimic its appearance, leading to incomplete or delayed management and several inter-observer variabilities. This study evaluated the performance and clinical utility of an artificial intelligence (AI)-based application designed to assist clinicians in the detection of PE on CT pulmonary angiography (CTPA).

PATIENTS AND METHODS

CTPAs from 230 US cities acquired on 57 scanner models from 6 different vendors were retrospectively collected. Three US board certified expert radiologists defined the ground truth by majority agreement. The same cases were analyzed by CINA-PE, an AI-driven algorithm capable of detecting and highlighting suspected PE locations. The algorithm's performance at a per-case and per-finding level was evaluated. Furthermore, cases with PE not mentioned in the clinical report but correctly detected by the algorithm were analyzed.

RESULTS

A total of 1204 CTPAs (mean age 62.1 years ± 16.6[SD], 44.4 % female, 14.9 % positive) were included in the study. Per-case sensitivity and specificity were 93.9 % (95%CI: 89.3 %-96.9 %) and 94.8 % (95%CI: 93.3 %-96.1 %), respectively. Per-finding positive predictive value was 89.5 % (95%CI: 86.7 %-91.9 %). Among the 196 positive cases, 29 (15.6 %) were not mentioned in the clinical report. The algorithm detected 22/29 (76 %) of these cases, leading to a reduction in the miss rate from 15.6 % to 3.8 % (7/186).

CONCLUSIONS

The AI-based application may improve diagnostic accuracy in detecting PE and enhance patient outcomes through timely intervention. Integrating AI tools in clinical workflows can reduce missed or delayed diagnoses, and positively impact healthcare delivery and patient care.

Influence of thoracic radiology training on classification of interstitial lung diseases

INTRODUCTION

Interpretation of high-resolution CT images plays an important role in the diagnosis and management of interstitial lung diseases. However, interreader variation may exist due to varying levels of training and expertise. This study aims to evaluate interreader variation and the role of thoracic radiology training in classifying interstitial lung disease (ILD).

METHODS

This is a retrospective study where seven physicians (radiologists, thoracic radiologists, and a pulmonologist) classified the subtypes of ILD of 128 patients from a tertiary referral center, all selected from the Interstitial Lung Disease Registry which consists of patients from November 2014 to January 2021. Each patient was diagnosed with a subtype of interstitial lung disease by a consensus diagnosis from pathology, radiology, and pulmonology. Each reader was provided with only clinical history, only CT images, or both. Reader sensitivity and specificity and interreader agreements using Cohen's κ were calculated.

RESULTS

Interreader agreement based only on clinical history, only on radiologic information, or combination of both was most consistent amongst readers with thoracic radiology training, ranging from fair (Cohen's κ: 0.2-0.46), moderate to almost perfect (Cohen's κ: 0.55-0.92), and moderate to almost perfect (Cohen's κ: 0.53-0.91) respectively. Radiologists with any thoracic training showed both increased sensitivity and specificity for NSIP as compared to other radiologists and the pulmonologist when using only clinical history, only CT information, or combination of both (p < 0.05).

CONCLUSIONS

Readers with thoracic radiology training showed the least interreader variation and were more sensitive and specific at classifying certain subtypes of ILD.

SUMMARY SENTENCE

Thoracic radiology training may improve sensitivity and specificity in classifying ILD based on HRCT images and clinical history.

Accuracy of emergency physicians' interpretation of computed tomography for urgent-emergent diagnoses in nontraumatic cases

OBJECTIVE

The aim of this study is to evaluate the accuracy levels of the emergency physicians (EPs) managing the patient in the interpretation of the urgent-emergent pathological findings in thoracic and abdominal computed tomography (CT) scans.

METHODS

The EPs interpreted the CT scans of patients who visited the emergency department because of nontraumatic causes. Then, a radiology instructor made final assessments of these CT scans. Based on the interpretation of the radiology instructor, the false-positive rate, false-negative rate, sensitivity, specificity, positive predictive value, negative predictive value, and kappa coefficient (κ) of the EPs' interpretations of the CT scans were calculated.

RESULTS

A total of 268 thoracics and 185 abdominal CT scans were assessed in our study. The overall sensitivity and specificity of the EPs' interpretation of the thoracic CT scans were 90% and 89%, respectively, whereas the abdominal CT interpretation was 88% and 86%, respectively. There was excellent concordance between the EPs and the radiology instructor with regard to the diagnoses of pneumothorax, pulmonary embolism, pleural effusion, parenchymal pathology, and masses (κ: 0.90, κ: 0.87, κ: 0.71, κ: 0.79, and κ: 0.91, respectively) and to the diagnoses of intraabdominal free fluid, intraabdominal free gas, aortic pathology, splenic pathology, gallbladder pathology, mesenteric artery embolism, appendicitis, gynecological pathology, and renal pathology (κ: 1, κ: 0.92, κ: 0.96, κ: 0.88, κ: 0.80, κ: 0.79, κ: 0.89, κ: 0.88, and κ: 0.82, respectively).

CONCLUSION

The EPs are successful in the interpretation of the urgent-emergent pathological findings in thoracic and abdominal CT scans.

Research Priorities in the Utilization and Interpretation of Diagnostic Imaging: Education, Assessment, and Competency

The appropriate selection and accurate interpretation of diagnostic imaging is a crucial skill for emergency practitioners. To date, the majority of the published literature and research on competency assessment comes from the subspecialty of point-of-care ultrasound. A group of radiologists, physicists, and emergency physicians convened at the 2015 Academic Emergency Medicine consensus conference to discuss and prioritize a research agenda related to education, assessment, and competency in ordering and interpreting diagnostic imaging. A set of questions for the continued development of an educational curriculum on diagnostic imaging for trainees and competency assessment using specific assessment methods based on current best practices was delineated. The research priorities were developed through an iterative consensus-driven process using a modified nominal group technique that culminated in an in-person breakout session. The four recommendations are: 1) develop a diagnostic imaging curriculum for emergency medicine (EM) residency training; 2) develop, study, and validate tools to assess competency in diagnostic imaging interpretation; 3) evaluate the role of simulation in education, assessment, and competency measures for diagnostic imaging; 4) study is needed regarding the American College of Radiology Appropriateness Criteria, an evidence-based peer-reviewed resource in determining the use of diagnostic imaging, to maximize its value in EM. In this article, the authors review the supporting reliability and validity evidence and make specific recommendations for future research on the education, competency, and assessment of learning diagnostic imaging.

Cost-effectiveness of quantitative pretest probability intended to reduce unnecessary medical radiation exposure in emergency department patients with chest pain and dyspnea

OBJECTIVES

Quantitative pretest probability (qPTP) incorporated into a decision support tool with advice can reduce unnecessary diagnostic testing among patients with symptoms suggestive of acute coronary syndrome (ACS) and pulmonary embolism (PE), reducing 30-day costs without an increase in 90-day adverse outcomes. This study estimates long-term (beyond 90-day) costs and outcomes associated with qPTP. The authors hypothesized that qPTP reduces lifetime costs and improves outcomes in low-risk patients with symptoms suggestive of ACS and PE.

METHODS

This was a cost-effectiveness analysis of a multicenter, randomized controlled trial of adult emergency patients with dyspnea and chest pain, in which a clinician encountering a low-risk patient with symptoms suggestive of ACS or PE conducted either the intervention (qPTP for ACS and PE with advice) or the sham (no qPTP and no advice). Effect of the intervention over a patient's lifetime was assessed using a Markov microsimulation model. Short-term costs and outcomes were from the trial; long-term outcomes and costs were from the literature. Outcomes included lifetime transition to PE, ACS, and intracranial hemorrhage (ICH); mortality from cancer, ICH, PE, ACS, renal failure, and ischemic stroke; quality-adjusted life-years (QALYs); and total medical costs compared between simulated intervention and sham groups.

RESULTS

Markov microsimulation for a 40-year-old patient receiving qPTP found lifetime cost savings of $497 for women and $528 for men, associated with small gains in QALYs (2 and 6 days, respectively) and lower rates of cancer mortality in both sexes, but a reduction in ICH only in males. Sensitivity analysis for patients aged 60 years predicted that qPTP would continue to save costs and also reduce mortality from both ICH and cancer. Use of qPTP significantly reduced the lifetime probability of PE diagnosis, with lower probability of death from PE in both sexes aged 40 to 60 years. However, use of qPTP reduced the rate of ACS diagnosis and death from ACS at age 40, but increased the death rate from ACS at age 60 for both sexes.

CONCLUSIONS

Widespread use of a combined qPTP for both ACS and PE has the potential to decrease costs by reducing diagnostic testing, while improving most long-term outcomes in emergency patients with chest pain and dyspnea.