Chest X-ray interpretation in UK intensive care units: A survey 2014

Development of image-based decision support systems utilizing information extracted from radiological free-text report databases with text-based transformers

OBJECTIVES

To investigate the potential and limitations of utilizing transformer-based report annotation for on-site development of image-based diagnostic decision support systems (DDSS).

METHODS

The study included 88,353 chest X-rays from 19,581 intensive care unit (ICU) patients. To label the presence of six typical findings in 17,041 images, the corresponding free-text reports of the attending radiologists were assessed by medical research assistants ("gold labels"). Automatically generated "silver" labels were extracted for all reports by transformer models trained on gold labels. To investigate the benefit of such silver labels, the image-based models were trained using three approaches: with gold labels only (MG), with silver labels first, then with gold labels (MS/G), and with silver and gold labels together (MS+G). To investigate the influence of invested annotation effort, the experiments were repeated with different numbers (N) of gold-annotated reports for training the transformer and image-based models and tested on 2099 gold-annotated images. Significant differences in macro-averaged area under the receiver operating characteristic curve (AUC) were assessed by non-overlapping 95% confidence intervals.

RESULTS

Utilizing transformer-based silver labels showed significantly higher macro-averaged AUC than training solely with gold labels (N = 1000: MG 67.8 [66.0-69.6], MS/G 77.9 [76.2-79.6]; N = 14,580: MG 74.5 [72.8-76.2], MS/G 80.9 [79.4-82.4]). Training with silver and gold labels together was beneficial using only 500 gold labels (MS+G 76.4 [74.7-78.0], MS/G 75.3 [73.5-77.0]).

CONCLUSIONS

Transformer-based annotation has potential for unlocking free-text report databases for the development of image-based DDSS. However, on-site development of image-based DDSS could benefit from more sophisticated annotation pipelines including further information than a single radiological report.

CLINICAL RELEVANCE STATEMENT

Leveraging clinical databases for on-site development of artificial intelligence (AI)-based diagnostic decision support systems by text-based transformers could promote the application of AI in clinical practice by circumventing highly regulated data exchanges with third parties.

KEY POINTS

• The amount of data from a database that can be used to develop AI-assisted diagnostic decision systems is often limited by the need for time-consuming identification of pathologies by radiologists. • The transformer-based structuring of free-text radiological reports shows potential to unlock corresponding image databases for on-site development of image-based diagnostic decision support systems. • However, the quality of image annotations generated solely on the content of a single radiology report may be limited by potential inaccuracies and incompleteness of this report.

Medical transformer for multimodal survival prediction in intensive care: integration of imaging and non-imaging data

When clinicians assess the prognosis of patients in intensive care, they take imaging and non-imaging data into account. In contrast, many traditional machine learning models rely on only one of these modalities, limiting their potential in medical applications. This work proposes and evaluates a transformer-based neural network as a novel AI architecture that integrates multimodal patient data, i.e., imaging data (chest radiographs) and non-imaging data (clinical data). We evaluate the performance of our model in a retrospective study with 6,125 patients in intensive care. We show that the combined model (area under the receiver operating characteristic curve [AUROC] of 0.863) is superior to the radiographs-only model (AUROC = 0.811, p < 0.001) and the clinical data-only model (AUROC = 0.785, p < 0.001) when tasked with predicting in-hospital survival per patient. Furthermore, we demonstrate that our proposed model is robust in cases where not all (clinical) data points are available.

Artificial Intelligence for Clinical Interpretation of Bedside Chest Radiographs

Background Supine chest radiography for bedridden patients in intensive care units (ICUs) is one of the most frequently ordered imaging studies worldwide. Purpose To evaluate the diagnostic performance of a neural network-based model that is trained on structured semiquantitative radiologic reports of bedside chest radiographs. Materials and Methods For this retrospective single-center study, children and adults in the ICU of a university hospital who had been imaged using bedside chest radiography from January 2009 to December 2020 were reported by using a structured and itemized template. Ninety-eight radiologists rated the radiographs semiquantitatively for the severity of disease patterns. These data were used to train a neural network to identify cardiomegaly, pulmonary congestion, pleural effusion, pulmonary opacities, and atelectasis. A held-out internal test set (100 radiographs from 100 patients) that was assessed independently by an expert panel of six radiologists provided the ground truth. Individual assessments by each of these six radiologists, by two nonradiologist physicians in the ICU, and by the neural network were compared with the ground truth. Separately, the nonradiologist physicians assessed the images without and with preliminary readings provided by the neural network. The weighted Cohen κ coefficient was used to measure agreement between the readers and the ground truth. Results A total of 193 566 radiographs in 45 016 patients (mean age, 66 years ± 16 [SD]; 61% men) were included and divided into training (n = 122 294; 64%), validation (n = 31 243; 16%), and test (n = 40 029; 20%) sets. The neural network exhibited higher agreement with a majority vote of the expert panel (κ = 0.86) than each individual radiologist compared with the majority vote of the expert panel (κ = 0.81 to ≤0.84). When the neural network provided preliminary readings, the reports of the nonradiologist physicians improved considerably (aided vs unaided, κ = 0.87 vs 0.79, respectively; P < .001). Conclusion A neural network trained with structured semiquantitative bedside chest radiography reports allowed nonradiologist physicians improved interpretations compared with the consensus reading of expert radiologists. © RSNA, 2022 Supplemental material is available for this article. See also the editorial by Wielpütz in this issue.

Lost Venous Access Guidewire Identified by Its Late Cervical Protrusion: Case Report

We report a pediatric patient who underwent a central venous catheter (CVC) insertion and presented with a sudden protrusion of a guidewire from the neck 26 months later. The guidewire was extracted via femoral venotomy. A 5-cm portion of the guidewire adhering to the superior vena cava wall was left in place. We recommend always using a CVC checklist, inspecting the guidewire before and after insertion, and carefully examining the postinsertion radiographs. This checklist should be mandatory with every CVC insertion, including the perioperative period.

Deep-Learning-Based Diagnosis of Bedside Chest X-ray in Intensive Care and Emergency Medicine

OBJECTIVES

Validation of deep learning models should separately consider bedside chest radiographs (CXRs) as they are the most challenging to interpret, while at the same time the resulting diagnoses are important for managing critically ill patients. Therefore, we aimed to develop and evaluate deep learning models for the identification of clinically relevant abnormalities in bedside CXRs, using reference standards established by computed tomography (CT) and multiple radiologists.

MATERIALS AND METHODS

In this retrospective study, a dataset consisting of 18,361 bedside CXRs of patients treated at a level 1 medical center between January 2009 and March 2019 was used. All included CXRs occurred within 24 hours before or after a chest CT. A deep learning algorithm was developed to identify 8 findings on bedside CXRs (cardiac congestion, pleural effusion, air-space opacification, pneumothorax, central venous catheter, thoracic drain, gastric tube, and tracheal tube/cannula). For the training dataset, 17,275 combined labels were extracted from the CXR and CT reports by a deep learning natural language processing (NLP) tool. In case of a disagreement between CXR and CT, human-in-the-loop annotations were used. The test dataset consisted of 583 images, evaluated by 4 radiologists. Performance was assessed by area under the receiver operating characteristic curve analysis, sensitivity, specificity, and positive predictive value.

RESULTS

Areas under the receiver operating characteristic curve for cardiac congestion, pleural effusion, air-space opacification, pneumothorax, central venous catheter, thoracic drain, gastric tube, and tracheal tube/cannula were 0.90 (95% confidence interval [CI], 0.87-0.93; 3 radiologists on the receiver operating characteristic [ROC] curve), 0.95 (95% CI, 0.93-0.96; 3 radiologists on the ROC curve), 0.85 (95% CI, 0.82-0.89; 1 radiologist on the ROC curve), 0.92 (95% CI, 0.89-0.95; 1 radiologist on the ROC curve), 0.99 (95% CI, 0.98-0.99), 0.99 (95% CI, 0.98-0.99), 0.98 (95% CI, 0.97-0.99), and 0.99 (95% CI, 0.98-1.00), respectively.

CONCLUSIONS

A deep learning model used specifically for bedside CXRs showed similar performance to expert radiologists. It could therefore be used to detect clinically relevant findings during after-hours and help emergency and intensive care physicians to focus on patient care.