Comparison of on-call radiology resident and faculty interpretation of 4- and 16-row multidetector CT pulmonary angiography with indirect CT venography

An Enhanced Mask R-CNN Approach for Pulmonary Embolism Detection and Segmentation

Pulmonary embolism (PE) refers to the occlusion of pulmonary arteries by blood clots, posing a mortality risk of approximately 30%. The detection of pulmonary embolism within segmental arteries presents greater challenges compared with larger arteries and is frequently overlooked. In this study, we developed a computational method to automatically identify pulmonary embolism within segmental arteries using computed tomography (CT) images. The system architecture incorporates an enhanced Mask R-CNN deep neural network trained on PE-containing images. This network accurately localizes pulmonary embolisms in CT images and effectively delineates their boundaries. This study involved creating a local data set and evaluating the model predictions against pulmonary embolisms manually identified by expert radiologists. The sensitivity, specificity, accuracy, Dice coefficient, and Jaccard index values were obtained as 96.2%, 93.4%, 96.%, 0.95, and 0.89, respectively. The enhanced Mask R-CNN model outperformed the traditional Mask R-CNN and U-Net models. This study underscores the influence of Mask R-CNN's loss function on model performance, providing a basis for the potential improvement of Mask R-CNN models for object detection and segmentation tasks in CT images.

Deep Learning-Based Algorithm for Automatic Detection of Pulmonary Embolism in Chest CT Angiograms

Purpose: Since the prompt recognition of acute pulmonary embolism (PE) and the immediate initiation of treatment can significantly reduce the risk of death, we developed a deep learning (DL)-based application aimed to automatically detect PEs on chest computed tomography angiograms (CTAs) and alert radiologists for an urgent interpretation. Convolutional neural networks (CNNs) were used to design the application. The associated algorithm used a hybrid 3D/2D UNet topology. The training phase was performed on datasets adequately distributed in terms of vendors, patient age, slice thickness, and kVp. The objective of this study was to validate the performance of the algorithm in detecting suspected PEs on CTAs. Methods: The validation dataset included 387 anonymized real-world chest CTAs from multiple clinical sites (228 U.S. cities). The data were acquired on 41 different scanner models from five different scanner makers. The ground truth (presence or absence of PE on CTA images) was established by three independent U.S. board-certified radiologists. Results: The algorithm correctly identified 170 of 186 exams positive for PE (sensitivity 91.4% [95% CI: 86.4–95.0%]) and 184 of 201 exams negative for PE (specificity 91.5% [95% CI: 86.8–95.0%]), leading to an accuracy of 91.5%. False negative cases were either chronic PEs or PEs at the limit of subsegmental arteries and close to partial volume effect artifacts. Most of the false positive findings were due to contrast agent-related fluid artifacts, pulmonary veins, and lymph nodes. Conclusions: The DL-based algorithm has a high degree of diagnostic accuracy with balanced sensitivity and specificity for the detection of PE on CTAs.

A deep learning approach for automated diagnosis of pulmonary embolism on computed tomographic pulmonary angiography

Background

Computed tomographic pulmonary angiography (CTPA) is the diagnostic standard for confirming pulmonary embolism (PE). Since PE is a life-threatening condition, early diagnosis and treatment are critical to avoid PE-associated morbidity and mortality. However, PE remains subject to misdiagnosis.

Methods

We retrospectively identified 251 CTPAs performed at a tertiary care hospital between January 2018 to January 2021. The scans were classified as positive (n = 55) and negative (n = 196) for PE based on the annotations made by board-certified radiologists. A fully anonymized CT slice served as input for the detection of PE by the 2D segmentation model comprising U-Net architecture with Xception encoder. The diagnostic performance of the model was calculated at both the scan and the slice levels.

Results

The model correctly identified 44 out of 55 scans as positive for PE and 146 out of 196 scans as negative for PE with a sensitivity of 0.80 [95% CI 0.68, 0.89], a specificity of 0.74 [95% CI 0.68, 0.80], and an accuracy of 0.76 [95% CI 0.70, 0.81]. On slice level, 4817 out of 5183 slices were marked as positive for the presence of emboli with a specificity of 0.89 [95% CI 0.88, 0.89], a sensitivity of 0.93 [95% CI 0.92, 0.94], and an accuracy of 0.89 [95% CI 0.887, 0.890]. The model also achieved an AUROC of 0.85 [0.78, 0.90] and 0.94 [0.936, 0.941] at scan level and slice level, respectively for the detection of PE.

Conclusion

The development of an AI model and its use for the identification of pulmonary embolism will support healthcare workers by reducing the rate of missed findings and minimizing the time required to screen the scans.

Deep learning for pulmonary embolism detection on computed tomography pulmonary angiogram: a systematic review and meta-analysis

Computed tomographic pulmonary angiography (CTPA) is the gold standard for pulmonary embolism (PE) diagnosis. However, this diagnosis is susceptible to misdiagnosis. In this study, we aimed to perform a systematic review of current literature applying deep learning for the diagnosis of PE on CTPA. MEDLINE/PUBMED were searched for studies that reported on the accuracy of deep learning algorithms for PE on CTPA. The risk of bias was evaluated using the QUADAS-2 tool. Pooled sensitivity and specificity were calculated. Summary receiver operating characteristic curves were plotted. Seven studies met our inclusion criteria. A total of 36,847 CTPA studies were analyzed. All studies were retrospective. Five studies provided enough data to calculate summary estimates. The pooled sensitivity and specificity for PE detection were 0.88 (95% CI 0.803-0.927) and 0.86 (95% CI 0.756-0.924), respectively. Most studies had a high risk of bias. Our study suggests that deep learning models can detect PE on CTPA with satisfactory sensitivity and an acceptable number of false positive cases. Yet, these are only preliminary retrospective works, indicating the need for future research to determine the clinical impact of automated PE detection on patient care. Deep learning models are gradually being implemented in hospital systems, and it is important to understand the strengths and limitations of these algorithms.

PENet-a scalable deep-learning model for automated diagnosis of pulmonary embolism using volumetric CT imaging

Pulmonary embolism (PE) is a life-threatening clinical problem and computed tomography pulmonary angiography (CTPA) is the gold standard for diagnosis. Prompt diagnosis and immediate treatment are critical to avoid high morbidity and mortality rates, yet PE remains among the diagnoses most frequently missed or delayed. In this study, we developed a deep learning model-PENet, to automatically detect PE on volumetric CTPA scans as an end-to-end solution for this purpose. The PENet is a 77-layer 3D convolutional neural network (CNN) pretrained on the Kinetics-600 dataset and fine-tuned on a retrospective CTPA dataset collected from a single academic institution. The PENet model performance was evaluated in detecting PE on data from two different institutions: one as a hold-out dataset from the same institution as the training data and a second collected from an external institution to evaluate model generalizability to an unrelated population dataset. PENet achieved an AUROC of 0.84 [0.82-0.87] on detecting PE on the hold out internal test set and 0.85 [0.81-0.88] on external dataset. PENet also outperformed current state-of-the-art 3D CNN models. The results represent successful application of an end-to-end 3D CNN model for the complex task of PE diagnosis without requiring computationally intensive and time consuming preprocessing and demonstrates sustained performance on data from an external institution. Our model could be applied as a triage tool to automatically identify clinically important PEs allowing for prioritization for diagnostic radiology interpretation and improved care pathways via more efficient diagnosis.

Radiologist-initiated double reading of abdominal CT: retrospective analysis of the clinical importance of changes to radiology reports

Background

Misinterpretation of radiological examinations is an important contributing factor to diagnostic errors. Consultant radiologists in Norwegian hospitals frequently request second reads by colleagues in real time. Our objective was to estimate the frequency of clinically important changes to radiology reports produced by these prospectively obtained double readings.

Methods

We retrospectively compared the preliminary and final reports from 1071 consecutive double-read abdominal CT examinations of surgical patients at five public hospitals in Norway. Experienced gastrointestinal surgeons rated the clinical importance of changes from the preliminary to final report. The severity of the radiological findings in clinically important changes was classified as increased, unchanged or decreased.

Results

Changes were classified as clinically important in 146 of 1071 reports (14%). Changes to 3 reports (0.3%) were critical (demanding immediate action), 35 (3%) were major (implying a change in treatment) and 108 (10%) were intermediate (requiring further investigations). The severity of the radiological findings was increased in 118 (81%) of the clinically important changes. Important changes were made less frequently when abdominal radiologists were first readers, more frequently when they were second readers, and more frequently to urgent examinations.

Conclusion

A 14% rate of clinically important changes made during double reading may justify quality assurance of radiological interpretation. Using expert second readers and a targeted selection of urgent cases and radiologists reading outside their specialty may increase the yield of discrepant cases.

The clinical impact of subspecialized radiologist reinterpretation of abdominal imaging studies, with analysis of the types and relative frequency of interpretation discrepancies

PURPOSE

The primary objective of this study was to determine the clinical impact and value of abdominal imaging reinterpretations by subspecialized abdominal imagers.

METHODS

Secondary interpretations for computed tomography (CT), magnetic resonance (MR), and ultrasound (US) abdominal studies performed outside our institution over a 7-month period were retrospectively compared to the primary (outside) interpretation, with interpretive differences recorded. Clinical notes, pathology and subsequent imaging determined ground truth diagnosis and the clinical impact of any interpretive discrepancies were graded as having high, medium, or little/no clinical impact. Interpretive comparisons were scored into categories: (1) no difference; (2) incidental findings of no clinical impact; (3) finding not reported; (4) significance of finding undercalled; (5) significance of finding overcalled; (6) finding misinterpreted; and (7) multiple discrepancy types in one report.

RESULTS

398 report comparisons were reviewed on 380 patients. There were 300 CT, 60 MR, and 38 US examinations. The primary report had 5.0% (20/398) high clinical impact interpretive discrepancies and 7.5% (30/398) medium clinical impact discrepancies. The subspecialized secondary report had no high clinical impact discrepancies and 8/398 (2.0%) medium clinical impact discrepancies. In order of frequency, high and medium impact discrepancies in the primary report consisted of 50% overcalls, 26% unreported findings, 18% undercalls, 4% misinterpretations, and 2% multiple discrepancies.

CONCLUSIONS

Subspecialty review of abdominal imaging exams can provide clinical benefit. Half of the discrepancies in this series of abdominal reinterpretations were due to overcalls.

Interobserver Agreement between On-Call Radiology Resident and General Radiologist Interpretations of CT Pulmonary Angiograms and CT Venograms

Objectives

To evaluate the interobserver agreement (IOA) between the initial radiology resident and the final staff radiologist reports of combined computed tomographic pulmonary angiograms (CTPA) and computed tomographic venograms (CTV) performed during on-call hours.

Materials and Methods

Approval by the institutional review board was obtained. Six-hundred and ninety-six consecutive studies (CTPA or CTPA with CTV) performed during on-call hours and interpreted by 30 residents were identified. Radiology residents’ reports were compared to the final staff reports. Three tests outcomes were considered (positive, P; negative, N; indeterminate, I). Discordant cases were reviews by a chest radiologist.

Results

CTPAs were reported by staff radiologists as positive for pulmonary embolism (PE) in 18% (126/694), with a kappa of 0.81 (95% CI 0.77-0.86) with 3 outcomes (P, N, I), and a kappa of 0.89 (95% CI 0.85-0.94) with 2 outcomes (P, N). Regarding PE location, good concordance was observed for positive studies, with a kappa of 0.86 (95% CI 0.78 – 0.95). CTVs were reported as positive by staff radiologists in 8.5% (33/388), with a kappa of 0.66 (95% CI 0.55-0.77) with 3 outcomes (P, N, I), and a kappa of 0.89 (95% CI 0.8-1.0) with 2 outcomes (P, N). The IOA between residents and staff radiologists increased with increasing residency year level for CTPAs, but did not for CTVs.

Conclusions

Very good and good IOA were observed between resident and staff radiologist interpretations for CTPA and CTV, respectively, with tendency towards improved IOA as residency level of training increased for CTPA, but not for CTV.

Preserving the educational value of call in a diagnostic radiology residency program

PURPOSE

Our study was designed to determine residents' opinions of the advantages, disadvantages and educational value of a traditional "Tandem Call" (TC) model as compared to night float (NF). Because TC is more representative of adult learning principles and constructivist theory, we hypothesized that resident satisfaction and educational outcomes would demonstrate a preference for, and the educational efficacy of, the TC model.

METHODS

We surveyed all residents in a university-based radiology residency on their opinions of TC and its educational value. Aggregate data from annual Graduate Medical Education Committee institutional surveys (2008-2012) and annual radiology alumni surveys (2009-2012) were reviewed as measures of satisfaction with TC. Performance on the ABR oral exam was a proxy for educational outcome. Quality data for the year of study and prior years in which TC was in effect were reviewed as a measure of patient safety.

RESULTS

The great majority of respondents attributed confidence/competence on call and added value to their education directly to TC. A majority believed that teamwork required for TC facilitated more positive relationships among residents and more peer teaching. Most said that they would not prefer NF. Almost all believed indirect supervision with attending backup aided in developing confidence in performance. Quality data confirmed a low number of discrepancies between preliminary resident and final attending reads.

CONCLUSIONS

TC provides a more consistent call experience throughout residency than NF. TC is valued by residents, facilitates retrieval-based learning and development of independence and efficiency, and parallels essential elements of team-based learning. Quality data suggests that lack of 24-hour attending supervision is not detrimental to patient safety.

Reliability of on-call radiology residents' interpretation of 64-slice CT pulmonary angiography for the detection of pulmonary embolism

BACKGROUND

Computed tomography (CT) angiography for pulmonary embolism (PE) is the present standard for diagnosing PE. In many teaching hospitals, radiology residents are the first to review the case and to make an initial interpretation of the images. Accurate diagnosis of PE is crucial, especially in the emergency care setting.

PURPOSE

To evaluate the discrepancies between resident and staff interpretations of 64-slice CT angiogram for PE.

MATERIAL AND METHODS

Discrepancies between the preliminary reports by the on-call radiology resident were compared to the final report by the staff radiologist in 215 consecutive cases of 64-slice CT angiogram performed for PE, from May 2005 to March 2008.

RESULTS

Discrepancies were noted in 25 of the 215 studies (11.6%). These residents' discrepancies consisted of three false-positive, four false-negative, and 18 equivocal cases. There was a decrease in the discrepancy rate from the second year to the fifth year of training by approximately 60%.

CONCLUSION

The rate of discrepancy fell steeply between the second and fifth year of the residents training from 18.5% to 6.9%. Our study suggests that it is reasonable to have on-call radiology residents perform the preliminary interpretations of 64-slice CT for PE studies.

CT in adults: systematic review and meta-analysis of interpretation discrepancy rates

PURPOSE

To use meta-analysis to determine the discrepancy rate when interpreting computed tomography (CT) studies performed in adult patients and to determine whether discrepancy rate differs on the basis of body region or level of radiologist training.

MATERIALS AND METHODS

MEDLINE and EMBASE were searched from 1946 to June 2012 by using the combination "radiology AND (error OR peer review)." Two reviewers independently selected studies that met the inclusion criteria and extracted study data. Total and major discrepancy rates were investigated with a random-effects meta-analysis, and subgroups were compared by using the χ(2) Q statistic. Subgroup analyses were performed on the basis of the level of training of the initial radiologist and the body system scanned.

RESULTS

Fifty-eight studies met the inclusion criteria (388 123 CT examinations). The pooled total discrepancy rate was 7.7% (95% confidence interval [CI]: 5.6%, 10.3%), and the major discrepancy rate was 2.4% (95% CI: 1.7%, 3.2%). The pooled major discrepancy rate was comparable for staff (2.9%; 95% CI: 1.2%, 6.7%) and residents (2.2%; 95% CI: 1.7%, 2.9%) (Q = 0.92, P = .633). The pooled major discrepancy rates for head CT (0.8%; 95% CI: 0.4%, 1.6%) and spine CT (0.7%; 95% CI: 0.2%, 2.7%) were lower than those for chest CT (2.8%; 95% CI: 1.5%, 5.4%) and abdominal CT (2.6%; 95% CI: 1.0%, 6.7%) (Q = 8.28, P = .041). Lack of blinding of the reference radiologist to the initial report was associated with a lower major discrepancy rate (2.0%; 95% CI: 1.4%, 2.7%; 43 studies) than when blinding was present (12.1%; 95% CI: 4.4%, 29.4%; five studies) (Q = 10.65, P = .001).

CONCLUSION

Potentially useful reference ranges were identified in the subgroup analyses on the basis of body region scanned at adult CT. However, considerable heterogeneity that is only partially explained by subgroup analysis signifies that further research is necessary--particularly regarding the question of blinding of the reference radiologist.

Pulmonary embolism

Pulmonary embolism (PE) remains one of the most challenging medical diseases in the emergency department. PE is a potentially life threatening diagnosis that is seen in patients with chest pain and/or dyspnea but can span the clinical spectrum of medical presentations. In addition, it does not have any particular clinical feature, laboratory test, or diagnostic modality that can independently and confidently exclude its possibility. This article offers a review of PE in the emergency department. It emphasizes the appropriate determination of pretest probability, the approach to diagnosis and management, and special considerations related to pregnancy and radiation exposure.

Incremental value of CT venography combined with pulmonary CT angiography for the detection of thromboembolic disease: systematic review and meta-analysis

OBJECTIVE

The objective of our study was to assess the incremental role of CT venography (CTV) combined with pulmonary CT angiography (CTA) in detecting venous thromboembolic disease with a systematic review and meta-analysis of the literature.

MATERIALS AND METHODS

MEDLINE, Embase, and Web of Science were searched for relevant original articles published from January 1, 1995, to December 31, 2009. A random-effects model was used to obtain the incremental value of CTV in detecting thromboembolic disease.

RESULTS

Twenty-four studies, which included 17,373 patients, met our inclusion criteria. A meta-analysis showed that CTV increased detection rates of venous thromboembolic disease by identifying an additional 3% of cases (95% CI, 2-4%) of isolated deep venous thrombosis (DVT). A subgroup analysis of a high-risk group did not show any difference in the detection of isolated DVT.

CONCLUSION

The addition of CTV results in the increased detection of thromboembolic disease. CTV combined with pulmonary CTA has a promising role as a quick and efficient test for venous thromboembolism.

Effect of work hours, caseload, shift type, and experience on resident call performance

RATIONALE AND OBJECTIVES

To analyze the independent effects of multiple variables on resident call performance.

MATERIALS AND METHODS

Independent radiology resident "on call" cross-sectional imaging interpretation quality assurance (QA) data obtained during a 171-day period at a single tertiary care Level 1 trauma teaching institution was reviewed. Clinically significant resident-faculty discrepancies were compared among three different call types: traditional single-day overnight call (OC, 15 hours/night after 9 daytime hours on weekdays), 7-night nightfloat (NF, 9 hours/night), and weekend day call (WD, 10 hours/day). Logistic regression analyses were performed to evaluate associations.

RESULTS

There were 119 (0.89%) clinically significant resident-faculty discordances from 13,424 cross-sectional interpretations: 56 (0.79%) from 7102 interpretations on 172 OC shifts, 39 (0.85%) from 4567 interpretations on 165 NF shifts, and 24 (1.4%) from 1755 interpretations on 49 WD shifts. Individual residents (n = 20) had a mean discrepancy rate of 0.9% (0.45%-1.9%). Overall, 102 (26.2%) of the shifts had at least one discordance. The following were associated with significantly (P < .001) increased discrepancy rates: junior vs. senior residents (odds ratio [OR] = 1.3 [1.2-1.4]), OC vs. NF (OR = 1.5 [1.3-1.6], WD vs. NF (OR = 1.4 [1.2-1.6]), weekend vs. weekday (OR = 1.3 [1.2-1.4]), and increasing cases/hour (OR = 1.6 [1.5-1.7]). Weekend OC shifts had a higher discrepancy rate (OR 1.3[1.2-1.5], P < .001) than weekday OC shifts despite a shorter workday (15 vs. 24 hours).

CONCLUSION

Increasing caseload, junior residents, and weekends are associated with a significantly higher discrepancy rate. OC is associated with a significantly higher discrepancy rate than NF. Measured discrepancy rates are low, regardless of call type.

[ECG-gated chest CT angiography: value for atypical chest pain evaluation]

PURPOSE

The aim of this study was to evaluate ECG-gated whole chest CTA as a routine triage tool for patients with acute chest pain.

MATERIAL AND METHODS

Whole chest CTA with retrospective ECG-gating was performed in 30 patients with acute atypical chest pain. The ten main segments of the coronary arteries, the pulmonary arteries, the aorta, and the myocardium (function, morphology) were independently analyzed by a resident and two senior radiologists. The inter-observer agreement between resident and senior radiologists was calculated. A final diagnosis was determined by consensus.

RESULTS

Thirty patients were included. The coronary artery segments, myocardium and pulmonary arteries were considered analyzable in 84%, 90% and 97% of cases respectively. A final diagnosis for the cause of pain was retained in 19 patients: significant coronary artery stenosis (5), pulmonary embolus (5), aortic dissection (1), hypokinetic cardiomyopathy (2), lung parenchymal abnormalities (5), and hiatus hernia (1). Inter-observer agreement ranged from 0.76 to 1 between senior radiologists and from 0.76 to 1 between resident and senior radiologists. The average time of image interpretation ranged from 14 to 15 minutes.

CONCLUSION

ECG-gated whole chest CT angiography appears as a promising tool for the evaluation of acute chest pain. Combined evaluation of appearance and function of the myocardium can reveal additional interesting information.