Real world validation of an AI-based CT hemorrhage detection tool

Introduction

Intracranial hemorrhage (ICH) is a potentially life-threatening medical event that requires expedited diagnosis with computed tomography (CT). Automated medical imaging triaging tools can rapidly bring scans containing critical abnormalities, such as ICH, to the attention of radiologists and clinicians. Here, we retrospectively investigated the real-world performance of VeriScout™, an artificial intelligence-based CT hemorrhage detection and triage tool.

Methods

Ground truth for the presence or absence of ICH was iteratively determined by expert consensus in an unselected dataset of 527 consecutively acquired non-contrast head CT scans, which were sub-grouped according to the presence of artefact, post-operative features and referral source. The performance of VeriScout™ was compared with the ground truths for all groups.

Results

VeriScout™ detected hemorrhage with a sensitivity of 0.92 (CI 0.84-0.96) and a specificity of 0.96 (CI 0.94-0.98) in the global dataset, exceeding the sensitivity of general radiologists (0.88) with only a minor relative decrement in specificity (0.98). Crucially, the AI tool detected 13/14 cases of subarachnoid hemorrhage, a potentially fatal condition that is often missed in emergency department settings. There was no decrement in the performance of VeriScout™ in scans containing artefact or postoperative change. Using an integrated informatics platform, VeriScout™ was deployed into the existing radiology workflow. Detected hemorrhage cases were flagged in the hospital radiology information system (RIS) and relevant, annotated, preview images made available in the picture archiving and communications system (PACS) within 10 min.

Conclusion

AI-based radiology worklist prioritization for critical abnormalities, such as ICH, may enhance patient care without adding to radiologist or clinician burden.

Deep learning algorithm in detecting intracranial hemorrhages on emergency computed tomographies

BACKGROUND

Highly accurate detection of intracranial hemorrhages (ICH) on head computed tomography (HCT) scans can prove challenging at high-volume centers. This study aimed to determine the number of additional ICHs detected by an artificial intelligence (AI) algorithm and to evaluate reasons for erroneous results at a level I trauma center with teleradiology services.

METHODS

In a retrospective multi-center cohort study, consecutive emergency non-contrast HCT scans were analyzed by a commercially available ICH detection software (AIDOC, Tel Aviv, Israel). Discrepancies between AI analysis and initial radiology report (RR) were reviewed by a blinded neuroradiologist to determine the number of additional ICHs detected and evaluate reasons leading to errors.

RESULTS

4946 HCT (05/2020-09/2020) from 18 hospitals were included in the analysis. 205 reports (4.1%) were classified as hemorrhages by both radiology report and AI. Out of a total of 162 (3.3%) discrepant reports, 62 were confirmed as hemorrhages by the reference neuroradiologist. 33 ICHs were identified exclusively via RRs. The AI algorithm detected an additional 29 instances of ICH, missed 12.4% of ICH and overcalled 1.9%; RRs missed 10.9% of ICHs and overcalled 0.2%. Many of the ICHs missed by the AI algorithm were located in the subarachnoid space (42.4%) and under the calvaria (48.5%). 85% of ICHs missed by RRs occurred outside of regular working-hours. Calcifications (39.3%), beam-hardening artifacts (18%), tumors (15.7%), and blood vessels (7.9%) were the most common reasons for AI overcalls. ICH size, image quality, and primary examiner experience were not found to be significantly associated with likelihood of incorrect AI results.

CONCLUSION

Complementing human expertise with AI resulted in a 12.2% increase in ICH detection. The AI algorithm overcalled 1.9% HCT.

TRIAL REGISTRATION

German Clinical Trials Register (DRKS-ID: DRKS00023593).

On-call radiology 2020: Where trainees look for help in a high stakes and time sensitive environment

BACKGROUND

The radiology trainee on-call experience has undergone many changes in the past decade. The development of numerous online information sources has changed the landscape of opportunities for trainees seeking information while on-call. In this study, we sought to understand the current on-call information seeking behaviors of radiology trainees.

METHODS

We surveyed radiology fellows and residents at three major metropolitan area academic institutions. Survey topics included demographic information, on-call volumes, on-call resource seeking behaviors, preferred first and second line on-call resources and rationale for particular resource usage.

RESULTS

A total of 78 responses from trainees were recorded, 30.5% of the entire surveyed population. 70.5% of trainees preferred Radiopaedia as their first line resource. 26.9% of trainees preferred StatDx as their second line resource. 75.6% of respondents preferred their first line resource because it was easiest and fastest to access. 70.3% of respondents assigned a rating of 4 out of 5 when asked how often information they look for is found while on-call. There was a statistically significant difference according to gender (p = 0.002) with a higher percentage of males listing Radiopaedia as their first line resource compared to females.

DISCUSSION

The radiology trainee on-call experience is influenced by various factors. Over the past decade, online resources, particularly the open access resource Radiopaedia and the paid service StatDx, have overwhelmingly become the preferred first and second line options, as demonstrated by our study results.

Update on establishing and managing an overnight emergency radiology division

Emergency department (ED) radiology divisions that serve to provide overnight attending coverage have become an increasingly common feature of radiology departments. The purpose of this article is to review the common ED radiology coverage models, describe desirable traits of emergency radiologists, and discuss workflow in the ED radiology setting. ED radiologists may be trained as ED radiologists or may develop the necessary skills and adopt the subspecialty. Choosing radiologists with the correct traits such as being a “night owl” and remaining calm under pressure and implementing an acceptable work schedule such as shift length of 9–10 h and a “one week on, two weeks off” schedule contribute to sustainability of the position. Strategies to address the unique stressors and workflow challenges of overnight emergency radiology coverage are also presented. Workflow facilitators including trainees, PAs, radiology assistants, and clerks all have roles to play in managing high case volumes and in making sure that the service is well staffed. Usage of artificial intelligence software is the latest technique to streamline workflow by identifying cases which should be prioritized on a busy worklist. Implementing such strategies will maintain quality of care for patients regardless of time of day as well as sustainability and quality of life for overnight emergency radiologists.