Integrating artificial intelligence and natural language processing for computer-assisted reporting and report understanding in nuclear cardiology

Natural language processing (NLP) offers many opportunities in Nuclear Cardiology. These opportunities include applications in converting nuclear cardiology imaging reports to digital searchable information that may be used as Big Data for machine learning and registries. Another major NLP application is, with the support of AI, in automatically translating MPI image features directly into nuclear cardiology reports. This review describes the symbiotic relationship between AI and NLP in that NLP is being used to facilitate AI applications and, AI techniques are being used to facilitate NLP. This article reviews the fundamentals of NLP and describes various conventional and AI techniques that have been applied in imaging. Key nuclear cardiology applications are reviewed such as conversion of MPI free-text reports to digital documents as well as direct conversion of MPI images into structured medical reports.

Multi-center Integrating Radiomics, Structured Reports, and Machine Learning Algorithms for Assisted Classification of COVID-19 in Lung Computed Tomography

Purpose

To evaluate the classification performance of structured report features, radiomics, and machine learning (ML) models to differentiate between Coronavirus Disease 2019 (COVID-19) and other types of pneumonia using chest computed tomography (CT) scans.

Methods

Sixty-four COVID-19 subjects and 64 subjects with non-COVID-19 pneumonia were selected. The data was split into two independent cohorts: one for the structured report, radiomic feature selection and model building (n = 73), and another for model validation (n = 55). Physicians performed readings with and without machine learning support. The model's sensitivity and specificity were calculated, and inter-rater reliability was assessed using Cohen's Kappa agreement coefficient.

Results

Physicians performed with mean sensitivity and specificity of 83.4 and 64.3%, respectively. When assisted with machine learning, the mean sensitivity and specificity increased to 87.1 and 91.1%, respectively. In addition, machine learning improved the inter-rater reliability from moderate to substantial.

Conclusion

Integrating structured reports and radiomics promises assisted classification of COVID-19 in CT chest scans.

Highdicom: a Python Library for Standardized Encoding of Image Annotations and Machine Learning Model Outputs in Pathology and Radiology

Machine learning (ML) is revolutionizing image-based diagnostics in pathology and radiology. ML models have shown promising results in research settings, but the lack of interoperability between ML systems and enterprise medical imaging systems has been a major barrier for clinical integration and evaluation. The DICOM® standard specifies information object definitions (IODs) and services for the representation and communication of digital images and related information, including image-derived annotations and analysis results. However, the complexity of the standard represents an obstacle for its adoption in the ML community and creates a need for software libraries and tools that simplify working with datasets in DICOM format. Here we present the highdicom library, which provides a high-level application programming interface (API) for the Python programming language that abstracts low-level details of the standard and enables encoding and decoding of image-derived information in DICOM format in a few lines of Python code. The highdicom library leverages NumPy arrays for efficient data representation and ties into the extensive Python ecosystem for image processing and machine learning. Simultaneously, by simplifying creation and parsing of DICOM-compliant files, highdicom achieves interoperability with the medical imaging systems that hold the data used to train and run ML models, and ultimately communicate and store model outputs for clinical use. We demonstrate through experiments with slide microscopy and computed tomography imaging, that, by bridging these two ecosystems, highdicom enables developers and researchers to train and evaluate state-of-the-art ML models in pathology and radiology while remaining compliant with the DICOM standard and interoperable with clinical systems at all stages. To promote standardization of ML research and streamline the ML model development and deployment process, we made the library available free and open-source at https://github.com/herrmannlab/highdicom .

What Influences the Way Radiologists Express Themselves in Their Reports? A Quantitative Assessment Using Natural Language Processing

Although using standardized reports is encouraged, most emergency radiological reports in France remain in free-text format that can be mined with natural language processing for epidemiological purposes, activity monitoring or data collection. These reports are obtained under various on-call conditions by radiologists with various backgrounds. Our aim was to investigate what influences the radiologists’ written expressions. To do so, this retrospective multicentric study included 30,227 emergency radiological reports of computed tomography scans and magnetic resonance imaging involving exactly one body region, only with pathological findings, interpreted from 2019–09-01 to 2020–02-28 by 165 radiologists. After text pre-processing, one-word tokenization and use of dictionaries for stop words, polarity, sentiment and uncertainty, 11 variables depicting the structure and content of words and sentences in the reports were extracted and summarized to 3 principal components capturing 93.7% of the dataset variance. In multivariate analysis, the 1^st principal component summarized the length and lexical diversity of the reports and was significantly influenced by the weekday, time slot, workload, number of examinations previously interpreted by the radiologist during the on-call period, type of examination, emergency level and radiologists’ gender (P value range: < 0.0001–0.0029). The 2^nd principal component summarized negative formulations, polarity and sentence length and was correlated with the number of examination previously interpreted by the radiologist, type of examination, emergency level, imaging modality and radiologists’ experience (P value range: < 0.0001–0.0032). The last principal component summarized questioning, uncertainty and polarity and was correlated with the type of examination and emergency level (all P values < 0.0001). Thus, the length, structure and content of emergency radiological reports were significantly influenced by organizational, radiologist- and examination-related characteristics, highlighting the subjectivity and variability in the way radiologists express themselves during their clinical activity. These findings advocate for more homogeneous practices in radiological reporting and stress the need to consider these influential features when developing models based on natural language processing.

Structured reporting of CT scans of patients with trauma leads to faster, more detailed diagnoses: An experimental study

PURPOSE

This study aimed to determine whether structured reports (SRs) reduce reporting time and/or increase the level of detail for trauma CT scans compared to free-text reports (FTRs).

METHOD

Eight radiology residents used SRs and FTRs to describe 14 whole-body CT scans of patients with polytrauma in a simulated emergency room setting. Each resident created both a brief report and a detailed report for each case using one of the two formats. We measured the time to complete the detailed reports and established a scoring system to objectively measure report completeness and the level of detail. Scoring sheets divided the CT findings into main and secondary criteria. Finally, the radiological residents completed a questionnaire on their opinions of the SRs and FTRs.

RESULTS

The detailed SRs were completed significantly faster than the detailed FTRs (mean 19 min vs. 25 min; p < 0.001). The maximum allowance of 25 min was used for 25% of SRs and 59% of FTRs. For brief reports, the SRs contained more secondary criteria than the FTRs (p = 0.001), but no significant differences were detected in main criteria. Study participants rated their own SRs as significantly more time-efficient, concise, and clearly structured compared to the FTRs. However, SRs and FTRs were rated similarly for quality, accuracy, and completeness.

CONCLUSION

We found that SRs for whole-body trauma CT add clinical value compared to FTRs because SRs reduce reporting time and increase the level of detail for trauma CT scans.

Feasibility of integrating computer-aided diagnosis with structured reports of prostate multiparametric MRI

OBJECTIVES

To evaluate the feasibility of integrating computer-aided diagnosis (CAD) with structured reports of prostate multiparametric MRI (mpMRI).

METHODS

This retrospective study enrolled 153 patients who underwent prostate mpMRI for the purpose of targeted biopsy; patients were divided into a group with clinically significant prostate cancer (csPCa, Gleason score ≥ 3 + 4, n = 89) and a group with non-csPCa (n = 64). Ten inexperienced radiologists retrospectively evaluated these cases (single reader per case) twice using structured reports, and they were blinded to the pathologic results. Initially, the readers interpreted mpMRI without CAD. Six weeks later, they evaluated the same cases again with CAD assistance. At each time of image interpretation, lesions detected by the readers were marked on the prostate vector map in structured reports, and a PI-RADS score was given to each lesion. Diagnostic efficacy and reading time were evaluated for the two reading sessions.

RESULTS

With the assistance of CAD, the overall diagnostic efficacy was improved, i.e., the AUC increased from 0.83 to 0.89 (p = 0.018). Specifically, per-patient sensitivity (84.3% vs. 93.3%) and per-lesion sensitivity (76.7% vs. 88.8%) were significantly improved (all p < 0.05). Per-patient specificity with CAD (65.6%) was higher than that without CAD (56.3%), but statistical significance was not reached (p = 0.238). The reading time for each case decreased from 10.9 min to 7.8 min (p < 0.001).

CONCLUSIONS

It is feasible to integrate CAD with structured reports of prostate mpMRI. This reading paradigm can improve the diagnostic sensitivity of csPCa detection and reduce reading time among inexperienced radiologists.

Structured reporting adds clinical value in primary CT staging of diffuse large B-cell lymphoma

OBJECTIVES

To evaluate whether template-based structured reports (SRs) add clinical value to primary CT staging in patients with diffuse large B-cell lymphoma (DLBCL) compared to free-text reports (FTRs).

METHODS

In this two-centre study SRs and FTRs were acquired for 16 CT examinations. Thirty-two reports were independently scored by four haematologists using a questionnaire addressing completeness of information, structure, guidance for patient management and overall quality. The questionnaire included yes-no, 10-point Likert scale and 5-point scale questions. Altogether 128 completed questionnaires were evaluated. Non-parametric Wilcoxon signed-rank test and McNemar's test were used for statistical analysis.

RESULTS

SRs contained information on affected organs more often than FTRs (95 % vs. 66 %). More SRs commented on extranodal involvement (91 % vs. 62 %). Sufficient information for Ann-Arbor classification was included in more SRs (89 % vs. 64 %). Information extraction was quicker from SRs (median rating on 10-point Likert scale=9 vs. 6; 7-10 vs. 4-8 interquartile range). SRs had better comprehensibility (9 vs. 7; 8-10 vs. 5-8). Contribution of SRs to clinical decision-making was higher (9 vs. 6; 6-10 vs. 3-8). SRs were of higher quality (p < 0.001). All haematologists preferred SRs over FTRs.

CONCLUSIONS

Structured reporting of CT examinations for primary staging in patients with DLBCL adds clinical value compared to FTRs by increasing completeness of reports, facilitating information extraction and improving patient management.

KEY POINTS

• Structured reporting in CT helps clinicians to assess patients with lymphoma. • This two-centre study showed that structured reporting improves information content and extraction. • Patient management may be improved by structured reporting. • Clinicians preferred structured reports over free-text reports.