Fully automated convolutional neural network-based affine algorithm improves liver registration and lesion co-localization on hepatobiliary phase T1-weighted MR images

Deep learning-based arterial subtraction images improve the detection of LR-TR algorithm for viable HCC on extracellular agents-enhanced MRI

PURPOSE

To determine the role of deep learning-based arterial subtraction images in viability assessment on extracellular agents-enhanced MRI using LR-TR algorithm.

METHODS

Patients diagnosed with HCC who underwent locoregional therapy were retrospectively collected. We constructed a deep learning-based subtraction model and automatically generated arterial subtraction images. Two radiologists evaluated LR-TR category on ordinary images and then evaluated again on ordinary images plus arterial subtraction images after a 2-month washout period. The reference standard for viability was tumor stain on the digital subtraction hepatic angiography within 1 month after MRI.

RESULTS

286 observations of 105 patients were ultimately enrolled. 157 observations were viable and 129 observations were nonviable according to the reference standard. The sensitivity and accuracy of LR-TR algorithm for detecting viable HCC significantly increased with the application of arterial subtraction images (87.9% vs. 67.5%, p < 0.001; 86.4% vs. 75.9%, p < 0.001). And the specificity slightly decreased without significant difference when the arterial subtraction images were added (84.5% vs. 86.0%, p = 0.687). The AUC of LR-TR algorithm significantly increased with the addition of arterial subtraction images (0.862 vs. 0.768, p < 0.001). The arterial subtraction images also improved inter-reader agreement (0.857 vs. 0.727).

CONCLUSION

Extended application of deep learning-based arterial subtraction images on extracellular agents-enhanced MRI can increase the sensitivity of LR-TR algorithm for detecting viable HCC without significant change in specificity.

Deep learning-based affine medical image registration for multimodal minimal-invasive image-guided interventions - A comparative study on generalizability

Multimodal image registration is applied in medical image analysis as it allows the integration of complementary data from multiple imaging modalities. In recent years, various neural network-based approaches for medical image registration have been presented in papers, but due to the use of different datasets, a fair comparison is not possible. In this research 20 different neural networks for an affine registration of medical images were implemented. The networks' performance and the networks' generalizability to new datasets were evaluated using two multimodal datasets - a synthetic and a real patient dataset - of three-dimensional CT and MR images of the liver. The networks were first trained semi-supervised using the synthetic dataset and then evaluated on the synthetic dataset and the unseen patient dataset. Afterwards, the networks were finetuned on the patient dataset and subsequently evaluated on the patient dataset. The networks were compared using our own developed CNN as benchmark and a conventional affine registration with SimpleElastix as baseline. Six networks improved the pre-registration Dice coefficient of the synthetic dataset significantly (p-value < 0.05) and nine networks improved the pre-registration Dice coefficient of the patient dataset significantly and are therefore able to generalize to the new datasets used in our experiments. Many different machine learning-based methods have been proposed for affine multimodal medical image registration, but few are generalizable to new data and applications. It is therefore necessary to conduct further research in order to develop medical image registration techniques that can be applied more widely.

Unsupervised deep learning registration model for multimodal brain images

Multimodal image registration is a key for many clinical image-guided interventions. However, it is a challenging task because of complicated and unknown relationships between different modalities. Currently, deep supervised learning is the state-of-theart method at which the registration is conducted in end-to-end manner and one-shot. Therefore, a huge ground-truth data is required to improve the results of deep neural networks for registration. Moreover, supervised methods may yield models that bias towards annotated structures. Here, to deal with above challenges, an alternative approach is using unsupervised learning models. In this study, we have designed a novel deep unsupervised Convolutional Neural Network (CNN)-based model based on computer tomography/magnetic resonance (CT/MR) co-registration of brain images in an affine manner. For this purpose, we created a dataset consisting of 1100 pairs of CT/MR slices from the brain of 110 neuropsychic patients with/without tumor. At the next step, 12 landmarks were selected by a well-experienced radiologist and annotated on each slice resulting in the computation of series of metrics evaluation, target registration error (TRE), Dice similarity, Hausdorff, and Jaccard coefficients. The proposed method could register the multimodal images with TRE 9.89, Dice similarity 0.79, Hausdorff 7.15, and Jaccard 0.75 that are appreciable for clinical applications. Moreover, the approach registered the images in an acceptable time 203 ms and can be appreciable for clinical usage due to the short registration time and high accuracy. Here, the results illustrated that our proposed method achieved competitive performance against other related approaches from both reasonable computation time and the metrics evaluation.

Segmentation-guided multi-modal registration of liver images for dose estimation in SIRT

Purpose

Selective internal radiation therapy (SIRT) requires a good liver registration of multi-modality images to obtain precise dose prediction and measurement. This study investigated the feasibility of liver registration of CT and MR images, guided by segmentation of the liver and its landmarks. The influence of the resulting lesion registration on dose estimation was evaluated.

Methods

The liver segmentation was done with a convolutional neural network (CNN), and the landmarks were segmented manually. Our image-based registration software and its liver-segmentation-guided extension (CNN-guided) were tuned and evaluated with 49 CT and 26 MR images from 20 SIRT patients. Each liver registration was evaluated by the root mean square distance (RMSD) of mean surface distance between manually delineated liver contours and mass center distance between manually delineated landmarks (lesions, clips, etc.). The root mean square of RMSDs (RRMSD) was used to evaluate all liver registrations. The CNN-guided registration was further extended by incorporating landmark segmentations (CNN&LM-guided) to assess the value of additional landmark guidance. To evaluate the influence of segmentation-guided registration on dose estimation, mean dose and volume percentages receiving at least 70 Gy (V70) estimated on the ^99mTc-labeled macro-aggregated albumin (^99mTc-MAA) SPECT were computed, either based on lesions from the reference ^99mTc-MAA CT (reference lesions) or from the registered floating CT or MR images (registered lesions) using the CNN- or CNN&LM-guided algorithms.

Results

The RRMSD decreased for the floating CTs and MRs by 1.0 mm (11%) and 3.4 mm (34%) using CNN guidance for the image-based registration and by 2.1 mm (26%) and 1.4 mm (21%) using landmark guidance for the CNN-guided registration. The quartiles for the relative mean dose difference (the V70 difference) between the reference and registered lesions and their correlations [25th, 75th; r] are as follows: [− 5.5% (− 1.3%), 5.6% (3.4%); 0.97 (0.95)] and [− 12.3% (− 2.1%), 14.8% (2.9%); 0.96 (0.97)] for the CNN&LM- and CNN-guided CT to CT registrations, [− 7.7% (− 6.6%), 7.0% (3.1%); 0.97 (0.90)] and [− 15.1% (− 11.3%), 2.4% (2.5%); 0.91 (0.78)] for the CNN&LM- and CNN-guided MR to CT registrations.

Conclusion

Guidance by CNN liver segmentations and landmarks markedly improves the performance of the image-based registration. The small mean dose change between the reference and registered lesions demonstrates the feasibility of applying the CNN&LM- or CNN-guided registration to volume-level dose prediction. The CNN&LM- and CNN-guided registrations for CTs can be applied to voxel-level dose prediction according to their small V70 change for most lesions. The CNN-guided MR to CT registration still needs to incorporate landmark guidance for smaller change of voxel-level dose estimation.

The Application and Development of Deep Learning in Radiotherapy: A Systematic Review

With the massive use of computers, the growth and explosion of data has greatly promoted the development of artificial intelligence (AI). The rise of deep learning (DL) algorithms, such as convolutional neural networks (CNN), has provided radiation oncologists with many promising tools that can simplify the complex radiotherapy process in the clinical work of radiation oncology, improve the accuracy and objectivity of diagnosis, and reduce the workload, thus enabling clinicians to spend more time on advanced decision-making tasks. As the development of DL gets closer to clinical practice, radiation oncologists will need to be more familiar with its principles to properly evaluate and use this powerful tool. In this paper, we explain the development and basic concepts of AI and discuss its application in radiation oncology based on different task categories of DL algorithms. This work clarifies the possibility of further development of DL in radiation oncology.

CNN color-coded difference maps accurately display longitudinal changes in liver MRI-PDFF

OBJECTIVES

To assess the feasibility of a CNN-based liver registration algorithm to generate difference maps for visual display of spatiotemporal changes in liver PDFF, without needing manual annotations.

METHODS

This retrospective exploratory study included 25 patients with suspected or confirmed NAFLD, who underwent PDFF-MRI at two time points at our institution. PDFF difference maps were generated by applying a CNN-based liver registration algorithm, then subtracting follow-up from baseline PDFF maps. The difference maps were post-processed by smoothing (5 cm² round kernel) and applying a categorical color scale. Two fellowship-trained abdominal radiologists and one radiology resident independently reviewed difference maps to visually determine segmental PDFF change. Their visual assessment was compared with manual ROI-based measurements of each Couinaud segment and whole liver PDFF using intraclass correlation (ICC) and Bland-Altman analysis. Inter-reader agreement for visual assessment was calculated (ICC).

RESULTS

The mean patient age was 49 years (12 males). Baseline and follow-up PDFF ranged from 2.0 to 35.3% and 3.5 to 32.0%, respectively. PDFF changes ranged from - 20.4 to 14.1%. ICCs against the manual reference exceeded 0.95 for each reader, except for segment 2 (2 readers ICC = 0.86-0.91) and segment 4a (reader 3 ICC = 0.94). Bland-Altman limits of agreement were within 5% across all three readers. Inter-reader agreement for visually assessed PDFF change (whole liver and segmental) was excellent (ICCs > 0.96), except for segment 2 (ICC = 0.93).

CONCLUSIONS

Visual assessment of liver segmental PDFF changes using a CNN-generated difference map strongly agreed with manual estimates performed by an expert reader and yielded high inter-reader agreement.

KEY POINTS

• Visual assessment of longitudinal changes in quantitative liver MRI can be performed using a CNN-generated difference map and yields strong agreement with manual estimates performed by expert readers.

Alternative approach of hepatocellular carcinoma surveillance: abbreviated MRI

This review focuses on emerging abbreviated magnetic resonance imaging (AMRI) surveillance of patients with chronic liver disease for hepatocellular carcinoma (HCC). This surveillance strategy has been proposed as a high-sensitivity alternative to ultrasound for identification of patients with early-stage HCC, particularly in patients with cirrhosis or obesity, in whom sonographic visualization of small tumors may be compromised. Three general AMRI approaches have been developed and studied in the literature - non-contrast AMRI, dynamic contrast-enhanced AMRI, and hepatobiliary phase contrast-enhanced AMRI - each comprising a small number of selected sequences specifically tailored for HCC detection. The rationale, general technique, advantages and disadvantages, and diagnostic performance of each AMRI approach is explained. Additionally, current gaps in knowledge and future directions are discussed. Based on emerging evidence, we cautiously recommend the use of AMRI for HCC surveillance in situations where ultrasound is compromised.