Improving automatic liver tumor segmentation in late-phase MRI using multi-model training and 3D convolutional neural networks

Accurate segmentation of liver tumor from multi-modality non-contrast images using a dual-stream multi-level fusion framework

The use of multi-modality non-contrast images (i.e., T1FS, T2FS and DWI) for segmenting liver tumors provides a solution by eliminating the use of contrast agents and is crucial for clinical diagnosis. However, this remains a challenging task to discover the most useful information to fuse multi-modality images for accurate segmentation due to inter-modal interference. In this paper, we propose a dual-stream multi-level fusion framework (DM-FF) to, for the first time, accurately segment liver tumors from non-contrast multi-modality images directly. Our DM-FF first designs an attention-based encoder-decoder to effectively extract multi-level feature maps corresponding to a specified representation of each modality. Then, DM-FF creates two types of fusion modules, in which a module fuses learned features to obtain a shared representation across multi-modality images to exploit commonalities and improve the performance, and a module fuses the decision evidence of segment to discover differences between modalities to prevent interference caused by modality's conflict. By integrating these three components, DM-FF enables multi-modality non-contrast images to cooperate with each other and enables an accurate segmentation. Evaluation on 250 patients including different types of tumors from two MRI scanners, DM-FF achieves a Dice of 81.20%, and improves performance (Dice by at least 11%) when comparing the eight state-of-the-art segmentation architectures. The results indicate that our DM-FF significantly promotes the development and deployment of non-contrast liver tumor technology.

Deep Learning-Based Approach for Identifying and Measuring Focal Liver Lesions on Contrast-Enhanced MRI

BACKGROUND

The number of focal liver lesions (FLLs) detected by imaging has increased worldwide, highlighting the need to develop a robust, objective system for automatically detecting FLLs.

PURPOSE

To assess the performance of the deep learning-based artificial intelligence (AI) software in identifying and measuring lesions on contrast-enhanced magnetic resonance imaging (MRI) images in patients with FLLs.

STUDY TYPE

Retrospective.

SUBJECTS

395 patients with 1149 FLLs.

FIELD STRENGTH/SEQUENCE

The 1.5 T and 3 T scanners, including T1-, T2-, diffusion-weighted imaging, in/out-phase imaging, and dynamic contrast-enhanced imaging.

ASSESSMENT

The diagnostic performance of AI, radiologist, and their combination was compared. Using 20 mm as the cut-off value, the lesions were divided into two groups, and then divided into four subgroups: <10, 10-20, 20-40, and ≥40 mm, to evaluate the sensitivity of radiologists and AI in the detection of lesions of different sizes. We compared the pathologic sizes of 122 surgically resected lesions with measurements obtained using AI and those made by radiologists.

STATISTICAL TESTS

McNemar test, Bland-Altman analyses, Friedman test, Pearson's chi-squared test, Fisher's exact test, Dice coefficient, and intraclass correlation coefficients. A P-value <0.05 was considered statistically significant.

RESULTS

The average Dice coefficient of AI in segmentation of liver lesions was 0.62. The combination of AI and radiologist outperformed the radiologist alone, with a significantly higher detection rate (0.894 vs. 0.825) and sensitivity (0.883 vs. 0.806). The AI showed significantly sensitivity than radiologists in detecting all lesions <20 mm (0.848 vs. 0.788). Both AI and radiologists achieved excellent detection performance for lesions ≥20 mm (0.867 vs. 0.881, P = 0.671). A remarkable agreement existed in the average tumor sizes among the three measurements (P = 0.174).

DATA CONCLUSION

AI software based on deep learning exhibited practical value in automatically identifying and measuring liver lesions.

LEVEL OF EVIDENCE: 4

TECHNICAL EFFICACY

Stage 2.

Convolutional Neural Networks to Study Contrast-Enhanced Magnetic Resonance Imaging-Based Skeletal Calf Muscle Perfusion in Peripheral Artery Disease

Peripheral artery disease (PAD) is associated with impaired blood flow in the lower extremities and histopathologic changes of the skeletal calf muscles, resulting in abnormal microvascular perfusion. We studied the use of convolution neural networks (CNNs) to differentiate patients with PAD from matched controls using perfusion pattern features from contrast-enhanced magnetic resonance imaging (CE-MRI) of the skeletal calf muscles. We acquired CE-MRI based skeletal calf muscle perfusion in 56 patients (36 patients with PAD and 20 matched controls). Microvascular perfusion imaging was performed after reactive hyperemia at the midcalf level, with a temporal resolution of 409 ms. We analyzed perfusion scans up to 2 minutes indexed from the local precontrast arrival time frame. Skeletal calf muscles, including the anterior muscle, lateral muscle, deep posterior muscle group, and the soleus and gastrocnemius muscles, were segmented semiautomatically. Segmented muscles were represented as 3-dimensional Digital Imaging and Communications in Medicine stacks of CE-MRI perfusion scans for deep learning (DL) analysis. We tested several CNN models for the 3-dimensional CE-MRI perfusion stacks to classify patients with PAD from matched controls. A total of 2 of the best performing CNNs (resNet and divNet) were selected to develop the final classification model. A peak accuracy of 75% was obtained for resNet and divNet. Specificity was 80% and 94% for resNet and divNet, respectively. In conclusion, DL using CNNs and CE-MRI skeletal calf muscle perfusion can discriminate patients with PAD from matched controls. DL methods may be of interest for the study of PAD.

Advances in gross tumor target volume determination in radiotherapy for patients with hepatocellular carcinoma

Hepatocellular Carcinoma (HCC) is one of the most common malignant neoplasms. With the advancement of technology, the precision of radiotherapy (RT) for HCC has considerably increased, and it is an indispensable modality in the comprehensive management of HCC. Some RT techniques increase the radiation dose to HCC, which decreases the radiation dose delivered to the surrounding normal liver tissue. This approach significantly improves the efficacy of HCC treatment and reduces the incidence of Radiation-induced Liver Disease (RILD). Clear imaging and precise determination of the Gross Target Volume (GTV) are prerequisites of precise RT of HCC. The main hindrances in determining the HCC GTV include indistinct tumor boundaries on imaging and the impact on respiratory motion. The integration of multimodal imaging, four-dimensional imaging, and artificial intelligence (AI) techniques can help overcome challenges for HCC GTV. In this article, the advancements in medical imaging and precise determination for HCC GTV have been reviewed, providing a framework for the precise RT of HCC.

Deep learning for the automatic detection and segmentation of parotid gland tumors on MRI

OBJECTIVES

Parotid gland tumors (PGTs) often occur as incidental findings on magnetic resonance images (MRI) that may be overlooked. This study aimed to construct and validate a deep learning model to automatically identify parotid glands (PGs) with a PGT from normal PGs, and in those with a PGT to segment the tumor.

MATERIALS AND METHODS

The nnUNet combined with a PG-specific post-processing procedure was used to develop the deep learning model trained on T1-weighed images (T1WI) in 311 patients (180 PGs with tumors and 442 normal PGs) and fat-suppressed (FS)-T2WI in 257 patients (125 PGs with tumors and 389 normal PGs), for detecting and segmenting PGTs with five-fold cross-validation. Additional validation set separated by time, comprising T1WI in 34 and FS-T2WI in 41 patients, was used to validate the model performance.

RESULTS AND CONCLUSION

To identify PGs with tumors from normal PGs, using combined T1WI and FS-T2WI, the deep learning model achieved an accuracy, sensitivity and specificity of 98.2% (497/506), 100% (119/119) and 97.7% (378/387), respectively, in the cross-validation set and 98.5% (67/68), 100% (20/20) and 97.9% (47/48), respectively, in the validation set. For patients with PGTs, automatic segmentation of PGTs on T1WI and FS-T2WI achieved mean dice coefficients of 86.1% and 84.2%, respectively, in the cross-validation set, and of 85.9% and 81.0%, respectively, in the validation set. The proposed deep learning model may assist the detection and segmentation of PGTs and, by acting as a second pair of eyes, ensure that incidentally detected PGTs on MRI are not missed.

Automatic segmentation of hepatocellular carcinoma on dynamic contrast-enhanced MRI based on deep learning

Objective. Precise hepatocellular carcinoma (HCC) detection is crucial for clinical management. While studies focus on computed tomography-based automatic algorithms, there is a rareness of research on automatic detection based on dynamic contrast enhanced (DCE) magnetic resonance imaging. This study is to develop an automatic detection and segmentation deep learning model for HCC using DCE.Approach: DCE images acquired from 2016 to 2021 were retrospectively collected. Then, 382 patients (301 male; 81 female) with 466 lesions pathologically confirmed were included and divided into an 80% training-validation set and a 20% independent test set. For external validation, 51 patients (42 male; 9 female) in another hospital from 2018 to 2021 were included. The U-net architecture was modified to accommodate multi-phasic DCE input. The model was trained with the training-validation set using five-fold cross-validation, and furtherly evaluated with the independent test set using comprehensive metrics for segmentation and detection performance. The proposed automatic segmentation model consisted of five main steps: phase registration, automatic liver region extraction using a pre-trained model, automatic HCC lesion segmentation using the multi-phasic deep learning model, ensemble of five-fold predictions, and post-processing using connected component analysis to enhance the performance to refine predictions and eliminate false positives.Main results. The proposed model achieved a mean dice similarity coefficient (DSC) of 0.81 ± 0.11, a sensitivity of 94.41 ± 15.50%, a precision of 94.19 ± 17.32%, and 0.14 ± 0.48 false positive lesions per patient in the independent test set. The model detected 88% (80/91) HCC lesions in the condition of DSC > 0.5, and the DSC per tumor was 0.80 ± 0.13. In the external set, the model detected 92% (58/62) lesions with 0.12 ± 0.33 false positives per patient, and the DSC per tumor was 0.75 ± 0.10.Significance.This study developed an automatic detection and segmentation deep learning model for HCC using DCE, which yielded promising post-processed results in accurately identifying and delineating HCC lesions.

Radiomics and liver: Where we are and where we are headed?

Hepatic diffuse conditions and focal liver lesions represent two of the most common scenarios to face in everyday radiological clinical practice. Thanks to the advances in technology, radiology has gained a central role in the management of patients with liver disease, especially due to its high sensitivity and specificity. Since the introduction of computed tomography (CT) and magnetic resonance imaging (MRI), radiology has been considered the non-invasive reference modality to assess and characterize liver pathologies. In recent years, clinical practice has moved forward to a quantitative approach to better evaluate and manage each patient with a more fitted approach. In this setting, radiomics has gained an important role in helping radiologists and clinicians characterize hepatic pathological entities, in managing patients, and in determining prognosis. Radiomics can extract a large amount of data from radiological images, which can be associated with different liver scenarios. Thanks to its wide applications in ultrasonography (US), CT, and MRI, different studies were focused on specific aspects related to liver diseases. Even if broadly applied, radiomics has some advantages and different pitfalls. This review aims to summarize the most important and robust studies published in the field of liver radiomics, underlying their main limitations and issues, and what they can add to the current and future clinical practice and literature.

Deep Learning Framework for Liver Segmentation from T1-Weighted MRI Images

The human liver exhibits variable characteristics and anatomical information, which is often ambiguous in radiological images. Machine learning can be of great assistance in automatically segmenting the liver in radiological images, which can be further processed for computer-aided diagnosis. Magnetic resonance imaging (MRI) is preferred by clinicians for liver pathology diagnosis over volumetric abdominal computerized tomography (CT) scans, due to their superior representation of soft tissues. The convenience of Hounsfield unit (HoU) based preprocessing in CT scans is not available in MRI, making automatic segmentation challenging for MR images. This study investigates multiple state-of-the-art segmentation networks for liver segmentation from volumetric MRI images. Here, T1-weighted (in-phase) scans are investigated using expert-labeled liver masks from a public dataset of 20 patients (647 MR slices) from the Combined Healthy Abdominal Organ Segmentation grant challenge (CHAOS). The reason for using T1-weighted images is that it demonstrates brighter fat content, thus providing enhanced images for the segmentation task. Twenty-four different state-of-the-art segmentation networks with varying depths of dense, residual, and inception encoder and decoder backbones were investigated for the task. A novel cascaded network is proposed to segment axial liver slices. The proposed framework outperforms existing approaches reported in the literature for the liver segmentation task (on the same test set) with a dice similarity coefficient (DSC) score and intersect over union (IoU) of 95.15% and 92.10%, respectively.

Joint liver and hepatic lesion segmentation in MRI using a hybrid CNN with transformer layers

Backgound and Objective: Deep learning-based segmentation of the liver and hepatic lesions therein steadily gains relevance in clinical practice due to the increasing incidence of liver cancer each year. Whereas various network variants with overall promising results in the field of medical image segmentation have been successfully developed over the last years, almost all of them struggle with the challenge of accurately segmenting hepatic lesions in magnetic resonance imaging (MRI). This led to the idea of combining elements of convolutional and transformer-based architectures to overcome the existing limitations.

METHODS

This work presents a hybrid network called SWTR-Unet, consisting of a pretrained ResNet, transformer blocks as well as a common Unet-style decoder path. This network was primarily applied to single-modality non-contrast-enhanced liver MRI and additionally to the publicly available computed tomography (CT) data of the liver tumor segmentation (LiTS) challenge to verify the applicability on other modalities. For a broader evaluation, multiple state-of-the-art networks were implemented and applied, ensuring direct comparability. Furthermore, correlation analysis and an ablation study were carried out, to investigate various influencing factors on the segmentation accuracy of the presented method.

RESULTS

With Dice similarity scores of averaged 98±2% for liver and 81±28% lesion segmentation on the MRI dataset and 97±2% and 79±25%, respectively on the CT dataset, the proposed SWTR-Unet proved to be a precise approach for liver and hepatic lesion segmentation with state-of-the-art results for MRI and competing accuracy in CT imaging.

CONCLUSION

The achieved segmentation accuracy was found to be on par with manually performed expert segmentations as indicated by inter-observer variabilities for liver lesion segmentation. In conclusion, the presented method could save valuable time and resources in clinical practice.

Automatic Liver Tumor Segmentation from CT Images Using Graph Convolutional Network

Segmenting the liver and liver tumors in computed tomography (CT) images is an important step toward quantifiable biomarkers for a computer-aided decision-making system and precise medical diagnosis. Radiologists and specialized physicians use CT images to diagnose and classify liver organs and tumors. Because these organs have similar characteristics in form, texture, and light intensity values, other internal organs such as the heart, spleen, stomach, and kidneys confuse visual recognition of the liver and tumor division. Furthermore, visual identification of liver tumors is time-consuming, complicated, and error-prone, and incorrect diagnosis and segmentation can hurt the patient's life. Many automatic and semi-automatic methods based on machine learning algorithms have recently been suggested for liver organ recognition and tumor segmentation. However, there are still difficulties due to poor recognition precision and speed and a lack of dependability. This paper presents a novel deep learning-based technique for segmenting liver tumors and identifying liver organs in computed tomography maps. Based on the LiTS17 database, the suggested technique comprises four Chebyshev graph convolution layers and a fully connected layer that can accurately segment the liver and liver tumors. Thus, the accuracy, Dice coefficient, mean IoU, sensitivity, precision, and recall obtained based on the proposed method according to the LiTS17 dataset are around 99.1%, 91.1%, 90.8%, 99.4%, 99.4%, and 91.2%, respectively. In addition, the effectiveness of the proposed method was evaluated in a noisy environment, and the proposed network could withstand a wide range of environmental signal-to-noise ratios (SNRs). Thus, at SNR = -4 dB, the accuracy of the proposed method for liver organ segmentation remained around 90%. The proposed model has obtained satisfactory and favorable results compared to previous research. According to the positive results, the proposed model is expected to be used to assist radiologists and specialist doctors in the near future.

Radiomics-Based Prediction of Future Portal Vein Tumor Infiltration in Patients with HCC-A Proof-of-Concept Study

Portal vein infiltration (PVI) is a typical complication of HCC. Once diagnosed, it leads to classification as BCLC C with an enormous impact on patient management, as systemic therapies are henceforth recommended. Our aim was to investigate whether radiomics analysis using imaging at initial diagnosis can predict the occurrence of PVI in the course of disease. Between 2008 and 2018, we retrospectively identified 44 patients with HCC and an in-house, multiphase CT scan at initial diagnosis who presented without CT-detectable PVI but developed it in the course of disease. Accounting for size and number of lesions, growth type, arterial enhancement pattern, Child-Pugh stage, AFP levels, and subsequent therapy, we matched 44 patients with HCC who did not develop PVI to those developing PVI in the course of disease (follow-up ended December 2021). After segmentation of the tumor at initial diagnosis and texture analysis, we used LASSO regression to find radiomics features suitable for PVI detection in this matched set. Using an 80:20 split between training and holdout validation dataset, 17 radiomics features remained in the fitted model. Applying the model to the holdout validation dataset, sensitivity to detect occurrence of PVI was 0.78 and specificity was 0.78. Radiomics feature extraction had the ability to detect aggressive HCC morphology likely to result in future PVI. An additional radiomics evaluation at initial diagnosis might be a useful tool to identify patients with HCC at risk for PVI during follow-up benefiting from a closer surveillance.