Scale-adaptive supervoxel-based random forests for liver tumor segmentation in dynamic contrast-enhanced CT scans

CFANet: Context fusing attentional network for preoperative CT image segmentation in robotic surgery

Accurate segmentation of CT images is crucial for clinical diagnosis and preoperative evaluation of robotic surgery, but challenges arise from fuzzy boundaries and small-sized targets. In response, a novel 2D segmentation network named Context Fusing Attentional Network (CFANet) is proposed. CFANet incorporates three key modules to address these challenges, namely pyramid fusing module (PFM), parallel dilated convolution module (PDCM) and scale attention module (SAM). Integration of these modules into the encoder-decoder structure enables effective utilization of multi-level and multi-scale features. Compared with advanced segmentation method, the Dice score improved by 2.14% on the dataset of liver tumor. This improvement is expected to have a positive impact on the preoperative evaluation of robotic surgery and to support clinical diagnosis, especially in early tumor detection.

Performance and clinical applicability of machine learning in liver computed tomography imaging: a systematic review

OBJECTIVES

Machine learning (ML) for medical imaging is emerging for several organs and image modalities. Our objectives were to provide clinicians with an overview of this field by answering the following questions: (1) How is ML applied in liver computed tomography (CT) imaging? (2) How well do ML systems perform in liver CT imaging? (3) What are the clinical applications of ML in liver CT imaging?

METHODS

A systematic review was carried out according to the guidelines from the PRISMA-P statement. The search string focused on studies containing content relating to artificial intelligence, liver, and computed tomography.

RESULTS

One hundred ninety-one studies were included in the study. ML was applied to CT liver imaging by image analysis without clinicians' intervention in majority of studies while in newer studies the fusion of ML method with clinical intervention have been identified. Several were documented to perform very accurately on reliable but small data. Most models identified were deep learning-based, mainly using convolutional neural networks. Potentially many clinical applications of ML to CT liver imaging have been identified through our review including liver and its lesion segmentation and classification, segmentation of vascular structure inside the liver, fibrosis and cirrhosis staging, metastasis prediction, and evaluation of chemotherapy.

CONCLUSION

Several studies attempted to provide transparent result of the model. To make the model convenient for a clinical application, prospective clinical validation studies are in urgent call. Computer scientists and engineers should seek to cooperate with health professionals to ensure this.

KEY POINTS

• ML shows great potential for CT liver image tasks such as pixel-wise segmentation and classification of liver and liver lesions, fibrosis staging, metastasis prediction, and retrieval of relevant liver lesions from similar cases of other patients. • Despite presenting the result is not standardized, many studies have attempted to provide transparent results to interpret the machine learning method performance in the literature. • Prospective studies are in urgent call for clinical validation of ML method, preferably carried out by cooperation between clinicians and computer scientists.

The Liver Tumor Segmentation Benchmark (LiTS)

In this work, we report the set-up and results of the Liver Tumor Segmentation Benchmark (LiTS), which was organized in conjunction with the IEEE International Symposium on Biomedical Imaging (ISBI) 2017 and the International Conferences on Medical Image Computing and Computer-Assisted Intervention (MICCAI) 2017 and 2018. The image dataset is diverse and contains primary and secondary tumors with varied sizes and appearances with various lesion-to-background levels (hyper-/hypo-dense), created in collaboration with seven hospitals and research institutions. Seventy-five submitted liver and liver tumor segmentation algorithms were trained on a set of 131 computed tomography (CT) volumes and were tested on 70 unseen test images acquired from different patients. We found that not a single algorithm performed best for both liver and liver tumors in the three events. The best liver segmentation algorithm achieved a Dice score of 0.963, whereas, for tumor segmentation, the best algorithms achieved Dices scores of 0.674 (ISBI 2017), 0.702 (MICCAI 2017), and 0.739 (MICCAI 2018). Retrospectively, we performed additional analysis on liver tumor detection and revealed that not all top-performing segmentation algorithms worked well for tumor detection. The best liver tumor detection method achieved a lesion-wise recall of 0.458 (ISBI 2017), 0.515 (MICCAI 2017), and 0.554 (MICCAI 2018), indicating the need for further research. LiTS remains an active benchmark and resource for research, e.g., contributing the liver-related segmentation tasks in http://medicaldecathlon.com/. In addition, both data and online evaluation are accessible via https://competitions.codalab.org/competitions/17094.

Automatic Liver Tumor Segmentation on Dynamic Contrast Enhanced MRI Using 4D Information: Deep Learning Model Based on 3D Convolution and Convolutional LSTM

OBJECTIVE

Accurate segmentation of liver tumors, which could help physicians make appropriate treatment decisions and assess the effectiveness of surgical treatment, is crucial for the clinical diagnosis of liver cancer. In this study, we propose a 4-dimensional (4D) deep learning model based on 3D convolution and convolutional long short-term memory (C-LSTM) for hepatocellular carcinoma (HCC) lesion segmentation.

METHODS

The proposed deep learning model utilizes 4D information on dynamic contrast enhanced (DCE) magnetic resonance imaging (MRI) images to assist liver tumor segmentation. Specifically, a shallow U-net based 3D CNN module was designed to extract 3D spatial domain features from each DCE phase, followed by a 4-layer C-LSTM network module for time domain information exploitation. The combined information of multi-phase DCE images and the manner by which tissue imaging features change on multi-contrast images allow the network to more effectively learn the characteristics of HCC, resulting in better segmentation performance.

RESULTS

The proposed model achieved a Dice score of 0.825± 0.077, a Hausdorff distance of 12.84± 8.14 mm, and a volume similarity of 0.891± 0.080 for liver tumor segmentation, which outperformed the 3D U-net model, RA-UNet model and other models in the ablation study in both internal and external test sets. Moreover, the performance of the proposed model is comparable to the nnU-Net model, which showed state-of-the-art performance in many segmentation tasks, with significantly reduced prediction time.

CONCLUSION

The proposed 3D convolution and C-LSTM based model can achieve accurate segmentation of HCC lesions.

Adaptive Attention Convolutional Neural Network for Liver Tumor Segmentation

Purpose

Accurate segmentation of liver and liver tumors is critical for radiotherapy. Liver tumor segmentation, however, remains a difficult and relevant problem in the field of medical image processing because of the various factors like complex and variable location, size, and shape of liver tumors, low contrast between tumors and normal tissues, and blurred or difficult-to-define lesion boundaries. In this paper, we proposed a neural network (S-Net) that can incorporate attention mechanisms to end-to-end segmentation of liver tumors from CT images.

Methods

First, this study adopted a classical coding-decoding structure to realize end-to-end segmentation. Next, we introduced an attention mechanism between the contraction path and the expansion path so that the network could encode a longer range of semantic information in the local features and find the corresponding relationship between different channels. Then, we introduced long-hop connections between the layers of the contraction path and the expansion path, so that the semantic information extracted in both paths could be fused. Finally, the application of closed operation was used to dissipate the narrow interruptions and long, thin divide. This eliminated small cavities and produced a noise reduction effect.

Results

In this paper, we used the MICCAI 2017 liver tumor segmentation (LiTS) challenge dataset, 3DIRCADb dataset and doctors' manual contours of Hubei Cancer Hospital dataset to test the network architecture. We calculated the Dice Global (DG) score, Dice per Case (DC) score, volumetric overlap error (VOE), average symmetric surface distance (ASSD), and root mean square error (RMSE) to evaluate the accuracy of the architecture for liver tumor segmentation. The segmentation DG for tumor was found to be 0.7555, DC was 0.613, VOE was 0.413, ASSD was 1.186 and RMSE was 1.804. For a small tumor, DG was 0.3246 and DC was 0.3082. For a large tumor, DG was 0.7819 and DC was 0.7632.

Conclusion

S-Net obtained more semantic information with the introduction of an attention mechanism and long jump connection. Experimental results showed that this method effectively improved the effect of tumor recognition in CT images and could be applied to assist doctors in clinical treatment.

State of the Art in Artificial Intelligence and Radiomics in Hepatocellular Carcinoma

The most common liver malignancy is hepatocellular carcinoma (HCC), which is also associated with high mortality. Often HCC develops in a chronic liver disease setting, and early diagnosis as well as accurate screening of high-risk patients is crucial for appropriate and effective management of these patients. While imaging characteristics of HCC are well-defined in the diagnostic phase, challenging cases still occur, and current prognostic and predictive models are limited in their accuracy. Radiomics and machine learning (ML) offer new tools to address these issues and may lead to scientific breakthroughs with the potential to impact clinical practice and improve patient outcomes. In this review, we will present an overview of these technologies in the setting of HCC imaging across different modalities and a range of applications. These include lesion segmentation, diagnosis, prognostic modeling and prediction of treatment response. Finally, limitations preventing clinical application of radiomics and ML at the present time are discussed, together with necessary future developments to bring the field forward and outside of a purely academic endeavor.

PA-ResSeg: A phase attention residual network for liver tumor segmentation from multiphase CT images

PURPOSE

Liver tumor segmentation is a crucial prerequisite for computer-aided diagnosis of liver tumors. In the clinical diagnosis of liver tumors, radiologists usually examine multiphase CT images as these images provide abundant and complementary information of tumors. However, most known automatic segmentation methods extract tumor features from CT images merely of a single phase, in which valuable multiphase information is ignored. Therefore, it is highly demanded to develop a method effectively incorporating multiphase information for automatic and accurate liver tumor segmentation.

METHODS

In this paper, we propose a phase attention residual network (PA-ResSeg) to model multiphase features for accurate liver tumor segmentation. A phase attention (PA) is newly proposed to additionally exploit the images of arterial (ART) phase to facilitate the segmentation of portal venous (PV) phase. The PA block consists of an intraphase attention (intra-PA) module and an interphase attention (inter-PA) module to capture channel-wise self-dependencies and cross-phase interdependencies, respectively. Thus, it enables the network to learn more representative multiphase features by refining the PV features according to the channel dependencies and recalibrating the ART features based on the learned interdependencies between phases. We propose a PA-based multiscale fusion (MSF) architecture to embed the PA blocks in the network at multiple levels along the encoding path to fuse multiscale features from multiphase images. Moreover, a 3D boundary-enhanced loss (BE-loss) is proposed for training to make the network more sensitive to boundaries.

RESULTS

To evaluate the performance of our proposed PA-ResSeg, we conducted experiments on a multiphase CT dataset of focal liver lesions (MPCT-FLLs). Experimental results show the effectiveness of the proposed method by achieving a dice per case (DPC) of 0.7787, a dice global (DG) of 0.8682, a volumetric overlap error (VOE) of 0.3328, and a relative volume difference (RVD) of 0.0443 on the MPCT-FLLs. Furthermore, to validate the effectiveness and robustness of PA-ResSeg, we conducted extra experiments on another multiphase liver tumor dataset and obtained a DPC of 0.8290, a DG of 0.9132, a VOE of 0.2637, and a RVD of 0.0163. The proposed method shows its robustness and generalization capability in different datasets and different backbones.

CONCLUSIONS

The study demonstrates that our method can effectively model information from multiphase CT images to segment liver tumors and outperforms other state-of-the-art methods. The PA-based MSF method can learn more representative multiphase features at multiple scales and thereby improve the segmentation performance. Besides, the proposed 3D BE-loss is conducive to tumor boundary segmentation by enforcing the network focus on boundary regions and marginal slices. Experimental results evaluated by quantitative metrics demonstrate the superiority of our PA-ResSeg over the best-known methods.

Abdominal multi-organ segmentation with cascaded convolutional and adversarial deep networks

Abdominal anatomy segmentation is crucial for numerous applications from computer-assisted diagnosis to image-guided surgery. In this context, we address fully-automated multi-organ segmentation from abdominal CT and MR images using deep learning. The proposed model extends standard conditional generative adversarial networks. Additionally to the discriminator which enforces the model to create realistic organ delineations, it embeds cascaded partially pre-trained convolutional encoder-decoders as generator. Encoder fine-tuning from a large amount of non-medical images alleviates data scarcity limitations. The network is trained end-to-end to benefit from simultaneous multi-level segmentation refinements using auto-context. Employed for healthy liver, kidneys and spleen segmentation, our pipeline provides promising results by outperforming state-of-the-art encoder-decoder schemes. Followed for the Combined Healthy Abdominal Organ Segmentation (CHAOS) challenge organized in conjunction with the IEEE International Symposium on Biomedical Imaging 2019, it gave us the first rank for three competition categories: liver CT, liver MR and multi-organ MR segmentation. Combining cascaded convolutional and adversarial networks strengthens the ability of deep learning pipelines to automatically delineate multiple abdominal organs, with good generalization capability. The comprehensive evaluation provided suggests that better guidance could be achieved to help clinicians in abdominal image interpretation and clinical decision making.

Regional Gas Transport During Conventional and Oscillatory Ventilation Assessed by Xenon-Enhanced Computed Tomography

Enhanced intrapulmonary gas transport enables oscillatory ventilation modalities to support gas exchange using extremely low tidal volumes at high frequencies. However, it is unknown whether gas transport rates can be improved by combining multiple frequencies of oscillation simultaneously. The goal of this study was to investigate distributed gas transport in vivo during multi-frequency oscillatory ventilation (MFOV) as compared with conventional mechanical ventilation (CMV) or high-frequency oscillatory ventilation (HFOV). We hypothesized that MFOV would result in more uniform rates of gas transport compared to HFOV, measured using contrast-enhanced CT imaging during wash-in of xenon gas. In 13 pigs, xenon wash-in equilibration rates were comparable between CMV and MFOV, but 21 to 39% slower for HFOV. By contrast, the root-mean-square delivered volume was lowest for MFOV, increased by 70% during HFOV and 365% during CMV. Overall gas transport heterogeneity was similar across all modalities, but gravitational gradients and regional patchiness of specific ventilation contributed to regional ventilation heterogeneity, depending on ventilator modality. We conclude that MFOV combines benefits of low lung stretch, similar to HFOV, but with fast rates of gas transport, similar to CMV.

Feasibility of automatic detection of small hepatocellular carcinoma (≤2 cm) in cirrhotic liver based on pattern matching and deep learning

Background and objective.Early detection of hepatocellular carcinoma (HCC) is crucial for clinical management. Current studies have reported large HCC detections using automatic algorithms, but there is a lack of research on automatic detection of small HCCs (sHCCs). This study is to investigate the feasibility of automatic detection of sHCC (≤2 cm) based on pattern matching and deep learning (PM-DL) model.Materials and methods. A retrospective study included 5376 image sets from 56 cirrhosis patients (28 sHCC patients with 32 pathologically confirmed lesions and 28 non-HCC cirrhosis patients) in the training-validation cohort to build and validate the model through five-fold cross-validation. In addition, an external test cohort including 6144 image sets from 64 cirrhosis patients (32 sHCC patients with 38 lesions and 32 non-HCC cirrhosis patients) was applied to further verify the generalization ability of the model. The proposed PM-DL model consisted of three main steps: 3D co-registration and liver segmentation, screening of suspicious lesions on diffusion-weighted imaging images based on pattern matching algorithm, and identification/segmentation of sHCC lesions on dynamic contrast-enhanced images with convolutional neural network.Results.The PM-DL model achieved a sensitivity of 89.74% and a positive predictive value of 85.00% in the external test cohort for per-lesion analysis. No significant difference was observed in volumes (P= 0.13) and the largest sizes (P= 0.89) between manually delineated and segmented lesions. The DICE coefficient reached 0.77 ± 0.16. Similar performances were identified in the validation cohort. Moreover, the PM-DL model outperformed Liver Imaging Reporting and Data System (LI-RADS) in sensitivity (probable HCCs: LR-5 or LR-4,P= 0.18; definite HCCs: LR-5,P< 0.001), with a similar high specificity for per-patient analysis.Conclusion. The PM-DL model may be feasible for accurate automatic detection of sHCC in cirrhotic liver.

Spatial feature fusion convolutional network for liver and liver tumor segmentation from CT images

PURPOSE

The accurate segmentation of liver and liver tumors from CT images can assist radiologists in decision-making and treatment planning. The contours of liver and liver tumors are currently obtained by manual labeling, which is time-consuming and subjective. Computer-aided segmentation methods have been widely used in the segmentation of liver and liver tumors. However, due to the diversity of shape, volume, and image intensity, the segmentation is still a difficult task. In this study, we present a Spatial Feature Fusion Convolutional Network (SFF-Net) to automatically segment liver and liver tumors from CT images.

METHODS

First, we extract side-outputs at each convolutional block in SFF-Net to make full use of multiscale features. Second, skip-connections are added in the down-sampling phase, therefore, the spatial information can be efficiently transferred to later layers. Third, we present feature fusion blocks (FFBs) to merge spatial features and high-level semantic features from early layers and later layers, respectively. Finally, a fully connected 3D conditional random fields (CRFs) is applied to refine the liver and liver tumor segmentation results.

RESULTS

We test our method on the MICCAI 2017 Liver Tumor Segmentation (LiTS) challenge dataset. The Dice Global (DG) score, Dice per case (DC) score, Volume Overlap Error (VOE), Average Symmetric Surface Distance (ASSD), and tumor precision score are calculated to evaluate the liver and liver tumor segmentation accuracies. For the liver segmentation, DG is 0.955; DC is 0.937; VOE is 0.106; and ASSD is 3.678. For the tumor segmentation, DG is 0.746; DC is 0.592; VOE is 0.416; ASSD is 1.585 and the tumor precision score is 0.369.

CONCLUSIONS

The SFF-Net learns more spatial information by adding skip-connections and feature fusion blocks. The experiments validate that our method can accurately segment liver and liver tumors from CT images.

Quantifying Regional Lung Deformation Using Four-Dimensional Computed Tomography: A Comparison of Conventional and Oscillatory Ventilation

Mechanical ventilation strategies that reduce the heterogeneity of regional lung stress and strain may reduce the risk of ventilator-induced lung injury (VILI). In this study, we used registration of four-dimensional computed tomographic (4DCT) images to assess regional lung aeration and deformation in 10 pigs under baseline conditions and following acute lung injury induced with oleic acid. CT images were obtained via dynamic axial imaging (Siemens SOMATOM Force) during conventional pressure-controlled mechanical ventilation (CMV), as well as high-frequency and multi-frequency oscillatory ventilation modalities (HFOV and MFOV, respectively). Our results demonstrate that oscillatory modalities reduce intratidal strain throughout the lung in comparison to conventional ventilation, as well as the spatial gradients of dynamic strain along the dorsal-ventral axis. Harmonic distortion of parenchymal deformation was observed during HFOV with a single discrete sinusoid delivered at the airway opening, suggesting inherent mechanical nonlinearity of the lung tissues. MFOV may therefore provide improved lung-protective ventilation by reducing strain magnitudes and spatial gradients of strain compared to either CMV or HFOV.

Liver tissue segmentation in multiphase CT scans using cascaded convolutional neural networks

PURPOSE

We address the automatic segmentation of healthy and cancerous liver tissues (parenchyma, active and necrotic parts of hepatocellular carcinoma (HCC) tumor) on multiphase CT images using a deep learning approach.

METHODS

We devise a cascaded convolutional neural network based on the U-Net architecture. Two strategies for dealing with multiphase information are compared: Single-phase images are concatenated in a multi-dimensional features map on the input layer, or output maps are computed independently for each phase before being merged to produce the final segmentation. Each network of the cascade is specialized in the segmentation of a specific tissue. The performances of these networks taken separately and of the cascaded architecture are assessed on both single-phase and on multiphase images.

RESULTS

In terms of Dice coefficients, the proposed method is on par with a state-of-the-art method designed for automatic MR image segmentation and outperforms previously used technique for interactive CT image segmentation. We validate the hypothesis that several cascaded specialized networks have a higher prediction accuracy than a single network addressing all tasks simultaneously. Although the portal venous phase alone seems to provide sufficient contrast for discriminating tumors from healthy parenchyma, the multiphase information brings significant improvement for the segmentation of cancerous tissues (active versus necrotic part).

CONCLUSION

The proposed cascaded multiphase architecture showed promising performances for the automatic segmentation of liver tissues, allowing to reliably estimate the necrosis rate, a valuable imaging biomarker of the clinical outcome.

H-DenseUNet: Hybrid Densely Connected UNet for Liver and Tumor Segmentation From CT Volumes

Liver cancer is one of the leading causes of cancer death. To assist doctors in hepatocellular carcinoma diagnosis and treatment planning, an accurate and automatic liver and tumor segmentation method is highly demanded in the clinical practice. Recently, fully convolutional neural networks (FCNs), including 2-D and 3-D FCNs, serve as the backbone in many volumetric image segmentation. However, 2-D convolutions cannot fully leverage the spatial information along the third dimension while 3-D convolutions suffer from high computational cost and GPU memory consumption. To address these issues, we propose a novel hybrid densely connected UNet (H-DenseUNet), which consists of a 2-D DenseUNet for efficiently extracting intra-slice features and a 3-D counterpart for hierarchically aggregating volumetric contexts under the spirit of the auto-context algorithm for liver and tumor segmentation. We formulate the learning process of the H-DenseUNet in an end-to-end manner, where the intra-slice representations and inter-slice features can be jointly optimized through a hybrid feature fusion layer. We extensively evaluated our method on the data set of the MICCAI 2017 Liver Tumor Segmentation Challenge and 3DIRCADb data set. Our method outperformed other state-of-the-arts on the segmentation results of tumors and achieved very competitive performance for liver segmentation even with a single model.