A workflow for automated segmentation of the liver surface, hepatic vasculature and biliary tree anatomy from multiphase MR images

Artificial intelligence techniques in liver cancer

Hepatocellular Carcinoma (HCC), the most common primary liver cancer, is a significant contributor to worldwide cancer-related deaths. Various medical imaging techniques, including computed tomography, magnetic resonance imaging, and ultrasound, play a crucial role in accurately evaluating HCC and formulating effective treatment plans. Artificial Intelligence (AI) technologies have demonstrated potential in supporting physicians by providing more accurate and consistent medical diagnoses. Recent advancements have led to the development of AI-based multi-modal prediction systems. These systems integrate medical imaging with other modalities, such as electronic health record reports and clinical parameters, to enhance the accuracy of predicting biological characteristics and prognosis, including those associated with HCC. These multi-modal prediction systems pave the way for predicting the response to transarterial chemoembolization and microvascular invasion treatments and can assist clinicians in identifying the optimal patients with HCC who could benefit from interventional therapy. This paper provides an overview of the latest AI-based medical imaging models developed for diagnosing and predicting HCC. It also explores the challenges and potential future directions related to the clinical application of AI techniques.

Vascular liver segmentation: a narrative review on methods and new insights brought by artificial intelligence

Liver vessel segmentation from routinely performed medical imaging is a useful tool for diagnosis, treatment planning and delivery, and prognosis evaluation for many diseases, particularly liver cancer. A precise representation of liver anatomy is crucial to define the extent of the disease and, when suitable, the consequent resective or ablative procedure, in order to guarantee a radical treatment without sacrificing an excessive volume of healthy liver. Once mainly performed manually, with notable cost in terms of time and human energies, vessel segmentation is currently realized through the application of artificial intelligence (AI), which has gained increased interest and development of the field. Many different AI-driven models adopted for this aim have been described and can be grouped into different categories: thresholding methods, edge- and region-based methods, model-based methods, and machine learning models. The latter includes neural network and deep learning models that now represent the principal algorithms exploited for vessel segmentation. The present narrative review describes how liver vessel segmentation can be realized through AI models, with a summary of model results in terms of accuracy, and an overview on the future progress of this topic.

Automated MRI liver segmentation for anatomical segmentation, liver volumetry, and the extraction of radiomics

OBJECTIVES

To develop and evaluate a deep convolutional neural network (DCNN) for automated liver segmentation, volumetry, and radiomic feature extraction on contrast-enhanced portal venous phase magnetic resonance imaging (MRI).

MATERIALS AND METHODS

This retrospective study included hepatocellular carcinoma patients from an institutional database with portal venous MRI. After manual segmentation, the data was randomly split into independent training, validation, and internal testing sets. From a collaborating institution, de-identified scans were used for external testing. The public LiverHccSeg dataset was used for further external validation. A 3D DCNN was trained to automatically segment the liver. Segmentation accuracy was quantified by the Dice similarity coefficient (DSC) with respect to manual segmentation. A Mann-Whitney U test was used to compare the internal and external test sets. Agreement of volumetry and radiomic features was assessed using the intraclass correlation coefficient (ICC).

RESULTS

In total, 470 patients met the inclusion criteria (63.9±8.2 years; 376 males) and 20 patients were used for external validation (41±12 years; 13 males). DSC segmentation accuracy of the DCNN was similarly high between the internal (0.97±0.01) and external (0.96±0.03) test sets (p=0.28) and demonstrated robust segmentation performance on public testing (0.93±0.03). Agreement of liver volumetry was satisfactory in the internal (ICC, 0.99), external (ICC, 0.97), and public (ICC, 0.85) test sets. Radiomic features demonstrated excellent agreement in the internal (mean ICC, 0.98±0.04), external (mean ICC, 0.94±0.10), and public (mean ICC, 0.91±0.09) datasets.

CONCLUSION

Automated liver segmentation yields robust and generalizable segmentation performance on MRI data and can be used for volumetry and radiomic feature extraction.

CLINICAL RELEVANCE STATEMENT

Liver volumetry, anatomic localization, and extraction of quantitative imaging biomarkers require accurate segmentation, but manual segmentation is time-consuming. A deep convolutional neural network demonstrates fast and accurate segmentation performance on T1-weighted portal venous MRI.

KEY POINTS

• This deep convolutional neural network yields robust and generalizable liver segmentation performance on internal, external, and public testing data. • Automated liver volumetry demonstrated excellent agreement with manual volumetry. • Automated liver segmentations can be used for robust and reproducible radiomic feature extraction.

Signal Intensity Trajectories Clustering for Liver Vasculature Segmentation and Labeling (LiVaS) on Contrast-Enhanced MR Images: A Feasibility Pilot Study

This study aims to develop a semiautomated pipeline and user interface (LiVaS) for rapid segmentation and labeling of MRI liver vasculature and evaluate its time efficiency and accuracy against manual reference standard. Retrospective feasibility pilot study. Liver MR images from different scanners from 36 patients were included, and 4/36 patients were randomly selected for manual segmentation as referenced standard. The liver was segmented in each contrast phase and masks registered to the pre-contrast segmentation. Voxel-wise signal trajectories were clustered using the k-means algorithm. Voxel clusters that best segment the liver vessels were selected and labeled by three independent radiologists and a research scientist using LiVaS. Segmentation times were compared using a paired-sample t-test on log-transformed data. The agreement was analyzed qualitatively and quantitatively using DSC for hepatic and portal vein segmentations. The mean segmentation time among four readers was significantly shorter than manual (3.6 ± 1.4 vs. 70.0 ± 29.2 min; p < 0.001), even when using a higher number of clusters to enhance accuracy. The DSC for portal and hepatic veins reached up to 0.69 and 0.70, respectively. LiVaS segmentations were overall of good quality, with variations in performance related to the presence/severity of liver disease, acquisition timing, and image quality. Our semi-automated pipeline was robust to different MRI vendors in producing segmentation and labeling of liver vasculature in agreement with expert manual annotations, with significantly higher time efficiency. LiVaS could facilitate the creation of large, annotated datasets for training and validation of neural networks for automated MRI liver vascularity segmentation. HIGHLIGHTS: Key Finding: In this pilot feasibility study, our semiautomated pipeline for segmentation of liver vascularity (LiVaS) on MR images produced segmentations with simultaneous labeling of portal and hepatic veins in good agreement with the manual reference standard but at significantly shorter times (mean LiVaS 3.6 ± 1.4 vs. mean manual 70.0 ± 29.2 min; p < 0.001). Importance: LiVaS was robust in producing liver MRI vascular segmentations across images from different scanners in agreement with expert manual annotations, with significant ly higher time efficiency, and therefore potential scalability.

A Review of the Clinical Applications of Artificial Intelligence in Abdominal Imaging

Artificial intelligence (AI) has been a topic of substantial interest for radiologists in recent years. Although many of the first clinical applications were in the neuro, cardiothoracic, and breast imaging subspecialties, the number of investigated and real-world applications of body imaging has been increasing, with more than 30 FDA-approved algorithms now available for applications in the abdomen and pelvis. In this manuscript, we explore some of the fundamentals of artificial intelligence and machine learning, review major functions that AI algorithms may perform, introduce current and potential future applications of AI in abdominal imaging, provide a basic understanding of the pathways by which AI algorithms can receive FDA approval, and explore some of the challenges with the implementation of AI in clinical practice.

Convolutional neural network for automated segmentation of the liver and its vessels on non-contrast T1 vibe Dixon acquisitions

We evaluated the effectiveness of automated segmentation of the liver and its vessels with a convolutional neural network on non-contrast T1 vibe Dixon acquisitions. A dataset of non-contrast T1 vibe Dixon liver magnetic resonance images was labelled slice-by-slice for the outer liver border, portal, and hepatic veins by an expert. A 3D U-Net convolutional neural network was trained with different combinations of Dixon in-phase, opposed-phase, water, and fat reconstructions. The neural network trained with the single-modal in-phase reconstructions achieved a high performance for liver parenchyma (Dice 0.936 ± 0.02), portal veins (0.634 ± 0.09), and hepatic veins (0.532 ± 0.12) segmentation. No benefit of using multi-modal input was observed (p = 1.0 for all experiments), combining in-phase, opposed-phase, fat, and water reconstruction. Accuracy for differentiation between portal and hepatic veins was 99% for portal veins and 97% for hepatic veins in the central region and slightly lower in the peripheral region (91% for portal veins, 80% for hepatic veins). In conclusion, deep learning-based automated segmentation of the liver and its vessels on non-contrast T1 vibe Dixon was highly effective. The single-modal in-phase input achieved the best performance in segmentation and differentiation between portal and hepatic veins.

Ultrasound-based navigation for open liver surgery using active liver tracking

PURPOSE

Despite extensive preoperative imaging, intraoperative localization of liver lesions after systemic treatment can be challenging. Therefore, an image-guided navigation setup is explored that links preoperative diagnostic scans and 3D models to intraoperative ultrasound (US), enabling overlay of detailed diagnostic images on intraoperative US. Aim of this study is to assess the workflow and accuracy of such a navigation system which compensates for liver motion.

METHODS

Electromagnetic (EM) tracking was used for organ tracking and movement of the transducer. After laparotomy, a sensor was attached to the liver surface while the EM-tracked US transducer enabled image acquisition and landmark digitization. Landmarks surrounding the lesion were selected during patient-specific preoperative 3D planning and identified for registration during surgery. Endpoints were accuracy and additional times of the investigative steps. Accuracy was computed at the center of the target lesion.

RESULTS

In total, 22 navigated procedures were performed. Navigation provided useful visualization of preoperative 3D models and their overlay on US imaging. Landmark-based registration resulted in a mean fiducial registration error of 10.3 ± 4.3 mm, and a mean target registration error of 8.5 ± 4.2 mm. Navigation was available after an average of 12.7 min.

CONCLUSION

We developed a navigation method combining ultrasound with active liver tracking for organ motion compensation, with an accuracy below 10 mm. Fixation of the liver sensor near the target lesion compensates for local movement and contributes to improved reliability during navigation. This represents an important step forward in providing surgical navigation throughout the procedure.

TRIAL REGISTRATION

This study is registered in the Netherlands Trial Register (number NL7951).

Techniques and Algorithms for Hepatic Vessel Skeletonization in Medical Images: A Survey

Hepatic vessel skeletonization serves as an important means of hepatic vascular analysis and vessel segmentation. This paper presents a survey of techniques and algorithms for hepatic vessel skeletonization in medical images. We summarized the latest developments and classical approaches in this field. These methods are classified into five categories according to their methodological characteristics. The overview and brief assessment of each category are provided in the corresponding chapters, respectively. We provide a comprehensive summary among the cited publications, image modalities and datasets from various aspects, which hope to reveal the pros and cons of every method, summarize its achievements and discuss the challenges and future trends.

Portal Vein and Hepatic Vein Segmentation in Multi-Phase MR Images Using Flow-Guided Change Detection

Segmenting portal vein (PV) and hepatic vein (HV) from magnetic resonance imaging (MRI) scans is important for hepatic tumor surgery. Compared with single phase-based methods, multiple phases-based methods have better scalability in distinguishing HV and PV by exploiting multi-phase information. However, these methods just coarsely extract HV and PV from different phase images. In this paper, we propose a unified framework to automatically and robustly segment 3D HV and PV from multi-phase MR images, which considers both the change and appearance caused by the vascular flow event to improve segmentation performance. Firstly, inspired by change detection, flow-guided change detection (FGCD) is designed to detect the changed voxels related to hepatic venous flow by generating hepatic venous phase map and clustering the map. The FGCD uniformly deals with HV and PV clustering by the proposed shared clustering, thus making the appearance correlated with portal venous flow robustly delineate without increasing framework complexity. Then, to refine vascular segmentation results produced by both HV and PV clustering, interclass decision making (IDM) is proposed by combining the overlapping region discrimination and neighborhood direction consistency. Finally, our framework is evaluated on multi-phase clinical MR images of the public dataset (TCGA) and local hospital dataset. The quantitative and qualitative evaluations show that our framework outperforms the existing methods.

Emerging artificial intelligence applications in liver magnetic resonance imaging

Chronic liver diseases (CLDs) are becoming increasingly more prevalent in modern society. The use of imaging techniques for early detection, such as magnetic resonance imaging (MRI), is crucial in reducing the impact of these diseases on healthcare systems. Artificial intelligence (AI) algorithms have been shown over the past decade to excel at image-based analysis tasks such as detection and segmentation. When applied to liver MRI, they have the potential to improve clinical decision making, and increase throughput by automating analyses. With Liver diseases becoming more prevalent in society, the need to implement these techniques to utilize liver MRI to its full potential, is paramount. In this review, we report on the current methods and applications of AI methods in liver MRI, with a focus on machine learning and deep learning methods. We assess four main themes of segmentation, classification, image synthesis and artefact detection, and their respective potential in liver MRI and the wider clinic. We provide a brief explanation of some of the algorithms used and explore the current challenges affecting the field. Though there are many hurdles to overcome in implementing AI methods in the clinic, we conclude that AI methods have the potential to positively aid healthcare professionals for years to come.

Clinical Artificial Intelligence Applications in Radiology: Chest and Abdomen

Organ segmentation, chest radiograph classification, and lung and liver nodule detections are some of the popular artificial intelligence (AI) tasks in chest and abdominal radiology due to the wide availability of public datasets. AI algorithms have achieved performance comparable to humans in less time for several organ segmentation tasks, and some lesion detection and classification tasks. This article introduces the current published articles of AI applied to chest and abdominal radiology, including organ segmentation, lesion detection, classification, and predicting prognosis.

Living Donor-Recipient Pair Matching for Liver Transplant via Ternary Tree Representation With Cascade Incremental Learning

Visual understanding of liver vessels anatomy between the living donor-recipient (LDR) pair can assist surgeons to optimize transplant planning by avoiding non-targeted arteries which can cause severe complications. We propose to visually analyze the anatomical variants of the liver vessels anatomy to maximize similarity for finding a suitable Living Donor-Recipient (LDR) pair. Liver vessels are segmented from computed tomography angiography (CTA) volumes by employing a cascade incremental learning (CIL) model. Our CIL architecture is able to find optimal solutions, which we use to update the model with liver vessel CTA images. A novel ternary tree based algorithm is proposed to map all the possible liver vessel variants into their respective tree topologies. The tree topologies of the recipient's and donor's liver vessels are then used for an appropriate matching. The proposed algorithm utilizes a set of defined vessel tree variants which are updated to maintain the maximum matching options by leveraging the accurate segmentation results of the vessels derived from the incremental learning ability of the CIL. We introduce a novel concept of in-order digital string based comparison to match the geometry of two anatomically varied trees. Experiments through visual illustrations and quantitative analysis demonstrated the effectiveness of our approach compared to state-of-the-art.

Technical note: Validation of 3D ultrasound for image registration during oncological liver surgery

PURPOSE

Registration of pre- and intraoperative images is a crucial step of surgical liver navigation, where rigid registration of vessel centerlines is currently commonly used. When using 3D ultrasound (US), accuracy during navigation might be influenced by the size of the intraoperative US volume, yet the relationship between registration accuracy and US volume size is understudied. In this study, we specify an optimal 3D US volume size for registration using varying volumes of liver vasculature. While previous studies measured accuracy at registered fiducials, in this work, accuracy is determined at the target lesion which is clinically the most relevant structure.

METHODS

Three-dimensional US volumes were acquired in 14 patients after laparotomy and liver mobilization. Manual segmentation of vasculature and centerline extraction was performed. Intraoperative and preoperative vasculature centerlines were registered with coherent point drift, using different sub-volumes (sphere with radius r = 30, 40, …, 120 mm). Accuracy was measured by fiducial registration error (FRE) between vessel centerlines and target registration error (TRE) at the center of the target lesion.

RESULTS

The lowest FRE for vessel registration was reached with r = 50 mm (6.5 ± 2.5 mm), the highest with r = 120 mm (7.1 ± 2.1 mm). Clinical accuracy at the target lesion, resulted most accurate (TRE = 8.8 ± 5.0 mm) in sub-volumes with a radius of 50 mm. Smaller US sub-volumes resulted in lower average TREs when compared to larger US sub-volumes (Pearson's correlation coefficient R = 0.91, p < 0.001).

CONCLUSION

Our results indicate that there is a linear correlation between US volume size and registration accuracy at the tumor. Volumes with radii of 50 mm around the target lesion yield higher accuracy (p < 0.05) (Trial number IRBd18032, 11 September 2018).

CBCT-based navigation system for open liver surgery: Accurate guidance toward mobile and deformable targets with a semi-rigid organ approximation and electromagnetic tracking of the liver

Purpose

The surgical navigation system that provides guidance throughout the surgery can facilitate safer and more radical liver resections, but such a system should also be able to handle organ motion. This work investigates the accuracy of intraoperative surgical guidance during open liver resection, with a semi‐rigid organ approximation and electromagnetic tracking of the target area.

Methods

The suggested navigation technique incorporates a preoperative 3D liver model based on diagnostic 4D MRI scan, intraoperative contrast‐enhanced CBCT imaging and electromagnetic (EM) tracking of the liver surface, as well as surgical instruments, by means of six degrees‐of‐freedom micro‐EM sensors.

Results

The system was evaluated during surgeries with 35 patients and resulted in an accurate and intuitive real‐time visualization of liver anatomy and tumor's location, confirmed by intraoperative checks on visible anatomical landmarks. Based on accuracy measurements verified by intraoperative CBCT, the system’s average accuracy was 4.0 ± 3.0 mm, while the total surgical delay due to navigation stayed below 20 min.

Conclusions

The electromagnetic navigation system for open liver surgery developed in this work allows for accurate localization of liver lesions and critical anatomical structures surrounding the resection area, even when the liver was manipulated. However, further clinically integrating the method requires shortening the guidance‐related surgical delay, which can be achieved by shifting to faster intraoperative imaging like ultrasound. Our approach is adaptable to navigation on other mobile and deformable organs, and therefore may benefit various clinical applications.

Clinical Implementation of In-House Developed MR-Based Patient-Specific 3D Models of Liver Anatomy

Knowledge of patient-specific liver anatomy is key to patient safety during major hepatobiliary surgery. Three-dimensional (3D) models of patient-specific liver anatomy based on diagnostic MRI images can provide essential vascular and biliary anatomical insight during surgery. However, a method for generating these is not yet publicly available. This paper describes how these 3D models of the liver can be generated using open source software, and then subsequently integrated into a sterile surgical environment. The most common image quality aspects that degrade the quality of the 3D models as well possible ways of eliminating these are also discussed. Per patient, a single diagnostic multiphase MRI scan with hepatospecific contrast agent was used for automated segmentation of liver contour, arterial, portal, and venous anatomy, and the biliary tree. Subsequently, lesions were delineated manually. The resulting interactive 3D model could be accessed during surgery on a sterile covered tablet. Up to now, such models have been used in 335 surgical procedures. Their use simplified the surgical treatment of patients with a high number of liver metastases and contributed to the localization of vanished lesions in cases of a radiological complete response to neoadjuvant treatment. They facilitated perioperative verification of the relationship of tumors and the surrounding vascular and biliary anatomy, and eased decision-making before and during surgery.

Channel width optimized neural networks for liver and vessel segmentation in liver iron quantification

INTRODUCTION

MRI T2* relaxometry protocols are often used for Liver Iron Quantification in patients with hemochromatosis. Several methods exist to semi-automatically segment parenchyma and exclude vessels for this calculation.

PURPOSE

To determine if inclusion of multiple echoes inputs to Convolutional Neural Networks (CNN) improves automated liver and vessel segmentation in MRI T2* relaxometry protocols and to determine if the resultant segmentations agree with manual segmentations for liver iron quantification analysis.

METHODS

Multi echo Gradient Recalled Echo (GRE) MRI sequence for T2* relaxometry was performed for 79 exams on 31 patients with hemochromatosis for iron quantification analysis. 275 axial liver slices were manually segmented as ground truth masks. A batch normalized U-Net with variable input width to incorporate multiple echoes is used for segmentation, using DICE as the accuracy metric. ANOVA is used to evaluate significance of channel width changes in segmentation accuracy. Linear regression is used to model the relationship of channel width on segmentation accuracy. Liver segmentations are applied to relaxometry data to calculate liver T2* yielding liver iron concentration(LIC) derived from literature based calibration curves. Manual and CNN based LIC values are compared with Pearson correlation. Bland altman plots are used to visualize differences between manual and CNN based LIC values.

RESULTS

Performance metrics are tested on 55 hold out slices. Linear regression indicates that there is a monotonic increase of DICE with increasing channel depth (p = 0.001) with a slope of 3.61e-3. ANOVA indicates a significant increase segmentation accuracy over single channel starting at 3 channels. Incorporation of all channels results in an average DICE of 0.86, an average increase of 0.07 over single channel. The calculated LIC from CNN segmented livers agrees well with manual segmentation (R = 0.998, slope = 0.914, p«0.001), with an average absolute difference 0.27 ± 0.99 mg Fe/g or 1.34 ± 4.3%.

CONCLUSION

More input echoes yields higher model accuracy until the noise floor. Echos beyond the first three echo times in GRE based T2* relaxometry do not contribute significant information for segmentation of liver for LIC calculation. Deep learning models with three channel width allow for generalization of model to protocols of more than three echoes, effectively a universal requirement for relaxometry. Deep learning segmentations achieve a good accuracy compared with manual segmentations with minimal preprocessing. Liver iron values calculated from hand segmented liver and Neural network segmented liver were not statistically different from each other.