A Multimodality-Contribution-Aware TripNet for Histologic Grading of Hepatocellular Carcinoma

A flexible deep learning framework for liver tumor diagnosis using variable multi-phase contrast-enhanced CT scans

BACKGROUND

Liver cancer is a significant cause of cancer-related mortality worldwide and requires tailored treatment strategies for different types. However, preoperative accurate diagnosis of the type presents a challenge. This study aims to develop an automatic diagnostic model based on multi-phase contrast-enhanced CT (CECT) images to distinguish between hepatocellular carcinoma (HCC), intrahepatic cholangiocarcinoma (ICC), and normal individuals.

METHODS

We designed a Hierarchical Long Short-Term Memory (H-LSTM) model, whose core components consist of a shared image feature extractor across phases, an internal LSTM for each phase, and an external LSTM across phases. The internal LSTM aggregates features from different layers of 2D CECT images, while the external LSTM aggregates features across different phases. H-LSTM can handle incomplete phases and varying numbers of CECT image layers, making it suitable for real-world decision support scenarios. Additionally, we applied phase augmentation techniques to process multi-phase CECT images, improving the model's robustness.

RESULTS

The H-LSTM model achieved an overall average AUROC of 0.93 (0.90, 1.00) on the test dataset, with AUROC for HCC classification reaching 0.97 (0.93, 1.00) and for ICC classification reaching 0.90 (0.78, 1.00). Comprehensive validation in scenarios with incomplete phases was performed, with the H-LSTM model consistently achieving AUROC values over 0.9.

CONCLUSION

The proposed H-LSTM model can be employed for classification tasks involving incomplete phases of CECT images in real-world scenarios, demonstrating high performance. This highlights the potential of AI-assisted systems in achieving accurate diagnosis and treatment of liver cancer. H-LSTM offers an effective solution for processing multi-phase data and provides practical value for clinical diagnostics.

Artificial intelligence techniques in liver cancer

Hepatocellular Carcinoma (HCC), the most common primary liver cancer, is a significant contributor to worldwide cancer-related deaths. Various medical imaging techniques, including computed tomography, magnetic resonance imaging, and ultrasound, play a crucial role in accurately evaluating HCC and formulating effective treatment plans. Artificial Intelligence (AI) technologies have demonstrated potential in supporting physicians by providing more accurate and consistent medical diagnoses. Recent advancements have led to the development of AI-based multi-modal prediction systems. These systems integrate medical imaging with other modalities, such as electronic health record reports and clinical parameters, to enhance the accuracy of predicting biological characteristics and prognosis, including those associated with HCC. These multi-modal prediction systems pave the way for predicting the response to transarterial chemoembolization and microvascular invasion treatments and can assist clinicians in identifying the optimal patients with HCC who could benefit from interventional therapy. This paper provides an overview of the latest AI-based medical imaging models developed for diagnosing and predicting HCC. It also explores the challenges and potential future directions related to the clinical application of AI techniques.

MRI-Based Radiomics and Deep Learning in Biological Characteristics and Prognosis of Hepatocellular Carcinoma: Opportunities and Challenges

Hepatocellular carcinoma (HCC) is the fifth most common malignancy and the third leading cause of cancer-related death worldwide. HCC exhibits strong inter-tumor heterogeneity, with different biological characteristics closely associated with prognosis. In addition, patients with HCC often distribute at different stages and require diverse treatment options at each stage. Due to the variability in tumor sensitivity to different therapies, determining the optimal treatment approach can be challenging for clinicians prior to treatment. Artificial intelligence (AI) technology, including radiomics and deep learning approaches, has emerged as a unique opportunity to improve the spectrum of HCC clinical care by predicting biological characteristics and prognosis in the medical imaging field. The radiomics approach utilizes handcrafted features derived from specific mathematical formulas to construct various machine-learning models for medical applications. In terms of the deep learning approach, convolutional neural network models are developed to achieve high classification performance based on automatic feature extraction from images. Magnetic resonance imaging offers the advantage of superior tissue resolution and functional information. This comprehensive evaluation plays a vital role in the accurate assessment and effective treatment planning for HCC patients. Recent studies have applied radiomics and deep learning approaches to develop AI-enabled models to improve accuracy in predicting biological characteristics and prognosis, such as microvascular invasion and tumor recurrence. Although AI-enabled models have demonstrated promising potential in HCC with biological characteristics and prognosis prediction with high performance, one of the biggest challenges, interpretability, has hindered their implementation in clinical practice. In the future, continued research is needed to improve the interpretability of AI-enabled models, including aspects such as domain knowledge, novel algorithms, and multi-dimension data sources. Overcoming these challenges would allow AI-enabled models to significantly impact the care provided to HCC patients, ultimately leading to their deployment for clinical use. LEVEL OF EVIDENCE: 5 TECHNICAL EFFICACY: Stage 2.

Radiomics: A radiological evidence-based artificial intelligence technique to facilitate personalized precision medicine in hepatocellular carcinoma

The high postoperative recurrence rates in hepatocellular carcinoma (HCC) remain a major hurdle in its management. Appropriate staging and treatment selection may alleviate the extent of fatal recurrence. However, effective methods to preoperatively evaluate pathophysiologic and molecular characteristics of HCC are lacking. Imaging plays a central role in HCC diagnosis and stratification due to the non-invasive diagnostic criteria. Vast and crucial information is hidden within image data. Other than providing a morphological sketch for lesion diagnosis, imaging could provide new insights to describe the pathophysiological and genetic landscape of HCC. Radiomics aims to facilitate diagnosis and prognosis of HCC using artificial intelligence techniques to harness the immense information contained in medical images. Radiomics produces a set of archetypal and robust imaging features that are correlated to key pathological or molecular biomarkers to preoperatively risk-stratify HCC patients. Inferred with outcome data, comprehensive combination of radiomic, clinical and/or multi-omics data could also improve direct prediction of response to treatment and prognosis. The evolution of radiomics is changing our understanding of personalized precision medicine in HCC management. Herein, we review the key techniques and clinical applications in HCC radiomics and discuss current limitations and future opportunities to improve clinical decision making.

Can a proposed double branch multimodality-contribution-aware TripNet improve the prediction performance of the microvascular invasion of hepatocellular carcinoma based on small samples?

OBJECTIVES

To evaluate the potential improvement of prediction performance of a proposed double branch multimodality-contribution-aware TripNet (MCAT) in microvascular invasion (MVI) of hepatocellular carcinoma (HCC) based on a small sample.

METHODS

In this retrospective study, 121 HCCs from 103 consecutive patients were included, with 44 MVI positive and 77 MVI negative, respectively. A MCAT model aiming to improve the accuracy of deep neural network and alleviate the negative effect of small sample size was proposed and the improvement of MCAT model was verified among comparisons between MCAT and other used deep neural networks including 2DCNN (two-dimentional convolutional neural network), ResNet (residual neural network) and SENet (squeeze-and-excitation network), respectively.

RESULTS

Through validation, the AUC value of MCAT is significantly higher than 2DCNN based on CT, MRI, and both imaging (P < 0.001 for all). The AUC value of model with single branch pretraining based on small samples is significantly higher than model with end-to-end training in CT branch and double branch (0.62 vs 0.69, p=0.016, 0.65 vs 0.83, p=0.010, respectively). The AUC value of the double branch MCAT based on both CT and MRI imaging (0.83) was significantly higher than that of the CT branch MCAT (0.69) and MRI branch MCAT (0.73) (P < 0.001, P = 0.03, respectively), which was also significantly higher than common-used ReNet (0.67) and SENet (0.70) model (P < 0.001, P = 0.005, respectively).

CONCLUSION

A proposed Double branch MCAT model based on a small sample can improve the effectiveness in comparison to other deep neural networks or single branch MCAT model, providing a potential solution for scenarios such as small-sample deep learning and fusion of multiple imaging modalities.

Predicting the Initial Treatment Response to Transarterial Chemoembolization in Intermediate-Stage Hepatocellular Carcinoma by the Integration of Radiomics and Deep Learning

Objectives

We aimed to develop radiology-based models for the preoperative prediction of the initial treatment response to transarterial chemoembolization (TACE) in patients with hepatocellular carcinoma (HCC) since the integration of radiomics and deep learning (DL) has not been reported for TACE.

Methods

Three hundred and ten intermediate-stage HCC patients who underwent TACE were recruited from three independent medical centers. Based on computed tomography (CT) images, recursive feature elimination (RFE) was used to select the most useful radiomics features. Five radiomics conventional machine learning (cML) models and a DL model were used for training and validation. Mutual correlations between each model were analyzed. The accuracies of integrating clinical variables, cML, and DL models were then evaluated.

Results

Good predictive accuracies were showed across the two cohorts in the five cML models, especially the random forest algorithm (AUC = 0.967 and 0.964, respectively). DL showed high accuracies in the training and validation cohorts (AUC = 0.981 and 0.972, respectively). Significant mutual correlations were revealed between tumor size and the five cML models and DL model (each P < 0.001). The highest accuracies were achieved by integrating DL and the random forest algorithm in the training and validation cohorts (AUC = 0.995 and 0.994, respectively).

Conclusion

The radiomics cML models and DL model showed notable accuracy for predicting the initial response to TACE treatment. Moreover, the integrated model could serve as a novel and accurate method for prediction in intermediate-stage HCC.