Development of deep neural network for individualized hepatobiliary toxicity prediction after liver SBRT

Clinical outcomes and safety of external beam radiotherapy with extensive intrahepatic targets for advanced hepatocellular carcinoma: A single institutional clinical experience

BACKGROUND

This study observed the clinical outcome of radiotherapy to extensive intrahepatic targets for advanced hepatocellular carcinoma (HCC) in a single institution.

METHODS

From September 2009 to July 2021, patients who underwent fractionated radiotherapy to a planning target volume (PTV) of over 100 ml with biological effective dose >30 Gy10 for advanced HCC were enrolled. Overall survival (OS) and radiation-induced liver toxicity (RILD) were evaluated. RILD was defined as an increase in Child-Pugh (CP) score ≥2 or liver function tests ≥2.5 times at 3 months after the end of radiotherapy.

RESULTS

A total of 136 patients were evaluated. Eighty-nine patients had portal vein tumor thrombus (PVTT), 37 patients were in CP B stage, and the median radiation dose to PTV was 48.8 Gy10. The median OS was 12.3 months. The factors most affecting OS were PVTT (P = 0.001), PTV (>500 ml, P = 0.001), incomplete coverage of the intrahepatic tumor (P = 0.004), and CP B (P = 0.006) in Cox regression. RILD occurred in 22.4% of the patients and was affected by PVTT (P = 0.003), PTV (P = 0.010), pretreatment bilirubin levels (>1.5 mg/ml, P = 0.016), and the mean normal liver dose (MNLD) (≥ EQD2 18 Gy3, P = 0.021) in binary logistic regression. As the PTV was in excess of >500 ml, RILD developed in 30.2% of patients and the prognostic importance of pretreatment bilirubin levels (P = 0.006) and the MNLD (P = 0.014) increased.

CONCLUSIONS

As PTV is more extensive, the bilirubin level and the MNLD have to be taken into consideration for safe radiotherapy, in addition to the traditional prognostic factors.

Artificial intelligence in liver imaging: methods and applications

Liver disease is regarded as one of the major health threats to humans. Radiographic assessments hold promise in terms of addressing the current demands for precisely diagnosing and treating liver diseases, and artificial intelligence (AI), which excels at automatically making quantitative assessments of complex medical image characteristics, has made great strides regarding the qualitative interpretation of medical imaging by clinicians. Here, we review the current state of medical-imaging-based AI methodologies and their applications concerning the management of liver diseases. We summarize the representative AI methodologies in liver imaging with focusing on deep learning, and illustrate their promising clinical applications across the spectrum of precise liver disease detection, diagnosis and treatment. We also address the current challenges and future perspectives of AI in liver imaging, with an emphasis on feature interpretability, multimodal data integration and multicenter study. Taken together, it is revealed that AI methodologies, together with the large volume of available medical image data, might impact the future of liver disease care.

Application of AI on cholangiocarcinoma

Cholangiocarcinoma, classified as intrahepatic, perihilar, and extrahepatic, is considered a deadly malignancy of the hepatobiliary system. Most cases of cholangiocarcinoma are asymptomatic. Therefore, early detection of cholangiocarcinoma is significant but still challenging. The routine screening of a tumor lacks specificity and accuracy. With the application of AI, high-risk patients can be easily found by analyzing their clinical characteristics, serum biomarkers, and medical images. Moreover, AI can be used to predict the prognosis including recurrence risk and metastasis. Although they have some limitations, AI algorithms will still significantly improve many aspects of cholangiocarcinoma in the medical field with the development of computing power and technology.

Development and validation of survival prognostic models for head and neck cancer patients using machine learning and dosiomics and CT radiomics features: a multicentric study

BACKGROUND

This study aimed to investigate the value of clinical, radiomic features extracted from gross tumor volumes (GTVs) delineated on CT images, dose distributions (Dosiomics), and fusion of CT and dose distributions to predict outcomes in head and neck cancer (HNC) patients.

METHODS

A cohort of 240 HNC patients from five different centers was obtained from The Cancer Imaging Archive. Seven strategies, including four non-fusion (Clinical, CT, Dose, DualCT-Dose), and three fusion algorithms (latent low-rank representation referred (LLRR),Wavelet, weighted least square (WLS)) were applied. The fusion algorithms were used to fuse the pre-treatment CT images and 3-dimensional dose maps. Overall, 215 radiomics and Dosiomics features were extracted from the GTVs, alongside with seven clinical features incorporated. Five feature selection (FS) methods in combination with six machine learning (ML) models were implemented. The performance of the models was quantified using the concordance index (CI) in one-center-leave-out 5-fold cross-validation for overall survival (OS) prediction considering the time-to-event.

RESULTS

The mean CI and Kaplan-Meier curves were used for further comparisons. The CoxBoost ML model using the Minimal Depth (MD) FS method and the glmnet model using the Variable hunting (VH) FS method showed the best performance with CI = 0.73 ± 0.15 for features extracted from LLRR fused images. In addition, both glmnet-Cindex and Coxph-Cindex classifiers achieved a CI of 0.72 ± 0.14 by employing the dose images (+ incorporated clinical features) only.

CONCLUSION

Our results demonstrated that clinical features, Dosiomics and fusion of dose and CT images by specific ML-FS models could predict the overall survival of HNC patients with acceptable accuracy. Besides, the performance of ML methods among the three different strategies was almost comparable.

Enhanced prediction of postoperative radiotherapy-induced esophagitis in non-small cell lung cancer: Dosiomic model development in a real-world cohort and validation in the PORT-C randomized controlled trial

BACKGROUND

Radiotherapy-induced esophagitis (RE) diminishes the quality of life and interrupts treatment in patients with non-small cell lung cancer (NSCLC) undergoing postoperative radiotherapy. Dosimetric models showed limited capability in predicting RE. We aimed to develop dosiomic models to predict RE.

METHODS

Models were trained with a real-world cohort and validated with PORT-C randomized controlled trial cohort. Patients with NSCLC undergoing resection followed by postoperative radiotherapy between 2004 and 2015 were enrolled. The endpoint was grade ≥2 RE. Esophageal three-dimensional dose distribution features were extracted using handcrafted and convolutional neural network (CNN) methods, screened using an entropy-based method, and selected using minimum redundancy and maximum relevance. Prediction models were built using logistic regression. The areas under the receiver operating characteristic curve (AUC) and precision-recall curve were used to evaluate prediction model performance. A dosimetric model was built for comparison.

RESULTS

A total of 190 and 103 patients were enrolled in the training and validation sets, respectively. Using handcrafted and CNN methods, 107 and 4096 features were derived, respectively. Three handcrafted, four CNN-extracted and three dosimetric features were selected. AUCs of training and validation sets were 0.737 and 0.655 for the dosimetric features, 0.730 and 0.724 for handcrafted features, and 0.812 and 0.785 for CNN-extracted features, respectively. Precision-recall curves revealed that CNN-extracted features outperformed dosimetric and handcrafted features.

CONCLUSIONS

Prediction models may identify patients at high risk of developing RE. Dosiomic models outperformed the dosimetric-feature model in predicting RE. CNN-extracted features were more predictive but less interpretable than handcrafted features.

Comparing Predicted Toxicities between Hypofractionated Proton and Photon Radiotherapy of Liver Cancer Patients with Different Adaptive Schemes

With the availability of MRI linacs, online adaptive intensity modulated radiotherapy (IMRT) has become a treatment option for liver cancer patients, often combined with hypofractionation. Intensity modulated proton therapy (IMPT) has the potential to reduce the dose to healthy tissue, but it is particularly sensitive to changes in the beam path and might therefore benefit from online adaptation. This study compares the normal tissue complication probabilities (NTCPs) for liver and duodenal toxicity for adaptive and non-adaptive IMRT and IMPT treatments of liver cancer patients. Adaptive and non-adaptive IMRT and IMPT plans were optimized to 50 Gy (RBE = 1.1 for IMPT) in five fractions for 10 liver cancer patients, using the original MRI linac images and physician-drawn structures. Three liver NTCP models were used to predict radiation-induced liver disease, an increase in albumin-bilirubin level, and a Child-Pugh score increase of more than 2. Additionally, three duodenal NTCP models were used to predict gastric bleeding, gastrointestinal (GI) toxicity with grades >3, and duodenal toxicity grades 2-4. NTCPs were calculated for adaptive and non-adaptive IMRT and IMPT treatments. In general, IMRT showed higher NTCP values than IMPT and the differences were often significant. However, the differences between adaptive and non-adaptive treatment schemes were not significant, indicating that the NTCP benefit of adaptive treatment regimens is expected to be smaller than the expected difference between IMRT and IMPT.

Cancer Informatics for Cancer Centers: Sharing Ideas on How to Build an Artificial Intelligence-Ready Informatics Ecosystem for Radiation Oncology

In August 2022, the Cancer Informatics for Cancer Centers brought together cancer informatics leaders for its biannual symposium, Precision Medicine Applications in Radiation Oncology, co-chaired by Quynh-Thu Le, MD (Stanford University), and Walter J. Curran, MD (GenesisCare). Over the course of 3 days, presenters discussed a range of topics relevant to radiation oncology and the cancer informatics community more broadly, including biomarker development, decision support algorithms, novel imaging tools, theranostics, and artificial intelligence (AI) for the radiotherapy workflow. Since the symposium, there has been an impressive shift in the promise and potential for integration of AI in clinical care, accelerated in large part by major advances in generative AI. AI is now poised more than ever to revolutionize cancer care. Radiation oncology is a field that uses and generates a large amount of digital data and is therefore likely to be one of the first fields to be transformed by AI. As experts in the collection, management, and analysis of these data, the informatics community will take a leading role in ensuring that radiation oncology is prepared to take full advantage of these technological advances. In this report, we provide highlights from the symposium, which took place in Santa Barbara, California, from August 29 to 31, 2022. We discuss lessons learned from the symposium for data acquisition, management, representation, and sharing, and put these themes into context to prepare radiation oncology for the successful and safe integration of AI and informatics technologies.

Prediction of toxicity outcomes following radiotherapy using deep learning-based models: A systematic review

PURPOSE

This study aims to perform a comprehensive systematic review of deep learning (DL) models in predicting RT-induced toxicity.

MATERIALS AND METHODS

A literature review was performed following the PRISMA guidelines. Systematic searches were performed in PubMed, Scopus, Cochrane and Embase databases from the earliest record up to September 2022. Related studies on deep learning models for radiotherapy toxicity prediction were selected based on predefined PICOS criteria.

RESULTS

Fourteen studies of radiotherapy-treated patients on different types of cancer [prostate (n=2), HNC (n=4), liver (n=2), lung (n=4), cervical (n=1), and oesophagus (n=1)] were eligible for inclusion in the systematic review. Information regarding patient characteristics and model development was summarized. Several approaches, such as ensemble learning, data augmentation, and transfer learning, that were utilized by selected studies were discussed.

CONCLUSION

Deep learning techniques are able to produce a consistent performance for toxicity prediction. Future research using large and diverse datasets and standardization of the study methodologies are required to improve the consistency of the research output.

Deep learning with test-time augmentation for radial endobronchial ultrasound image differentiation: a multicentre verification study

PURPOSE

Despite the importance of radial endobronchial ultrasound (rEBUS) in transbronchial biopsy, researchers have yet to apply artificial intelligence to the analysis of rEBUS images.

MATERIALS AND METHODS

This study developed a convolutional neural network (CNN) to differentiate between malignant and benign tumours in rEBUS images. This study retrospectively collected rEBUS images from medical centres in Taiwan, including 769 from National Taiwan University Hospital Hsin-Chu Branch, Hsinchu Hospital for model training (615 images) and internal validation (154 images) as well as 300 from National Taiwan University Hospital (NTUH-TPE) and 92 images were obtained from National Taiwan University Hospital Hsin-Chu Branch, Biomedical Park Hospital (NTUH-BIO) for external validation. Further assessments of the model were performed using image augmentation in the training phase and test-time augmentation (TTA).

RESULTS

Using the internal validation dataset, the results were as follows: area under the curve (AUC) (0.88 (95% CI 0.83 to 0.92)), sensitivity (0.80 (95% CI 0.73 to 0.88)), specificity (0.75 (95% CI 0.66 to 0.83)). Using the NTUH-TPE external validation dataset, the results were as follows: AUC (0.76 (95% CI 0.71 to 0.80)), sensitivity (0.58 (95% CI 0.50 to 0.65)), specificity (0.92 (95% CI 0.88 to 0.97)). Using the NTUH-BIO external validation dataset, the results were as follows: AUC (0.72 (95% CI 0.64 to 0.82)), sensitivity (0.71 (95% CI 0.55 to 0.86)), specificity (0.76 (95% CI 0.64 to 0.87)). After fine-tuning, the AUC values for the external validation cohorts were as follows: NTUH-TPE (0.78) and NTUH-BIO (0.82). Our findings also demonstrated the feasibility of the model in differentiating between lung cancer subtypes, as indicated by the following AUC values: adenocarcinoma (0.70; 95% CI 0.64 to 0.76), squamous cell carcinoma (0.64; 95% CI 0.54 to 0.74) and small cell lung cancer (0.52; 95% CI 0.32 to 0.72).

CONCLUSIONS

Our results demonstrate the feasibility of the proposed CNN-based algorithm in differentiating between malignant and benign lesions in rEBUS images.

Radiotherapy toxicity prediction using knowledge-constrained generalized linear model

Radiation therapy (RT) is a frontline approach to treating cancer. While the target of radiation dose delivery is the tumor, there is an inevitable spill of dose to nearby normal organs causing complications. This phenomenon is known as radiotherapy toxicity. To predict the outcome of the toxicity, statistical models can be built based on dosimetric variables received by the normal organ at risk (OAR), known as Normal Tissue Complication Probability (NTCP) models. To tackle the challenge of the high dimensionality of dosimetric variables and limited clinical sample sizes, statistical models with variable selection techniques are viable choices. However, existing variable selection techniques are data-driven and do not integrate medical domain knowledge into the model formulation. We propose a knowledge-constrained generalized linear model (KC-GLM). KC-GLM includes a new mathematical formulation to translate three pieces of domain knowledge into non-negativity, monotonicity, and adjacent similarity constraints on the model coefficients. We further propose an equivalent transformation of the KC-GLM formulation, which makes it possible to solve the model coefficients using existing optimization solvers. Furthermore, we compare KC-GLM and several well-known variable selection techniques via a simulation study and on two real datasets of prostate cancer and lung cancer, respectively. These experiments show that KC-GLM selects variables with better interpretability, avoids producing counter-intuitive and misleading results, and has better prediction accuracy.

Deep learning techniques in liver tumour diagnosis using CT and MR imaging - A systematic review

Deep learning has become a thriving force in the computer aided diagnosis of liver cancer, as it solves extremely complicated challenges with high accuracy over time and facilitates medical experts in their diagnostic and treatment procedures. This paper presents a comprehensive systematic review on deep learning techniques applied for various applications pertaining to liver images, challenges faced by the clinicians in liver tumour diagnosis and how deep learning bridges the gap between clinical practice and technological solutions with an in-depth summary of 113 articles. Since, deep learning is an emerging revolutionary technology, recent state-of-the-art research implemented on liver images are reviewed with more focus on classification, segmentation and clinical applications in the management of liver diseases. Additionally, similar review articles in literature are reviewed and compared. The review is concluded by presenting the contemporary trends and unaddressed research issues in the field of liver tumour diagnosis, offering directions for future research in this field.

Artificial intelligence-assisted selection and efficacy prediction of antineoplastic strategies for precision cancer therapy

The rapid development of artificial intelligence (AI) technologies in the context of the vast amount of collectable data obtained from high-throughput sequencing has led to an unprecedented understanding of cancer and accelerated the advent of a new era of clinical oncology with a tone of precision treatment and personalized medicine. However, the gains achieved by a variety of AI models in clinical oncology practice are far from what one would expect, and in particular, there are still many uncertainties in the selection of clinical treatment options that pose significant challenges to the application of AI in clinical oncology. In this review, we summarize emerging approaches, relevant datasets and open-source software of AI and show how to integrate them to address problems from clinical oncology and cancer research. We focus on the principles and procedures for identifying different antitumor strategies with the assistance of AI, including targeted cancer therapy, conventional cancer therapy, and cancer immunotherapy. In addition, we also highlight the current challenges and directions of AI in clinical oncology translation. Overall, we hope this article will provide researchers and clinicians with a deeper understanding of the role and implications of AI in precision cancer therapy, and help AI move more quickly into accepted cancer guidelines.

Image masking using convolutional networks improves performance classification of radiation pneumonitis for non-small cell lung cancer

Radiation pneumonitis (RP) is a serious side effect of radiotherapy in patients with locally advanced non-small-cell lung cancer (NSCLC). Image cropping reduces training noise and may improve classification accuracy. This study proposes a prediction model for RP grade ≥ 2 using a convolutional neural network (CNN) model with image cropping. The 3D computed tomography (CT) images cropped in the whole-body, normal lung (nLung), and nLung regions overlapping the region over 20 Gy (nLung∩20 Gy) used in treatment planning were used as the input data. The output classifies patients as RP grade < 2 or RP grade ≥ 2. The sensitivity, specificity, accuracy, and area under the curve (AUC) were evaluated using the receiver operating characteristic curve (ROC). The accuracy, specificity, sensitivity, and AUC were 53.9%, 80.0%, 25.5%, and 0.58, respectively, for the whole-body method, and 60.0%, 81.7%, 36.4%, and 0.64, respectively, for the nLung method. For the nLung∩20 Gy method, the accuracy, specificity, sensitivity, and AUC improved to 75.7%, 80.0%, 70.9%, and 0.84, respectively. The CNN model, in which the input image is segmented in the normal lung considering the dose distribution, can help predict an RP grade ≥ 2 for NSCLC patients after definitive radiotherapy.

Clinical impact of artificial intelligence-based solutions on imaging of the pancreas and liver

Artificial intelligence (AI) has experienced substantial progress over the last ten years in many fields of application, including healthcare. In hepatology and pancreatology, major attention to date has been paid to its application to the assisted or even automated interpretation of radiological images, where AI can generate accurate and reproducible imaging diagnosis, reducing the physicians’ workload. AI can provide automatic or semi-automatic segmentation and registration of the liver and pancreatic glands and lesions. Furthermore, using radiomics, AI can introduce new quantitative information which is not visible to the human eye to radiological reports. AI has been applied in the detection and characterization of focal lesions and diffuse diseases of the liver and pancreas, such as neoplasms, chronic hepatic disease, or acute or chronic pancreatitis, among others. These solutions have been applied to different imaging techniques commonly used to diagnose liver and pancreatic diseases, such as ultrasound, endoscopic ultrasonography, computerized tomography (CT), magnetic resonance imaging, and positron emission tomography/CT. However, AI is also applied in this context to many other relevant steps involved in a comprehensive clinical scenario to manage a gastroenterological patient. AI can also be applied to choose the most convenient test prescription, to improve image quality or accelerate its acquisition, and to predict patient prognosis and treatment response. In this review, we summarize the current evidence on the application of AI to hepatic and pancreatic radiology, not only in regard to the interpretation of images, but also to all the steps involved in the radiological workflow in a broader sense. Lastly, we discuss the challenges and future directions of the clinical application of AI methods.

3D deep convolution neural network for radiation pneumonitis prediction following stereotactic body radiotherapy

In this study, we investigated 3D convolutional neural networks (CNNs) with input from radiographic and dosimetric datasets of primary lung tumors and surrounding lung volumes to predict the likelihood of radiation pneumonitis (RP). Pre-treatment, 3- and 6-month follow-up computed tomography (CT) and 3D dose datasets from one hundred and ninety-three NSCLC patients treated with stereotactic body radiotherapy (SBRT) were retrospectively collected and analyzed for this study. DenseNet-121 and ResNet-50 models were selected for this study as they are deep neural networks and have been proven to have high accuracy for complex image classification tasks. Both were modified with 3D convolution and max pooling layers to accept 3D datasets. We used a minority class oversampling approach and data augmentation to address the challenges of data imbalance and data scarcity. We built two sets of models for classification of three (No RP, Grade 1 RP, Grade 2 RP) and two (No RP, Yes RP) classes as outputs. The 3D DenseNet-121 models performed better (F1 score [0.81], AUC [0.91] [three class]; F1 score [0.77], AUC [0.84] [two class]) than the 3D ResNet-50 models (F1 score [0.54], AUC [0.72] [three-class]; F1 score [0.68], AUC [0.71] [two-class]) (p = 0.017 for three class predictions). We also attempted to identify salient regions within the input 3D image dataset via integrated gradient (IG) techniques to assess the relevance of the tumor surrounding volume for RP stratification. These techniques appeared to indicate the significance of the tumor and surrounding regions in the prediction of RP. Overall, 3D CNNs performed well to predict clinical RP in our cohort based on the provided image sets and radiotherapy dose information.

Predictive Model of Liver Toxicity to Aid the Personalized Selection of Proton Versus Photon Therapy in Hepatocellular Carcinoma

PURPOSE

Our objective was to develop an externally validated model for predicting liver toxicity after radiation therapy in patients with hepatocellular carcinoma (HCC) that can integrate both photon and proton dose distributions with patient-specific characteristics.

METHODS AND MATERIALS

Training data consisted of all patients with HCC treated between 2008 and 2019 at our institution (n = 117, 60%/40% photon/proton). We developed a shallow convolutional neural network (CNN) to predict posttreatment liver dysfunction from the differential dose-volume histogram (DVH) and baseline liver metrics. To reduce bias and improve robustness, we used ensemble learning (CNNE). After a preregistered study analysis plan, we evaluated stability using internal bootstrap resampling and generalizability using a data set from a different institution (n = 88). Finally, we implemented a class activation map method to characterize the critical DVH subregions and benchmarked the model against logistic regression and XGBoost. The models were evaluated using the area under the receiver operating characteristic curve and area under the precision-recall curve.

RESULTS

The CNNE model showed similar internal performance and robustness compared with the benchmarks. CNNE exceeded the benchmark models in external validation, with an area under the receiver operating characteristic curve of 0.78 versus 0.55 to 0.70, and an area under the precision-recall curve of 0.6 versus 0.43 to 0.52. The model showed improved predictive power in the photon group, excellent specificity in both modalities, and high sensitivity in the photon high-risk group. Models built solely on DVHs confirm outperformance of the CNNE and indicate that the proposed structure efficiently abstracts features from both proton and photon dose distributions. The activation map method demonstrates the importance of the low-dose bath and its interaction with low liver function at baseline.

CONCLUSIONS

We developed and externally validated a patient-specific prediction model for hepatic toxicity based on the entire DVH and clinical factors that can integrate both photon and proton therapy cohorts. This model complements the new American Society for Radiation Oncology clinical practice guidelines and could support value-driven integration of proton therapy into the management of HCC.

Identifying Solitary Granulomatous Nodules from Solid Lung Adenocarcinoma: Exploring Robust Image Features with Cross-Domain Transfer Learning

PURPOSE

This study aimed to find suitable source domain data in cross-domain transfer learning to extract robust image features. Then, a model was built to preoperatively distinguish lung granulomatous nodules (LGNs) from lung adenocarcinoma (LAC) in solitary pulmonary solid nodules (SPSNs).

METHODS

Data from 841 patients with SPSNs from five centres were collected retrospectively. First, adaptive cross-domain transfer learning was used to construct transfer learning signatures (TLS) under different source domain data and conduct a comparative analysis. The Wasserstein distance was used to assess the similarity between the source domain and target domain data in cross-domain transfer learning. Second, a cross-domain transfer learning radiomics model (TLRM) combining the best performing TLS, clinical factors and subjective CT findings was constructed. Finally, the performance of the model was validated through multicentre validation cohorts.

RESULTS

Relative to other source domain data, TLS based on lung whole slide images as source domain data (TLS-LW) had the best performance in all validation cohorts (AUC range: 0.8228-0.8984). Meanwhile, the Wasserstein distance of TLS-LW was 1.7108, which was minimal. Finally, TLS-LW, age, spiculated sign and lobulated shape were used to build the TLRM. In all validation cohorts, The AUC ranges were 0.9074-0.9442. Compared with other models, decision curve analysis and integrated discrimination improvement showed that TLRM had better performance.

CONCLUSIONS

The TLRM could assist physicians in preoperatively differentiating LGN from LAC in SPSNs. Furthermore, compared with other images, cross-domain transfer learning can extract robust image features when using lung whole slide images as source domain data and has a better effect.

Toxicity Prediction in Pelvic Radiotherapy Using Multiple Instance Learning and Cascaded Attention Layers

Modern radiotherapy delivers treatment plans optimised on an individual patient level, using CT-based 3D models of patient anatomy. This optimisation is fundamentally based on simple assumptions about the relationship between radiation dose delivered to the cancer (increased dose will increase cancer control) and normal tissue (increased dose will increase rate of side effects). The details of these relationships are still not well understood, especially for radiation-induced toxicity. We propose a convolutional neural network based on multiple instance learning to analyse toxicity relationships for patients receiving pelvic radiotherapy. A dataset comprising of 315 patients were included in this study; with 3D dose distributions, pre-treatment CT scans with annotated abdominal structures, and patient-reported toxicity scores provided for each participant. In addition, we propose a novel mechanism for segregating the attentions over space and dose/imaging features independently for a better understanding of the anatomical distribution of toxicity. Quantitative and qualitative experiments were performed to evaluate the network performance. The proposed network could predict toxicity with 80% accuracy. Attention analysis over space demonstrated that there was a significant association between radiation dose to the anterior and right iliac of the abdomen and patient-reported toxicity. Experimental results showed that the proposed network had outstanding performance for toxicity prediction, localisation and explanation with the ability of generalisation for an unseen dataset.

Radiomics: A radiological evidence-based artificial intelligence technique to facilitate personalized precision medicine in hepatocellular carcinoma

The high postoperative recurrence rates in hepatocellular carcinoma (HCC) remain a major hurdle in its management. Appropriate staging and treatment selection may alleviate the extent of fatal recurrence. However, effective methods to preoperatively evaluate pathophysiologic and molecular characteristics of HCC are lacking. Imaging plays a central role in HCC diagnosis and stratification due to the non-invasive diagnostic criteria. Vast and crucial information is hidden within image data. Other than providing a morphological sketch for lesion diagnosis, imaging could provide new insights to describe the pathophysiological and genetic landscape of HCC. Radiomics aims to facilitate diagnosis and prognosis of HCC using artificial intelligence techniques to harness the immense information contained in medical images. Radiomics produces a set of archetypal and robust imaging features that are correlated to key pathological or molecular biomarkers to preoperatively risk-stratify HCC patients. Inferred with outcome data, comprehensive combination of radiomic, clinical and/or multi-omics data could also improve direct prediction of response to treatment and prognosis. The evolution of radiomics is changing our understanding of personalized precision medicine in HCC management. Herein, we review the key techniques and clinical applications in HCC radiomics and discuss current limitations and future opportunities to improve clinical decision making.

Machine learning-based normal tissue complication probability model for predicting albumin-bilirubin (ALBI) grade increase in hepatocellular carcinoma patients

PURPOSE

The aim of this study was to develop a normal tissue complication probability model using a machine learning approach (ML-based NTCP) to predict the risk of radiation-induced liver disease in hepatocellular carcinoma (HCC) patients.

MATERIALS AND METHODS

The study population included 201 HCC patients treated with radiotherapy. The patients' medical records were retrospectively reviewed to obtain the clinical and radiotherapy data. Toxicity was defined by albumin-bilirubin (ALBI) grade increase. The normal liver dose-volume histogram was reduced to mean liver dose (MLD) based on the fraction size-adjusted equivalent uniform dose (2 Gy/fraction and α/β = 2). Three types of ML-based classification models were used, a penalized logistic regression (PLR), random forest (RF), and gradient-boosted tree (GBT) model. Model performance was compared using the area under the receiver operating characteristic curve (AUROC). Internal validation was performed by 5-fold cross validation and external validation was done in 44 new patients.

RESULTS

Liver toxicity occurred in 87 patients (43.1%). The best individual model was the GBT model using baseline liver function, liver volume, and MLD as inputs and the best overall model was an ensemble of the PLR and GBT models. An AUROC of 0.82 with a standard deviation of 0.06 was achieved for the internal validation. An AUROC of 0.78 with a standard deviation of 0.03 was achieved for the external validation. The behaviors of the best GBT model were also in good agreement with the domain knowledge on NTCP.

CONCLUSION

We propose the methodology to develop an ML-based NTCP model to estimate the risk of ALBI grade increase.

Predicting lymphovascular invasion in clinically node-negative breast cancer detected by abbreviated magnetic resonance imaging: Transfer learning vs. radiomics

Objective

To compare the performance of abbreviated breast magnetic resonance imaging (AB-MRI)-based transfer learning (TL) algorithm and radionics analysis for lymphovascular invasion (LVI) prediction in patients with clinically node-negative invasive breast cancer (IBC).

Methods

Between November 2017 and October 2020, 233 clinically node-negative IBCs detected by AB-MRI were retrospectively enrolled. One hundred thirty IBCs from center 1 (37 LVI-positive and 93 LVI-negative) were assigned as the training cohort and 103 from center 2 (25 LVI-positive and 78 LVI-negative) as the validation cohort. Based on AB-MRI, a TL signature (TLS) and a radiomics signature (RS) were built with the least absolute shrinkage and selection operator (LASSO) logistic regression. Their diagnostic performances were validated and compared using areas under the receiver operating curve (AUCs), net reclassification improvement (NRI), integrated discrimination improvement (IDI), decision curve analysis (DCA), and stratification analysis. A convolutional filter visualization technique was used to map the response areas of LVI on the AB-MRI.

Results

In the validation cohort, compared with RS, the TLS showed better capability in discriminating LVI-positive from LVI-negative lesions (AUC: 0.852 vs. 0.726, p < 0.001; IDI = 0.092, p < 0.001; NRI = 0.554, p < 0.001). The diagnostic performance of TLS was not affected by the menstrual state, molecular subtype, or contrast agent type (all p > 0.05). Moreover, DCA showed that the TLS added more net benefit than RS for clinical utility.

Conclusions

An AB-MRI-based TLS was superior to RS for preoperative LVI prediction in patients with clinically node-negative IBC.

Application value of a deep learning method based on a 3D V-Net convolutional neural network in the recognition and segmentation of the auditory ossicles

Objective

To explore the feasibility of a deep learning three-dimensional (3D) V-Net convolutional neural network to construct high-resolution computed tomography (HRCT)-based auditory ossicle structure recognition and segmentation models.

Methods

The temporal bone HRCT images of 158 patients were collected retrospectively, and the malleus, incus, and stapes were manually segmented. The 3D V-Net and U-Net convolutional neural networks were selected as the deep learning methods for segmenting the auditory ossicles. The temporal bone images were randomized into a training set (126 cases), a test set (16 cases), and a validation set (16 cases). Taking the results of manual segmentation as a control, the segmentation results of each model were compared.

Results

The Dice similarity coefficients (DSCs) of the malleus, incus, and stapes, which were automatically segmented with a 3D V-Net convolutional neural network and manually segmented from the HRCT images, were 0.920 ± 0.014, 0.925 ± 0.014, and 0.835 ± 0.035, respectively. The average surface distance (ASD) was 0.257 ± 0.054, 0.236 ± 0.047, and 0.258 ± 0.077, respectively. The Hausdorff distance (HD) 95 was 1.016 ± 0.080, 1.000 ± 0.000, and 1.027 ± 0.102, respectively. The DSCs of the malleus, incus, and stapes, which were automatically segmented using the 3D U-Net convolutional neural network and manually segmented from the HRCT images, were 0.876 ± 0.025, 0.889 ± 0.023, and 0.758 ± 0.044, respectively. The ASD was 0.439 ± 0.208, 0.361 ± 0.077, and 0.433 ± 0.108, respectively. The HD 95 was 1.361 ± 0.872, 1.174 ± 0.350, and 1.455 ± 0.618, respectively. As these results demonstrated, there was a statistically significant difference between the two groups (P < 0.001).

Conclusion

The 3D V-Net convolutional neural network yielded automatic recognition and segmentation of the auditory ossicles and produced similar accuracy to manual segmentation results.

Impact of Interfractional Error on Dosiomic Features

Objectives

The purpose of this study was to investigate the stability of dosiomic features under random interfractional error. We investigated the differences in the values of features with different fractions and the error in the values of dosiomic features under interfractional error.

Material and Methods

The isocenters of the treatment plans of 15 lung cancer patients were translated by a maximum of ±3 mm in each axis with a mean of (0, 0, 0) and a standard deviation of (1.2, 1.2, 1.2) mm in the x, y, and z directions for each fraction. A total of 81 dose distributions for each patient were then calculated considering four fraction number groups (2, 10, 20, and 30). A total of 93 dosiomic features were extracted from each dose distribution in four different regions of interest (ROIs): gross tumor volume (GTV), planning target volume (PTV), heart, and both lungs. The stability of dosiomic features was analyzed for each fraction number group by the coefficient of variation (CV) and intraclass correlation coefficient (ICC). The agreements in the means of dosiomic features among the four fraction number groups were tested by ICC. The percent differences (PD) between the dosiomic features extracted from the original dose distribution and the dosiomic features extracted from the dose distribution with interfractional error were calculated.

Results

Eleven out of 93 dosiomic features demonstrated a large CV (CV ≥ 20%). Overall CV values were highest in GTV ROIs and lowest in lung ROIs. The stability of dosiomic features decreased as the total number of fractions decreased. The ICC results showed that five out of 93 dosiomic features had an ICC lower than 0.75, which indicates intermediate or poor stability under interfractional error. The mean dosiomic feature values were shown to be consistent with different numbers of fractions (ICC ≥ 0.9). Some of the dosiomic features had PD greater than 50% and showed different PD values with different numbers of fractions.

Conclusion

Some dosiomic features have low stability under interfractional error. The stability and values of the dosiomic features were affected by the total number of fractions. The effect of interfractional error on dosiomic features should be considered in further studies regarding dosiomics for reproducible results.

A Survey on Deep Learning for Precision Oncology

Precision oncology, which ensures optimized cancer treatment tailored to the unique biology of a patient’s disease, has rapidly developed and is of great clinical importance. Deep learning has become the main method for precision oncology. This paper summarizes the recent deep-learning approaches relevant to precision oncology and reviews over 150 articles within the last six years. First, we survey the deep-learning approaches categorized by various precision oncology tasks, including the estimation of dose distribution for treatment planning, survival analysis and risk estimation after treatment, prediction of treatment response, and patient selection for treatment planning. Secondly, we provide an overview of the studies per anatomical area, including the brain, bladder, breast, bone, cervix, esophagus, gastric, head and neck, kidneys, liver, lung, pancreas, pelvis, prostate, and rectum. Finally, we highlight the challenges and discuss potential solutions for future research directions.

Feasibility study of fast intensity-modulated proton therapy dose prediction method using deep neural networks for prostate cancer

PURPOSE

Compared to the pencil-beam algorithm, the Monte-Carlo (MC) algorithm is more accurate for dose calculation but time-consuming in proton therapy. To solve this problem, this study uses deep learning to provide fast 3D dose prediction for prostate cancer patients treated with intensity-modulated proton therapy (IMPT).

METHODS

A novel recurrent U-net (RU-net) architecture was trained to predict the 3D dose distribution. Doses, CT images, and beam spot information from IMPT plans were used to train the RU-net with a 5-fold cross-validation. However, predicting the complicated dose properties of the IMPT plan is difficult for neural networks. Instead of the Peak-MU model, this work develops the Multi-MU model that adopted more comprehensive inputs and was trained with a combinational loss function. The dose difference between the prediction dose and MC dose was evaluated with gamma analysis, dice similarity coefficient (DSC), and dose-volume histogram (DVH) metrics. The Monte-Carlo dropout was also added to the network to quantify the uncertainty of the model.

RESULTS

Compared to the Peak-MU model, the Multi-MU model led to smaller mean absolute errors (3.03% vs. 2.05%, p = 0.005), higher gamma-passing rate (2mm, 3%: 97.42% vs. 93.69%, p = 0.005), higher dice similarity coefficient, and smaller relative DVH metrics error (CTV D_98% : 3.03% vs. 6.08%, p = 0.017; in Bladder V30: 3.08% vs. 5.28%, p = 0.028; and in Bladder V20: 3.02% vs. 4.42%, p = 0.017). Considering more prior knowledge, the Multi-MU model had better-predicted accuracy with a prediction time of less than half a second for each fold. The mean uncertainty value of the Multi-MU model is 0.46%, with a dropout rate of 10%.

CONCLUSION

This method was a nearly real-time IMPT dose prediction algorithm with accuracy comparable to the PB analytical algorithms used in prostate cancer. This RU-net might be used in plan robustness optimization and robustness evaluation in the future. This article is protected by copyright. All rights reserved.

Assessment of Skin Toxicity in an in Vitro Reconstituted Human Epidermis Model Using Deep Learning

Skin toxicity is a common safety concern associated with drugs that inhibit epidermal growth factor receptors as well as other targets involved in epidermal growth and differentiation. Recently, the use of a three-dimensional reconstructed human epidermis model enabled large-scale drug screening and showed potential for predicting skin toxicity. Although a decrease in epidermal thickness was often observed when the three-dimensional reconstructed tissues were exposed to drugs causing skin toxicity, the thickness evaluation of epidermal layers from a pathologist was subjective and not easily reproducible or scalable. In addition, the subtle differences in thickness among tissues, as well as the large number of samples tested, made cross-study comparison difficult when a manual evaluation strategy was used. The current study used deep learning and image-processing algorithms to measure the viable epidermal thickness from multiple studies and found that the measured thickness was not only significantly correlated with a pathologist's semi-quantitative evaluation but was also in close agreement with the quantitative measurement performed by pathologists. Moreover, a sensitivity of 0.8 and a specificity of 0.75 were achieved when predicting the toxicity of 18 compounds with clinical observations with these epidermal thickness algorithms. This approach is fully automated, reproducible, and highly scalable. It not only shows reasonable accuracy in predicting skin toxicity but also enables cross-study comparison and high-throughput compound screening.

The application of artificial intelligence in hepatology: A systematic review

The integration of human and artificial intelligence (AI) in medicine has only recently begun but it has already become obvious that intelligent systems can dramatically improve the management of liver diseases. Big data made it possible to envisage transformative developments of the use of AI for diagnosing, predicting prognosis and treating liver diseases, but there is still a lot of work to do. If we want to achieve the 21^st century digital revolution, there is an urgent need for specific national and international rules, and to adhere to bioethical parameters when collecting data. Avoiding misleading results is essential for the effective use of AI. A crucial question is whether it is possible to sustain, technically and morally, the process of integration between man and machine. We present a systematic review on the applications of AI to hepatology, highlighting the current challenges and crucial issues related to the use of such technologies.

Deep Learning for Radiotherapy Outcome Prediction Using Dose Data - A Review

Artificial intelligence, and in particular deep learning using convolutional neural networks, has been used extensively for image classification and segmentation, including on medical images for diagnosis and prognosis prediction. Use in radiotherapy prognostic modelling is still limited, however, especially as applied to toxicity and tumour response prediction from radiation dose distributions. We review and summarise studies that applied deep learning to radiotherapy dose data, in particular studies that utilised full three-dimensional dose distributions. Ten papers have reported on deep learning models for outcome prediction utilising spatial dose information, whereas four studies used reduced dimensionality (dose volume histogram) information for prediction. Many of these studies suffer from the same issues that plagued early normal tissue complication probability modelling, including small, single-institutional patient cohorts, lack of external validation, poor data and model reporting, use of late toxicity data without taking time-to-event into account, and nearly exclusive focus on clinician-reported complications. They demonstrate, however, how radiation dose, imaging and clinical data may be technically integrated in convolutional neural networks-based models; and some studies explore how deep learning may help better understand spatial variation in radiosensitivity. In general, there are a number of issues specific to the intersection of radiotherapy outcome modelling and deep learning, for example translation of model developments into treatment plan optimisation, which will require further combined effort from the radiation oncology and artificial intelligence communities.

Artificial Intelligence: A New Tool in Oncologist's Armamentarium

Artificial intelligence (AI) has become an essential tool in human life because of its pivotal role in communications, transportation, media, and social networking. Inspired by the complex neuronal network and its functions in human beings, AI, using computer-based algorithms and training, had been explored since the 1950s. To tackle the enormous amount of patients' clinical data, imaging, histopathological data, and the increasing pace of research on new treatments and clinical trials, and ever-changing guidelines for treatment with the advent of novel drugs and evidence, AI is the need of the hour. There are numerous publications and active work on AI's role in the field of oncology. In this review, we discuss the fundamental terminology of AI, its applications in oncology on the whole, and its limitations. There is an inter-relationship between AI, machine learning and, deep learning. The virtual branch of AI deals with machine learning. While the physical branch of AI deals with the delivery of different forms of treatment—surgery, targeted drug delivery, and elderly care. The applications of AI in oncology include cancer screening, diagnosis (clinical, imaging, and histopathological), radiation therapy (image acquisition, tumor and organs at risk segmentation, image registration, planning, and delivery), prediction of treatment outcomes and toxicities, prediction of cancer cell sensitivity to therapeutics and clinical decision-making. A specific area of interest is in the development of effective drug combinations tailored to every patient and tumor with the help of AI. Radiomics, the new kid on the block, deals with the planning and administration of radiotherapy. As with any new invention, AI has its fallacies. The limitations include lack of external validation and proof of generalizability, difficulty in data access for rare diseases, ethical and legal issues, no precise logic behind the prediction, and last but not the least, lack of education and expertise among medical professionals. A collaboration between departments of clinical oncology, bioinformatics, and data sciences can help overcome these problems in the near future.

Deep Neural Network With Consistency Regularization of Multi-Output Channels for Improved Tumor Detection and Delineation

Deep learning is becoming an indispensable tool for imaging applications, such as image segmentation, classification, and detection. In this work, we reformulate a standard deep learning problem into a new neural network architecture with multi-output channels, which reflects different facets of the objective, and apply the deep neural network to improve the performance of image segmentation. By adding one or more interrelated auxiliary-output channels, we impose an effective consistency regularization for the main task of pixelated classification (i.e., image segmentation). Specifically, multi-output-channel consistency regularization is realized by residual learning via additive paths that connect main-output channel and auxiliary-output channels in the network. The method is evaluated on the detection and delineation of lung and liver tumors with public data. The results clearly show that multi-output-channel consistency implemented by residual learning improves the standard deep neural network. The proposed framework is quite broad and should find widespread applications in various deep learning problems.

Dosimetric Uncertainties in Dominant Intraprostatic Lesion Simultaneous Boost Using Intensity Modulated Proton Therapy

Purpose

While intensity modulated proton therapy can deliver simultaneous integrated boost (SIB) to the dominant intraprostatic lesion (DIL) with high precision, it is sensitive to anatomic changes. We investigated the dosimetric effects from these changes based on pretreatment cone-beam computed tomographic (CBCT) images and identified the most important factors using a multilayer perceptron neural network (MLPNN).

Methods and Materials

DILs were contoured based on coregistered multiparametric magnetic resonance images for 25 previously treated prostate cancer patients. SIB plans were created with (1) prostate clinical target volume − V70 Gy = 98%; (2) DIL − V98 Gy > 95%; and (3) all organs at risk (OARs)"?> within clinical constraints. SIB plans were applied to daily CBCT-based deformed planning computed tomography (CT)"?>. DIL − V98 Gy, bladder/rectum maximum dose (Dmax) and volume changes, femur shifts, and the distance from DIL to organs at riskOARs"?> in both planning computed tomogramsCT"?> and CBCT were calculated. Wilcoxon signed-ranks tests were used to compare the changes. MLPNNs were used to model the change in ΔDIL − V98 Gy > 10% and bladder/rectum Dmax > 80 Gy, and the relative importance factors for the model were provided. The performances of the models were evaluated with receiver operating characteristic curves.

Results

Comparing initial plan to the average from evaluation plans, respectively, DIL − V98 Gy was 89.3% ± 19.9% versus 86.2% ± 21.3% (P = .151); bladder Dmax 71.9 ± 0.6 Gy versus 74.5 ± 2.9 Gy (P < .001); and rectum Dmax 70.1 ± 2.4 Gy versus 74.9 ± 9.1Gy (P = .007). Bladder and rectal volumes were 99.6% ± 39.5% and 112.8% ± 27.2%, respectively, of their initial volume. The femur shift was 3.16 ± 2.52 mm. In the modeling of ΔDIL V98 Gy > 10%, DIL to rectum distance changes, DIL to bladder distance changes, and rectum volume changes ratio are the 3 most important factors. The areas under the receiver operating characteristic curves were 0.89, 1.00, and 0.99 for the modeling of ΔDIL − V98 Gy > 10%, and bladder and rectum Dmax > 80 Gy, respectively.

Conclusions

Dosimetric changes in DIL SIB with intensity modulated proton therapy can be modeled and classified based on anatomic changes on pretreatment images by an MLPNN.

Artificial intelligence for assisting cancer diagnosis and treatment in the era of precision medicine

Over the past decade, artificial intelligence (AI) has contributed substantially to the resolution of various medical problems, including cancer. Deep learning (DL), a subfield of AI, is characterized by its ability to perform automated feature extraction and has great power in the assimilation and evaluation of large amounts of complicated data. On the basis of a large quantity of medical data and novel computational technologies, AI, especially DL, has been applied in various aspects of oncology research and has the potential to enhance cancer diagnosis and treatment. These applications range from early cancer detection, diagnosis, classification and grading, molecular characterization of tumors, prediction of patient outcomes and treatment responses, personalized treatment, automatic radiotherapy workflows, novel anti-cancer drug discovery, and clinical trials. In this review, we introduced the general principle of AI, summarized major areas of its application for cancer diagnosis and treatment, and discussed its future directions and remaining challenges. As the adoption of AI in clinical use is increasing, we anticipate the arrival of AI-powered cancer care.

Artificial intelligence in gastroenterology: A state-of-the-art review

The development of artificial intelligence (AI) has increased dramatically in the last 20 years, with clinical applications progressively being explored for most of the medical specialties. The field of gastroenterology and hepatology, substantially reliant on vast amounts of imaging studies, is not an exception. The clinical applications of AI systems in this field include the identification of premalignant or malignant lesions (e.g., identification of dysplasia or esophageal adenocarcinoma in Barrett’s esophagus, pancreatic malignancies), detection of lesions (e.g., polyp identification and classification, small-bowel bleeding lesion on capsule endoscopy, pancreatic cystic lesions), development of objective scoring systems for risk stratification, predicting disease prognosis or treatment response [e.g., determining survival in patients post-resection of hepatocellular carcinoma), determining which patients with inflammatory bowel disease (IBD) will benefit from biologic therapy], or evaluation of metrics such as bowel preparation score or quality of endoscopic examination. The objective of this comprehensive review is to analyze the available AI-related studies pertaining to the entirety of the gastrointestinal tract, including the upper, middle and lower tracts; IBD; the hepatobiliary system; and the pancreas, discussing the findings and clinical applications, as well as outlining the current limitations and future directions in this field.

Application of deep learning to auto-delineation of target volumes and organs at risk in radiotherapy

The technological advancement heralded the arrival of precision radiotherapy (RT), thereby increasing the therapeutic ratio and decreasing the side effects from treatment. Contour of target volumes (TV) and organs at risk (OARs) in RT is a complicated process. In recent years, automatic contouring of TV and OARs has rapidly developed due to the advances in deep learning (DL). This technology has the potential to save time and to reduce intra- or inter-observer variability. In this paper, the authors provide an overview of RT, introduce the concept of DL, summarize the data characteristics of the included literature, summarize the possible challenges for DL in the future, and discuss the possible research directions.

Prediction of liver Dmean for proton beam therapy using deep learning and contour-based data augmentation

The prediction of liver Dmean with 3-dimensional radiation treatment planning (3DRTP) is time consuming in the selection of proton beam therapy (PBT), and deep learning prediction generally requires large and tumor-specific databases. We developed a simple dose prediction tool (SDP) using deep learning and a novel contour-based data augmentation (CDA) approach and assessed its usability. We trained the SDP to predict the liver Dmean immediately. Five and two computed tomography (CT) data sets of actual patients with liver cancer were used for the training and validation. Data augmentation was performed by artificially embedding 199 contours of virtual clinical target volume (CTV) into CT images for each patient. The data sets of the CTVs and OARs are labeled with liver Dmean for six different treatment plans using two-dimensional calculations assuming all tissue densities as 1.0. The test of the validated model was performed using 10 unlabeled CT data sets of actual patients. Contouring only of the liver and CTV was required as input. The mean relative error (MRE), the mean percentage error (MPE) and regression coefficient between the planned and predicted Dmean was 0.1637, 6.6%, and 0.9455, respectively. The mean time required for the inference of liver Dmean of the six different treatment plans for a patient was 4.47±0.13 seconds. We conclude that the SDP is cost-effective and usable for gross estimation of liver Dmean in the clinic although the accuracy should be improved further if we need the accuracy of liver Dmean to be compatible with 3DRTP.

Deep learning method for prediction of patient-specific dose distribution in breast cancer

Background

Patient-specific dose prediction improves the efficiency and quality of radiation treatment planning and reduces the time required to find the optimal plan. In this study, a patient-specific dose prediction model was developed for a left-sided breast clinical case using deep learning, and its performance was compared with that of conventional knowledge-based planning using RapidPlan™.

Methods

Patient-specific dose prediction was performed using a contour image of the planning target volume (PTV) and organs at risk (OARs) with a U-net-based modified dose prediction neural network. A database of 50 volumetric modulated arc therapy (VMAT) plans for left-sided breast cancer patients was utilized to produce training and validation datasets. The dose prediction deep neural network (DpNet) feature weights of the previously learned convolution layers were applied to the test on a cohort of 10 test sets. With the same patient data set, dose prediction was performed for the 10 test sets after training in RapidPlan. The 3D dose distribution, absolute dose difference error, dose-volume histogram, 2D gamma index, and iso-dose dice similarity coefficient were used for quantitative evaluation of the dose prediction.

Results

The mean absolute error (MAE) and one standard deviation (SD) between the clinical and deep learning dose prediction models were 0.02 ± 0.04%, 0.01 ± 0.83%, 0.16 ± 0.82%, 0.52 ± 0.97, − 0.88 ± 1.83%, − 1.16 ± 2.58%, and − 0.97 ± 1.73% for D_95%, D_mean in the PTV, and the OARs of the body, left breast, heart, left lung, and right lung, respectively, and those measured between the clinical and RapidPlan dose prediction models were 0.02 ± 0.14%, 0.87 ± 0.63%, − 0.29 ± 0.98%, 1.30 ± 0.86%, − 0.32 ± 1.10%, 0.12 ± 2.13%, and − 1.74 ± 1.79, respectively.

Conclusions

In this study, a deep learning method for dose prediction was developed and was demonstrated to accurately predict patient-specific doses for left-sided breast cancer. Using the deep learning framework, the efficiency and accuracy of the dose prediction were compared to those of RapidPlan. The doses predicted by deep learning were superior to the results of the RapidPlan-generated VMAT plan.

Automated three-dimensional vessel reconstruction based on deep segmentation and bi-plane angiographic projections

Automated three-dimensional (3D) blood vessel reconstruction to improve vascular diagnosis and therapeutics is a challenging task in which the real-time implementation of automatic segmentation and specific vessel tracking for matching artery sequences is essential. Recently, a deep learning-based segmentation technique has been proposed; however, existing state-of-the-art deep architectures exhibit reduced performance when they are employed using real in-vivo imaging because of serious issues such as low contrast and noise contamination of the X-ray images. To overcome these limitations, we propose a novel methodology composed of the de-haze image enhancement technique as pre-processing and multi-level thresholding as post-processing to be applied to the lightweight multi-resolution U-shaped architecture. Specifically, (1) bi-plane two-dimensional (2D) vessel images were extracted simultaneously using the deep architecture, (2) skeletons of the vessels were computed via a morphology operation, (3) the corresponding skeleton structure between image sequences was matched using the shape-context technique, and (4) the 3D centerline was reconstructed using stereo geometry. The method was validated using both in-vivo and in-vitro models. The results show that the proposed technique could improve the segmentation quality, reduce computation time, and reconstruct the 3D skeleton automatically. The algorithm accurately reconstructed the phantom model and the real mouse vessel in 3D in 2 s. Our proposed technique has the potential to allow therapeutic micro-agent navigation in clinical practice, thereby providing the 3D position and orientation of the vessel.

Spatial descriptions of radiotherapy dose: normal tissue complication models and statistical associations

For decades, dose-volume information for segmented anatomy has provided the essential data for correlating radiotherapy dosimetry with treatment-induced complications. Dose-volume information has formed the basis for modelling those associations via normal tissue complication probability (NTCP) models and for driving treatment planning. Limitations to this approach have been identified. Many studies have emerged demonstrating that the incorporation of information describing the spatial nature of the dose distribution, and potentially its correlation with anatomy, can provide more robust associations with toxicity and seed more general NTCP models. Such approaches are culminating in the application of computationally intensive processes such as machine learning and the application of neural networks. The opportunities these approaches have for individualising treatment, predicting toxicity and expanding the solution space for radiation therapy are substantial and have clearly widespread and disruptive potential. Impediments to reaching that potential include issues associated with data collection, model generalisation and validation. This review examines the role of spatial models of complication and summarises relevant published studies. Sources of data for these studies, appropriate statistical methodology frameworks for processing spatial dose information and extracting relevant features are described. Spatial complication modelling is consolidated as a pathway to guiding future developments towards effective, complication-free radiotherapy treatment.

Application of Artificial Intelligence for the Diagnosis and Treatment of Liver Diseases

Modern medical care produces large volumes of multimodal patient data, which many clinicians struggle to process and synthesize into actionable knowledge. In recent years, artificial intelligence (AI) has emerged as an effective tool in this regard. The field of hepatology is no exception, with a growing number of studies published that apply AI techniques to the diagnosis and treatment of liver diseases. These have included machine-learning algorithms (such as regression models, Bayesian networks, and support vector machines) to predict disease progression, the presence of complications, and mortality; deep-learning algorithms to enable rapid, automated interpretation of radiologic and pathologic images; and natural-language processing to extract clinically meaningful concepts from vast quantities of unstructured data in electronic health records. This review article will provide a comprehensive overview of hepatology-focused AI research, discuss some of the barriers to clinical implementation and adoption, and suggest future directions for the field.

Atlas construction and spatial normalisation to facilitate radiation-induced late effects research in childhood cancer

Reducing radiation-induced side effects is one of the most important challenges in paediatric cancer treatment. Recently, there has been growing interest in using spatial normalisation to enable voxel-based analysis of radiation-induced toxicities in a variety of patient groups. The need to consider three-dimensional distribution of doses, rather than dose-volume histograms, is desirable but not yet explored in paediatric populations. In this paper, we investigate the feasibility of atlas construction and spatial normalisation in paediatric radiotherapy. We used planning computed tomography (CT) scans from twenty paediatric patients historically treated with craniospinal irradiation to generate a template CT that is suitable for spatial normalisation. This childhood cancer population representative template was constructed using groupwise image registration. An independent set of 53 subjects from a variety of childhood malignancies was then used to assess the quality of the propagation of new subjects to this common reference space using deformable image registration (i.e. spatial normalisation). The method was evaluated in terms of overall image similarity metrics, contour similarity and preservation of dose-volume properties. After spatial normalisation, we report a dice similarity coefficient of 0.95 ± 0.05, 0.85 ± 0.04, 0.96 ± 0.01, 0.91 ± 0.03, 0.83 ± 0.06 and 0.65 ± 0.16 for brain and spinal canal, ocular globes, lungs, liver, kidneys and bladder. We then demonstrated the potential advantages of an atlas-based approach to study the risk of second malignant neoplasms after radiotherapy. Our findings indicate satisfactory mapping between a heterogeneous group of patients and the template CT. The poorest performance was for organs in the abdominal and pelvic region, likely due to respiratory and physiological motion and to the highly deformable nature of abdominal organs. More specialised algorithms should be explored in the future to improve mapping in these regions. This study is the first step toward voxel-based analysis in radiation-induced toxicities following paediatric radiotherapy.

Current landscape and potential future applications of artificial intelligence in medical physics and radiotherapy

Artificial intelligence (AI) has seen tremendous growth over the past decade and stands to disrupts the medical industry. In medicine, this has been applied in medical imaging and other digitised medical disciplines, but in more traditional fields like medical physics, the adoption of AI is still at an early stage. Though AI is anticipated to be better than human in certain tasks, with the rapid growth of AI, there is increasing concerns for its usage. The focus of this paper is on the current landscape and potential future applications of artificial intelligence in medical physics and radiotherapy. Topics on AI for image acquisition, image segmentation, treatment delivery, quality assurance and outcome prediction will be explored as well as the interaction between human and AI. This will give insights into how we should approach and use the technology for enhancing the quality of clinical practice.

Artificial intelligence in gastroenterology and hepatology: Status and challenges

Originally proposed by John McCarthy in 1955, artificial intelligence (AI) has achieved a breakthrough and revolutionized the processing methods of clinical medicine with the increasing workloads of medical records and digital images. Doctors are paying attention to AI technologies for various diseases in the fields of gastroenterology and hepatology. This review will illustrate AI technology procedures for medical image analysis, including data processing, model establishment, and model validation. Furthermore, we will summarize AI applications in endoscopy, radiology, and pathology, such as detecting and evaluating lesions, facilitating treatment, and predicting treatment response and prognosis with excellent model performance. The current challenges for AI in clinical application include potential inherent bias in retrospective studies that requires larger samples for validation, ethics and legal concerns, and the incomprehensibility of the output results. Therefore, doctors and researchers should cooperate to address the current challenges and carry out further investigations to develop more accurate AI tools for improved clinical applications.

Artificial intelligence and machine learning for medical imaging: A technology review

Artificial intelligence (AI) has recently become a very popular buzzword, as a consequence of disruptive technical advances and impressive experimental results, notably in the field of image analysis and processing. In medicine, specialties where images are central, like radiology, pathology or oncology, have seized the opportunity and considerable efforts in research and development have been deployed to transfer the potential of AI to clinical applications. With AI becoming a more mainstream tool for typical medical imaging analysis tasks, such as diagnosis, segmentation, or classification, the key for a safe and efficient use of clinical AI applications relies, in part, on informed practitioners. The aim of this review is to present the basic technological pillars of AI, together with the state-of-the-art machine learning methods and their application to medical imaging. In addition, we discuss the new trends and future research directions. This will help the reader to understand how AI methods are now becoming an ubiquitous tool in any medical image analysis workflow and pave the way for the clinical implementation of AI-based solutions.

Closing the Gap Between Deep Neural Network Modeling and Biomedical Decision-Making Metrics in Segmentation via Adaptive Loss Functions

Deep learning is becoming an indispensable tool for various tasks in science and engineering. A critical step in constructing a reliable deep learning model is the selection of a loss function, which measures the discrepancy between the network prediction and the ground truth. While a variety of loss functions have been proposed in the literature, a truly optimal loss function that maximally utilizes the capacity of neural networks for deep learning-based decision-making has yet to be established. Here, we devise a generalized loss function with functional parameters determined adaptively during model training to provide a versatile framework for optimal neural network-based decision-making in small target segmentation. The method is showcased by more accurate detection and segmentation of lung and liver cancer tumors as compared with the current state-of-the-art. The proposed formalism opens new opportunities for numerous practical applications such as disease diagnosis, treatment planning, and prognosis.