Post-Radiotherapy PET Image Outcome Prediction by Deep Learning Under Biological Model Guidance: A Feasibility Study of Oropharyngeal Cancer Application

Characterization of arteriosclerosis based on computer-aided measurements of intra-arterial thickness

Purpose

Our purpose is to develop a computer vision approach to quantify intra-arterial thickness on digital pathology images of kidney biopsies as a computational biomarker of arteriosclerosis.

Approach

The severity of the arteriosclerosis was scored (0 to 3) in 753 arteries from 33 trichrome-stained whole slide images (WSIs) of kidney biopsies, and the outer contours of the media, intima, and lumen were manually delineated by a renal pathologist. We then developed a multi-class deep learning (DL) framework for segmenting the different intra-arterial compartments (training dataset: 648 arteries from 24 WSIs; testing dataset: 105 arteries from 9 WSIs). Subsequently, we employed radial sampling and made measurements of media and intima thickness as a function of spatially encoded polar coordinates throughout the artery. Pathomic features were extracted from the measurements to collectively describe the arterial wall characteristics. The technique was first validated through numerical analysis of simulated arteries, with systematic deformations applied to study their effect on arterial thickness measurements. We then compared these computationally derived measurements with the pathologists' grading of arteriosclerosis.

Results

Numerical validation shows that our measurement technique adeptly captured the decreasing smoothness in the intima and media thickness as the deformation increases in the simulated arteries. Intra-arterial DL segmentations of media, intima, and lumen achieved Dice scores of 0.84, 0.78, and 0.86, respectively. Several significant associations were identified between arteriosclerosis grade and pathomic features using our technique (e.g., intima-media ratio average [ τ = 0.52 , p < 0.0001 ]) through Kendall's tau analysis.

Conclusions

We developed a computer vision approach to computationally characterize intra-arterial morphology on digital pathology images and demonstrate its feasibility as a potential computational biomarker of arteriosclerosis.

Prediction of radiologic outcome-optimized dose plans and post-treatment magnetic resonance images: A proof-of-concept study in breast cancer brain metastases treated with stereotactic radiosurgery

Background and purpose

Information in multiparametric Magnetic Resonance (mpMR) images is relatable to voxel-level tumor response to Radiation Treatment (RT). We have investigated a deep learning framework to predict (i) post-treatment mpMR images from pre-treatment mpMR images and the dose map ("forward models"), and, (ii) the RT dose map that will produce prescribed changes within the Gross Tumor Volume (GTV) on post-treatment mpMR images ("inverse model"), in Breast Cancer Metastases to the Brain (BCMB) treated with Stereotactic Radiosurgery (SRS).

Materials and methods

Local outcomes, planning computed tomography (CT) images, dose maps, and pre-treatment and post-treatment Apparent Diffusion Coefficient of water (ADC) maps, T1-weighted unenhanced (T1w) and contrast-enhanced (T1wCE), T2-weighted (T2w) and Fluid-Attenuated Inversion Recovery (FLAIR) mpMR images were curated from 39 BCMB patients. mpMR images were co-registered to the planning CT and intensity-calibrated. A 2D pix2pix architecture was used to train 5 forward models (ADC, T2w, FLAIR, T1w, T1wCE) and 1 inverse model on 1940 slices from 18 BCMB patients, and tested on 437 slices from another 9 BCMB patients.

Results

Root Mean Square Percent Error (RMSPE) within the GTV between predicted and ground-truth post-RT images for the 5 forward models, in 136 test slices containing GTV, were (mean ± SD) 0.12 ± 0.044 (ADC), 0.14 ± 0.066 (T2w), 0.08 ± 0.038 (T1w), 0.13 ± 0.058 (T1wCE), and 0.09 ± 0.056 (FLAIR). RMSPE within the GTV on the same 136 test slices, between the predicted and ground-truth dose maps, was 0.37 ± 0.20 for the inverse model.

Conclusions

A deep learning-based approach for radiologic outcome-optimized dose planning in SRS of BCMB has been demonstrated.

PSMA-positive prostatic volume prediction with deep learning based on T2-weighted MRI

PURPOSE

High PSMA expression might be correlated with structural characteristics such as growth patterns on histopathology, not recognized by the human eye on MRI images. Deep structural image analysis might be able to detect such differences and therefore predict if a lesion would be PSMA positive. Therefore, we aimed to train a neural network based on PSMA PET/MRI scans to predict increased prostatic PSMA uptake based on the axial T2-weighted sequence alone.

MATERIAL AND METHODS

All patients undergoing simultaneous PSMA PET/MRI for PCa staging or biopsy guidance between April 2016 and December 2020 at our institution were selected. To increase the specificity of our model, the prostatic beds on PSMA PET scans were dichotomized in positive and negative regions using an SUV threshold greater than 4 to generate a PSMA PET map. Then, a C-ENet was trained on the T2 images of the training cohort to generate a predictive prostatic PSMA PET map.

RESULTS

One hundred and fifty-four PSMA PET/MRI scans were available (133 [68Ga]Ga-PSMA-11 and 21 [18F]PSMA-1007). Significant cancer was present in 127 of them. The whole dataset was divided into a training cohort (n = 124) and a test cohort (n = 30). The C-ENet was able to predict the PSMA PET map with a dice similarity coefficient of 69.5 ± 15.6%.

CONCLUSION

Increased prostatic PSMA uptake on PET might be estimated based on T2 MRI alone. Further investigation with larger cohorts and external validation is needed to assess whether PSMA uptake can be predicted accurately enough to help in the interpretation of mpMRI.

A Review of Machine Learning Approaches for Brain Positron Emission Tomography Data Analysis

Positron emission tomography (PET) imaging has moved forward the development of medical diagnostics and research across various domains, including cardiology, neurology, infection detection, and oncology. The integration of machine learning (ML) algorithms into PET data analysis has further enhanced their capabilities of including disease diagnosis and classification, image segmentation, and quantitative analysis. ML algorithms empower researchers and clinicians to extract valuable insights from complex big PET datasets, which enabling automated pattern recognition, predictive health outcome modeling, and more efficient data analysis. This review explains the basic knowledge of PET imaging, statistical methods for PET image analysis, and challenges of PET data analysis. We also discussed the improvement of analysis capabilities by combining PET data with machine learning algorithms and the application of this combination in various aspects of PET image research. This review also highlights current trends and future directions in PET imaging, emphasizing the driving and critical role of machine learning and big PET image data analytics in improving diagnostic accuracy and personalized medical approaches. Integration between PET imaging will shape the future of medical diagnosis and research.

Radiomics on spatial-temporal manifolds via Fokker-Planck dynamics

BACKGROUND

Delta radiomics is a high-throughput computational technique used to describe quantitative changes in serial, time-series imaging by considering the relative change in radiomic features of images extracted at two distinct time points. Recent work has demonstrated a lack of prognostic signal of radiomic features extracted using this technique. We hypothesize that this lack of signal is due to the fundamental assumptions made when extracting features via delta radiomics, and that other methods should be investigated.

PURPOSE

The purpose of this work was to show a proof-of-concept of a new radiomics paradigm for sparse, time-series imaging data, where features are extracted from a spatial-temporal manifold modeling the time evolution between images, and to assess the prognostic value on patients with oropharyngeal cancer (OPC).

METHODS

To accomplish this, we developed an algorithm to mathematically describe the relationship between two images acquired at timet = 0 $t = 0$ andt > 0 $t > 0$ . These images serve as boundary conditions of a partial differential equation describing the transition from one image to the other. To solve this equation, we propagate the position and momentum of each voxel according to Fokker-Planck dynamics (i.e., a technique common in statistical mechanics). This transformation is driven by an underlying potential force uniquely determined by the equilibrium image. The solution generates a spatial-temporal manifold (3 spatial dimensions + time) from which we define dynamic radiomic features. First, our approach was numerically verified by stochastically sampling dynamic Gaussian processes of monotonically decreasing noise. The transformation from high to low noise was compared between our Fokker-Planck estimation and simulated ground-truth. To demonstrate feasibility and clinical impact, we applied our approach to 18F-FDG-PET images to estimate early metabolic response of patients (n = 57) undergoing definitive (chemo)radiation for OPC. Images were acquired pre-treatment and 2-weeks intra-treatment (after 20 Gy). Dynamic radiomic features capturing changes in texture and morphology were then extracted. Patients were partitioned into two groups based on similar dynamic radiomic feature expression via k-means clustering and compared by Kaplan-Meier analyses with log-rank tests (p < 0.05). These results were compared to conventional delta radiomics to test the added value of our approach.

RESULTS

Numerical results confirmed our technique can recover image noise characteristics given sparse input data as boundary conditions. Our technique was able to model tumor shrinkage and metabolic response. While no delta radiomics features proved prognostic, Kaplan-Meier analyses identified nine significant dynamic radiomic features. The most significant feature was Gray-Level-Size-Zone-Matrix gray-level variance (p = 0.011), which demonstrated prognostic improvement over its corresponding delta radiomic feature (p = 0.722).

CONCLUSIONS

We developed, verified, and demonstrated the prognostic value of a novel, physics-based radiomics approach over conventional delta radiomics via data assimilation of quantitative imaging and differential equations.

A neural ordinary differential equation model for visualizing deep neural network behaviors in multi-parametric MRI-based glioma segmentation

PURPOSE

To develop a neural ordinary differential equation (ODE) model for visualizing deep neural network behavior during multi-parametric MRI-based glioma segmentation as a method to enhance deep learning explainability.

METHODS

By hypothesizing that deep feature extraction can be modeled as a spatiotemporally continuous process, we implemented a novel deep learning model, Neural ODE, in which deep feature extraction was governed by an ODE parameterized by a neural network. The dynamics of (1) MR images after interactions with the deep neural network and (2) segmentation formation can thus be visualized after solving the ODE. An accumulative contribution curve (ACC) was designed to quantitatively evaluate each MR image's utilization by the deep neural network toward the final segmentation results. The proposed Neural ODE model was demonstrated using 369 glioma patients with a 4-modality multi-parametric MRI protocol: T1, contrast-enhanced T1 (T1-Ce), T2, and FLAIR. Three Neural ODE models were trained to segment enhancing tumor (ET), tumor core (TC), and whole tumor (WT), respectively. The key MRI modalities with significant utilization by deep neural networks were identified based on ACC analysis. Segmentation results by deep neural networks using only the key MRI modalities were compared to those using all four MRI modalities in terms of Dice coefficient, accuracy, sensitivity, and specificity.

RESULTS

All Neural ODE models successfully illustrated image dynamics as expected. ACC analysis identified T1-Ce as the only key modality in ET and TC segmentations, while both FLAIR and T2 were key modalities in WT segmentation. Compared to the U-Net results using all four MRI modalities, the Dice coefficient of ET (0.784→0.775), TC (0.760→0.758), and WT (0.841→0.837) using the key modalities only had minimal differences without significance. Accuracy, sensitivity, and specificity results demonstrated the same patterns.

CONCLUSION

The Neural ODE model offers a new tool for optimizing the deep learning model inputs with enhanced explainability. The presented methodology can be generalized to other medical image-related deep-learning applications.

Application of artificial intelligence for overall survival risk stratification in oropharyngeal carcinoma: A validation of ProgTOOL

BACKGROUND

In recent years, there has been a surge in machine learning-based models for diagnosis and prognostication of outcomes in oncology. However, there are concerns relating to the model's reproducibility and generalizability to a separate patient cohort (i.e., external validation).

OBJECTIVES

This study primarily provides a validation study for a recently introduced and publicly available machine learning (ML) web-based prognostic tool (ProgTOOL) for overall survival risk stratification of oropharyngeal squamous cell carcinoma (OPSCC). Additionally, we reviewed the published studies that have utilized ML for outcome prognostication in OPSCC to examine how many of these models were externally validated, type of external validation, characteristics of the external dataset, and diagnostic performance characteristics on the internal validation (IV) and external validation (EV) datasets were extracted and compared.

METHODS

We used a total of 163 OPSCC patients obtained from the Helsinki University Hospital to externally validate the ProgTOOL for generalizability. In addition, PubMed, OvidMedline, Scopus, and Web of Science databases were systematically searched according to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines.

RESULTS

The ProgTOOL produced a predictive performance of 86.5% balanced accuracy, Mathew's correlation coefficient of 0.78, Net Benefit (0.7) and Brier score (0.06) for overall survival stratification of OPSCC patients as either low-chance or high-chance. In addition, out of a total of 31 studies found to have used ML for the prognostication of outcomes in OPSCC, only seven (22.6%) reported a form of EV. Three studies (42.9%) each used either temporal EV or geographical EV while only one study (14.2%) used expert as a form of EV. Most of the studies reported a reduction in performance when externally validated.

CONCLUSION

The performance of the model in this validation study indicates that it may be generalized, therefore, bringing recommendations of the model for clinical evaluation closer to reality. However, the number of externally validated ML-based models for OPSCC is still relatively small. This significantly limits the transfer of these models for clinical evaluation and subsequently reduces the likelihood of the use of these models in daily clinical practice. As a gold standard, we recommend the use of geographical EV and validation studies to reveal biases and overfitting of these models. These recommendations are poised to facilitate the implementation of these models in clinical practice.

Current status and future developments in predicting outcomes in radiation oncology

Advancements in data-driven technologies and the inclusion of information-rich multiomics features have significantly improved the performance of outcomes modeling in radiation oncology. For this current trend to be sustainable, challenges related to robust data modeling such as small sample size, low size to feature ratio, noisy data, as well as issues related to algorithmic modeling such as complexity, uncertainty, and interpretability, need to be mitigated if not resolved. Emerging computational technologies and new paradigms such as federated learning, human-in-the-loop, quantum computing, and novel interpretability methods show great potential in overcoming these challenges and bridging the gap towards precision outcome modeling in radiotherapy. Examples of these promising technologies will be presented and their potential role in improving outcome modeling will be discussed.