Non-Rigid Event-by-Event Continuous Respiratory Motion Compensated List-Mode Reconstruction for PET

Respiratory motion during positron emission tomography (PET)/computed tomography (CT) imaging can cause significant image blurring and underestimation of tracer concentration for both static and dynamic studies. In this paper, with the aim to eliminate both intra-cycle and inter-cycle motions, and apply to dynamic imaging, we developed a non-rigid event-by-event (NR-EBE) respiratory motion-compensated list-mode reconstruction algorithm. The proposed method consists of two components: the first component estimates a continuous non-rigid motion field of the internal organs using the internal-external motion correlation. This continuous motion field is then incorporated into the second component, non-rigid MOLAR (NR-MOLAR) reconstruction algorithm to deform the system matrix to the reference location where the attenuation CT is acquired. The point spread function (PSF) and time-of-flight (TOF) kernels in NR-MOLAR are incorporated in the system matrix calculation, and therefore are also deformed according to motion. We first validated NR-MOLAR using a XCAT phantom with a simulated respiratory motion. NR-EBE motion-compensated image reconstruction using both the components was then validated on three human studies injected with ¹⁸F-FPDTBZ and one with ¹⁸F-fluorodeoxyglucose (FDG) tracers. The human results were compared with conventional non-rigid motion correction using discrete motion field (NR-discrete, one motion field per gate) and a previously proposed rigid EBE motion-compensated image reconstruction (R-EBE) that was designed to correct for rigid motion on a target lesion/organ. The XCAT results demonstrated that NR-MOLAR incorporating both PSF and TOF kernels effectively corrected for non-rigid motion. The ¹⁸F-FPDTBZ studies showed that NR-EBE out-performed NR-Discrete, and yielded comparable results with R-EBE on target organs while yielding superior image quality in other regions. The FDG study showed that NR-EBE clearly improved the visibility of multiple moving lesions in the liver where some of them could not be discerned in other reconstructions, in addition to improving quantification. These results show that NR-EBE motion-compensated image reconstruction appears to be a promising tool for lesion detection and quantification when imaging thoracic and abdominal regions using PET.

Deep-learned placental vessel segmentation for intraoperative video enhancement in fetoscopic surgery

INTRODUCTION

Twin-to-twin transfusion syndrome (TTTS) is a potentially lethal condition that affects pregnancies in which twins share a single placenta. The definitive treatment for TTTS is fetoscopic laser photocoagulation, a procedure in which placental blood vessels are selectively cauterized. Challenges in this procedure include difficulty in quickly identifying placental blood vessels due to the many artifacts in the endoscopic video that the surgeon uses for navigation. We propose using deep-learned segmentations of blood vessels to create masks that can be recombined with the original fetoscopic video frame in such a way that the location of placental blood vessels is discernable at a glance.

METHODS

In a process approved by an institutional review board, intraoperative videos were acquired from ten fetoscopic laser photocoagulation surgeries performed at Yale New Haven Hospital. A total of 345 video frames were selected from these videos at regularly spaced time intervals. The video frames were segmented once by an expert human rater (a clinician) and once by a novice, but trained human rater (an undergraduate student). The segmentations were used to train a fully convolutional neural network of 25 layers.

RESULTS

The neural network was able to produce segmentations with a high similarity to ground truth segmentations produced by an expert human rater (sensitivity = 92.15% ± 10.69%) and produced segmentations that were significantly more accurate than those produced by a novice human rater (sensitivity = 56.87% ± 21.64%; p < 0.01).

CONCLUSION

A convolutional neural network can be trained to segment placental blood vessels with near-human accuracy and can exceed the accuracy of novice human raters. Recombining these segmentations with the original fetoscopic video frames can produced enhanced frames in which blood vessels are easily detectable. This has significant implications for aiding fetoscopic surgeons-especially trainees who are not yet at an expert level.

MR Imaging Biomarkers for the Prediction of Outcome after Radiofrequency Ablation of Hepatocellular Carcinoma: Qualitative and Quantitative Assessments of the Liver Imaging Reporting and Data System and Radiomic Features

PURPOSE

To assess the Liver Imaging Reporting and Data System (LI-RADS) and radiomic features in pretreatment magnetic resonance (MR) imaging for predicting progression-free survival (PFS) in patients with nodular hepatocellular carcinoma (HCC) treated with radiofrequency (RF) ablation.

MATERIAL AND METHODS

Sixty-five therapy-naïve patients with 85 nodular HCC tumors <5 cm in size were included in this Health Insurance Portability and Accountability Act-compliant, institutional review board-approved, retrospective study. All patients underwent RF ablation as first-line treatment and demonstrated complete response on the first follow-up imaging. Gadolinium-enhanced MR imaging biomarkers were analyzed for LI-RADS features by 2 board-certified radiologists or by analysis of nodular and perinodular radiomic features from 3-dimensional segmentations. A radiomic signature was calculated with the most informative features of a least absolute shrinkage and selection operator Cox regression model using leave-one-out cross-validation. The association between both LI-RADS features and radiomic signatures with PFS was assessed via the Kaplan-Meier analysis and a weighted log-rank test.

RESULTS

The median PFS was 19 months (95% confidence interval, 16.1-19.4) for a follow-up period of 24 months. Multifocality (P = .033); the appearance of capsular continuity, compared with an absent or discontinuous capsule (P = .012); and a higher radiomic signature based on nodular and perinodular features (P = .030) were associated with poorer PFS in early-stage HCC. The observation size, presence of arterial hyperenhancement, nonperipheral washout, and appearance of an enhancing "capsule" were not associated with PFS (P > .05).

CONCLUSIONS

Although multifocal HCC clearly indicates a more aggressive phenotype even in early-stage disease, the continuity of an enhancing capsule and a higher radiomic signature may add value as MR imaging biomarkers for poor PFS in HCC treated with RF ablation.

Incremental Learning Meets Transfer Learning: Application to Multi-site Prostate MRI Segmentation

Many medical datasets have recently been created for medical image segmentation tasks, and it is natural to question whether we can use them to sequentially train a single model that (1) performs better on all these datasets, and (2) generalizes well and transfers better to the unknown target site domain. Prior works have achieved this goal by jointly training one model on multi-site datasets, which achieve competitive performance on average but such methods rely on the assumption about the availability of all training data, thus limiting its effectiveness in practical deployment. In this paper, we propose a novel multi-site segmentation framework called incremental-transfer learning (ITL), which learns a model from multi-site datasets in an end-to-end sequential fashion. Specifically, "incremental" refers to training sequentially constructed datasets, and "transfer" is achieved by leveraging useful information from the linear combination of embedding features on each dataset. In addition, we introduce our ITL framework, where we train the network including a site-agnostic encoder with pretrained weights and at most two segmentation decoder heads. We also design a novel site-level incremental loss in order to generalize well on the target domain. Second, we show for the first time that leveraging our ITL training scheme is able to alleviate challenging catastrophic forgetting problems in incremental learning. We conduct experiments using five challenging benchmark datasets to validate the effectiveness of our incremental-transfer learning approach. Our approach makes minimal assumptions on computation resources and domain-specific expertise, and hence constitutes a strong starting point in multi-site medical image segmentation.

Supervised Machine Learning in Oncology: A Clinician's Guide

The widespread adoption of electronic health records has resulted in an abundance of imaging and clinical information. New data-processing technologies have the potential to revolutionize the practice of medicine by deriving clinically meaningful insights from large-volume data. Among those techniques is supervised machine learning, the study of computer algorithms that use self-improving models that learn from labeled data to solve problems. One clinical area of application for supervised machine learning is within oncology, where machine learning has been used for cancer diagnosis, staging, and prognostication. This review describes a framework to aid clinicians in understanding and critically evaluating studies applying supervised machine learning methods. Additionally, we describe current studies applying supervised machine learning techniques to the diagnosis, prognostication, and treatment of cancer, with a focus on gastroenterological cancers and other related pathologies.

Real-time computerized video enhancement for minimally invasive fetoscopic surgery

Background

The only definitive treatment for twin-to-twin transfusion syndrome is minimally invasive fetoscopic surgery for the selective coagulation of placental blood vessels. Fetoscopic surgery is a technically challenging operation, mainly due to the poor visibility conditions in the uterine environment. We present the design of an algorithm for the computerized enhancement of fetoscopic video and show that the enhanced video increases the ability of human users to identify blood vessels within fetoscopic video rapidly and accurately.

Methods

A computer algorithm for the enhancement of fetoscopic video frames was created. First, optical fiber artifacts were removed via a modification of unsharp masking. Second, image contrast was increased via Contrast Limited Adaptive Histogram Equalization (CLAHE). Third, the effect of contrast enhancements on stationary features was removed by normalizing to a windowed mean of the video frames. Fourth, color information was reincorporated by combining the mean-normalized result with the unnormalized contrast enhanced image using the soft light blending algorithm. Medical trainees (n = 16) were recruited into a study to validate the algorithm. Subjects were shown enhanced or unenhanced fetoscopic video frames on a screen and were asked to identify whether a randomly placed marker fell on a blood vessel or on background. The accuracy of their responses was recorded.

Results

On the subset of images where subjects had the lowest mean accuracy in identifying the placement of the marker, subjects performed better when viewing video frames enhanced by the computer (accuracy 74.27%; SE 0.97) than when viewing unenhanced video frames (accuracy 63.78%; SE 2.79). This result was statistically significant (p < 0.01).

Conclusion

Real-time computerized enhancement of fetoscopic video has the potential to ease the readability of video in poor lighting conditions, thus providing a benefit to the surgeon intraoperatively.

Patient-Specific Heart Geometry Modeling for Solid Biomechanics Using Deep Learning

Automated volumetric meshing of patient-specific heart geometry can help expedite various biomechanics studies, such as post-intervention stress estimation. Prior meshing techniques often neglect important modeling characteristics for successful downstream analyses, especially for thin structures like the valve leaflets. In this work, we present DeepCarve (Deep Cardiac Volumetric Mesh): a novel deformation-based deep learning method that automatically generates patient-specific volumetric meshes with high spatial accuracy and element quality. The main novelty in our method is the use of minimally sufficient surface mesh labels for precise spatial accuracy and the simultaneous optimization of isotropic and anisotropic deformation energies for volumetric mesh quality. Mesh generation takes only 0.13 seconds/scan during inference, and each mesh can be directly used for finite element analyses without any manual post-processing. Calcification meshes can also be subsequently incorporated for increased simulation accuracy. Numerous stent deployment simulations validate the viability of our approach for large-batch analyses. Our code is available at https://github.com/danpak94/Deep-Cardiac-Volumetric-Mesh.

Aging-Dependent Loss of Connectivity in Alzheimer's Model Mice with Rescue by mGluR5 Modulator

Amyloid accumulation in Alzheimer's disease (AD) is associated with synaptic damage and altered connectivity in brain networks. While measures of amyloid accumulation and biochemical changes in mouse models have utility for translational studies of certain therapeutics, preclinical analysis of altered brain connectivity using clinically relevant fMRI measures has not been well developed for agents intended to improve neural networks. Here, we conduct a longitudinal study in a double knock-in mouse model for AD ( App NL-G-F /hMapt ), monitoring brain connectivity by means of resting-state fMRI. While the 4-month-old AD mice are indistinguishable from wild-type controls (WT), decreased connectivity in the default-mode network is significant for the AD mice relative to WT mice by 6 months of age and is pronounced by 9 months of age. In a second cohort of 20-month-old mice with persistent functional connectivity deficits for AD relative to WT, we assess the impact of two-months of oral treatment with a silent allosteric modulator of mGluR5 (BMS-984923) known to rescue synaptic density. Functional connectivity deficits in the aged AD mice are reversed by the mGluR5-directed treatment. The longitudinal application of fMRI has enabled us to define the preclinical time trajectory of AD-related changes in functional connectivity, and to demonstrate a translatable metric for monitoring disease emergence, progression, and response to synapse-rescuing treatment.

Aging-dependent loss of functional connectivity in a mouse model of Alzheimer's disease and reversal by mGluR5 modulator

Amyloid accumulation in Alzheimer's disease (AD) is associated with synaptic damage and altered connectivity in brain networks. While measures of amyloid accumulation and biochemical changes in mouse models have utility for translational studies of certain therapeutics, preclinical analysis of altered brain connectivity using clinically relevant fMRI measures has not been well developed for agents intended to improve neural networks. Here, we conduct a longitudinal study in a double knock-in mouse model for AD (AppNL-G-F/hMapt), monitoring brain connectivity by means of resting-state fMRI. While the 4-month-old AD mice are indistinguishable from wild-type controls (WT), decreased connectivity in the default-mode network is significant for the AD mice relative to WT mice by 6 months of age and is pronounced by 9 months of age. In a second cohort of 20-month-old mice with persistent functional connectivity deficits for AD relative to WT, we assess the impact of two-months of oral treatment with a silent allosteric modulator of mGluR5 (BMS-984923/ALX001) known to rescue synaptic density. Functional connectivity deficits in the aged AD mice are reversed by the mGluR5-directed treatment. The longitudinal application of fMRI has enabled us to define the preclinical time trajectory of AD-related changes in functional connectivity, and to demonstrate a translatable metric for monitoring disease emergence, progression, and response to synapse-rescuing treatment.

Predicting tumor recurrence on baseline MR imaging in patients with early-stage hepatocellular carcinoma using deep machine learning

Tumor recurrence affects up to 70% of early-stage hepatocellular carcinoma (HCC) patients, depending on treatment option. Deep learning algorithms allow in-depth exploration of imaging data to discover imaging features that may be predictive of recurrence. This study explored the use of convolutional neural networks (CNN) to predict HCC recurrence in patients with early-stage HCC from pre-treatment magnetic resonance (MR) images. This retrospective study included 120 patients with early-stage HCC. Pre-treatment MR images were fed into a machine learning pipeline (VGG16 and XGBoost) to predict recurrence within six different time frames (range 1-6 years). Model performance was evaluated with the area under the receiver operating characteristic curves (AUC-ROC). After prediction, the model's clinical relevance was evaluated using Kaplan-Meier analysis with recurrence-free survival (RFS) as the endpoint. Of 120 patients, 44 had disease recurrence after therapy. Six different models performed with AUC values between 0.71 to 0.85. In Kaplan-Meier analysis, five of six models obtained statistical significance when predicting RFS (log-rank p < 0.05). Our proof-of-concept study indicates that deep learning algorithms can be utilized to predict early-stage HCC recurrence. Successful identification of high-risk recurrence candidates may help optimize follow-up imaging and improve long-term outcomes post-treatment.

Extent of High-Grade Prostatic Adenocarcinoma in Multiparametric Magnetic Resonance Imaging-Targeted Biopsy Enhances Prediction of Pathologic Stage

CONTEXT.—

Multiparametric magnetic resonance imaging (mpMRI) of prostate with targeted biopsy has enhanced detection of high-grade prostatic adenocarcinoma (HG PCa). However, utility of amount of HG PCa (Gleason pattern 4/5) in mpMRI-targeted biopsies versus standard 12-core biopsies in predicting adverse outcomes on radical prostatectomy (RP) is unknown.

OBJECTIVE.—

To examine the utility of amount of HG PCa in mpMRI-targeted biopsies versus standard 12-core biopsies in predicting adverse RP outcomes.

DESIGN.—

We performed a retrospective review of prostate biopsies, which had corresponding RP, 1 or more mpMRI-targeted biopsy, and grade group 2 disease or higher. For the 169 cases identified, total millimeters of carcinoma and HG PCa, and longest length HG PCa in a single core were recorded for 12-core biopsies and each set of mpMRI-targeted biopsies. For RP specimens, Gleason grade, extraprostatic extension, seminal vesicle involvement, and lymph node metastasis were recorded. The main outcome studied was prostate-confined disease at RP. A logistic regression model was used to test which pre-RP variables related to this outcome.

RESULTS.—

Univariate analysis showed significant associations with adverse RP outcomes in 5 of 8 quantifiable variables; longest millimeter HG PCa in a single 12-core biopsy, highest grade group in any core, and total millimeter HG in mpMRI-targeted biopsies showed no statistical association (P = .54, P = .13, and P = .55, respectively). In multivariate analysis, total millimeter carcinoma in all cores, highest GrGrp in any core, and longest millimeter HG PCa in a single mpMRI-targeted core provided additional predictive value (P < .001, P = .004, and P = .03, respectively).

CONCLUSIONS.—

Quantitation of HG PCa in mpMRI-targeted biopsies provides additional value over 12-core biopsies alone in predicting nonorgan confined prostate cancer at RP. Linear millimeters of HG PCa in mpMRI-targeted biopsies is a significant parameter associated with higher pathologic stage and could be of value in risk models.

Artificial Intelligence-Driven Patient Selection for Preoperative Portal Vein Embolization for Patients with Colorectal Cancer Liver Metastases

PURPOSE

To develop a machine learning algorithm to improve hepatic resection selection for patients with metastatic colorectal cancer (CRC) by predicting post-portal vein embolization (PVE) outcomes.

MATERIALS AND METHODS

This multicenter retrospective study (2000-2020) included 200 consecutive patients with CRC liver metastases planned for PVE before surgery. Data on radiomic features and laboratory values were collected. Patient-specific eigenvalues for each liver shape were calculated using a statistical shape model approach. After semiautomatic segmentation and review by a board-certified radiologist, the data were split 70%/30% for training and testing. Three machine learning algorithms predicting the total liver volume (TLV) after PVE, sufficient future liver remnant (FLR%), and kinetic growth rate (KGR%) were trained, with performance assessed using accuracy, sensitivity, specificity, area under the curve (AUC), or root mean squared error. Significance between the internal and external test sets was assessed by the Student t-test. One institution was kept separate as an external testing set.

RESULTS

A total of 114 (76 men; mean age, 56 years [SD± 12]) and 37 (19 men; mean age, 50 years ± [SD± 11]) patients met the inclusion criteria for the internal validation and external validation, respectively. Prediction accuracy and AUC for sufficient FLR% or liver growth potential (KGR%> 0%) were high in the internal testing set-85.81% (SD ± 1.01) and 0.91 (SD ± 0.01) or 87.44% (SD ± 0.10) and 0.66 (SD ± 0.03), respectively. Similar results occurred in the external testing set-79.66% (SD ± 0.60) and 0.88 (SD ± 0.00) or 72.06% (SD ± 0.30) and 0.69 (SD ± 0.01), respectively. TLV prediction showed discrepancy rates of 12.56% (SD ±4.20%; P = .86) internally and 13.57% (SD ± 3.76%; P = .91) externally.

CONCLUSIONS

Machine learning-based models incorporating radiomics and laboratory test results may help predict the FLR%, KGR%, and TLV as metrics for successful PVE.