Impact of Deep Learning Denoising Algorithm on Diffusion Tensor Imaging of the Growth Plate on Different Spatial Resolutions

To assess the impact of a deep learning (DL) denoising reconstruction algorithm applied to identical patient scans acquired with two different voxel dimensions, representing distinct spatial resolutions, this IRB-approved prospective study was conducted at a tertiary pediatric center in compliance with the Health Insurance Portability and Accountability Act. A General Electric Signa Premier unit (GE Medical Systems, Milwaukee, WI) was employed to acquire two DTI (diffusion tensor imaging) sequences of the left knee on each child at 3T: an in-plane 2.0 × 2.0 mm2 with section thickness of 3.0 mm and a 2 mm3 isovolumetric voxel; neither had an intersection gap. For image acquisition, a multi-band DTI with a fat-suppressed single-shot spin-echo echo-planar sequence (20 non-collinear directions; b-values of 0 and 600 s/mm2) was utilized. The MR vendor-provided a commercially available DL model which was applied with 75% noise reduction settings to the same subject DTI sequences at different spatial resolutions. We compared DTI tract metrics from both DL-reconstructed scans and non-denoised scans for the femur and tibia at each spatial resolution. Differences were evaluated using Wilcoxon-signed ranked test and Bland-Altman plots. When comparing DL versus non-denoised diffusion metrics in femur and tibia using the 2 mm × 2 mm × 3 mm voxel dimension, there were no significant differences between tract count (p = 0.1, p = 0.14) tract volume (p = 0.1, p = 0.29) or tibial tract length (p = 0.16); femur tract length exhibited a significant difference (p < 0.01). All diffusion metrics (tract count, volume, length, and fractional anisotropy (FA)) derived from the DL-reconstructed scans, were significantly different from the non-denoised scan DTI metrics in both the femur and tibial physes using the 2 mm3 voxel size (p < 0.001). DL reconstruction resulted in a significant decrease in femorotibial FA for both voxel dimensions (p < 0.01). Leveraging denoising algorithms could address the drawbacks of lower signal-to-noise ratios (SNRs) associated with smaller voxel volumes and capitalize on their better spatial resolutions, allowing for more accurate quantification of diffusion metrics.

Deep Learning-Assisted Diffusion Tensor Imaging for Evaluation of the Physis and Metaphysis

Diffusion tensor imaging of physis and metaphysis can be used as a biomarker to predict height change in the pediatric population. Current application of this technique requires manual segmentation of the physis which is time-consuming and introduces interobserver variability. UNET Transformers (UNETR) can be used for automatic segmentation to optimize workflow. Three hundred and eighty-five DTI scans from 191 subjects with mean age of 12.6 years ± 2.01 years were retrospectively used for training and validation. The mean Dice correlation coefficient was 0.81 for the UNETR model and 0.68 for the UNET. Manual extraction and segmentation took 15 min per volume, whereas both deep learning segmentation techniques took < 1 s per volume and were deterministic, always producing the same result for a given input. Intraclass correlation coefficient (ICC) for ROI-derived femur diffusion metrics was excellent for tract count (0.95), volume (0.95), and FA (0.97), and good for tract length (0.87). The results support the hypothesis that a hybrid UNETR model can be trained to replace the manual segmentation of physeal DTI images, therefore automating the process.

Prediction of locally advanced rectal cancer response to neoadjuvant chemoradiation therapy using volumetric multiparametric MRI-based radiomics

PURPOSE

To assess the role of pretreatment multiparametric (mp)MRI-based radiomic features in predicting pathologic complete response (pCR) of locally advanced rectal cancer (LARC) to neoadjuvant chemoradiation therapy (nCRT).

METHODS

This was a retrospective dual-center study including 98 patients (M/F 77/21, mean age 60 years) with LARC who underwent pretreatment mpMRI followed by nCRT and total mesorectal excision or watch and wait. Fifty-eight patients from institution 1 constituted the training set and 40 from institution 2 the validation set. Manual segmentation using volumes of interest was performed on T1WI pre-/post-contrast, T2WI and diffusion-weighted imaging (DWI) sequences. Demographic information and serum carcinoembryonic antigen (CEA) levels were collected. Shape, 1st and 2nd order radiomic features were extracted and entered in models based on principal component analysis used to predict pCR. The best model was obtained using a k-fold cross-validation method on the training set, and AUC, sensitivity and specificity for prediction of pCR were calculated on the validation set.

RESULTS

Stage distribution was T3 (n = 79) or T4 (n = 19). Overall, 16 (16.3%) patients achieved pCR. Demographics, MRI TNM stage, and CEA were not predictive of pCR (p range 0.59-0.96), while several radiomic models achieved high diagnostic performance for prediction of pCR (in the validation set), with AUCs ranging from 0.7 to 0.9, with the best model based on high b-value DWI demonstrating AUC of 0.9 [95% confidence intervals: 0.67, 1], sensitivity of 100% [100%, 100%], and specificity of 81% [66%, 96%].

CONCLUSION

Radiomic models obtained from pre-treatment MRI show good to excellent performance for the prediction of pCR in patients with LARC, superior to clinical parameters and CEA. A larger study is needed for confirmation of these results.

Comparison of lung cancer occurring in fibrotic versus non-fibrotic lung on chest CT

PURPOSE

Evaluate the behavior of lung nodules occurring in areas of pulmonary fibrosis and compare them to pulmonary nodules occurring in the non-fibrotic lung parenchyma.

METHODS

This retrospective review of chest CT scans and electronic medical records received expedited IRB approval and a waiver of informed consent. 4500 consecutive patients with a chest CT scan report containing the word fibrosis or a specific type of fibrosis were identified using the system M*Model Catalyst (Maplewood, Minnesota, U.S.). The largest nodule was measured in the longest dimension and re-evaluated, in the same way, on the follow-up exam if multiple time points were available. The nodule doubling time was calculated. If the patient developed cancer, the histologic diagnosis was documented.

RESULTS

Six hundred and nine patients were found to have at least one pulmonary nodule on either the first or the second CT scan. 274 of the largest pulmonary nodules were in the fibrotic tissue and 335 were in the non-fibrotic lung parenchyma. Pathology proven cancer was more common in nodules occurring in areas of pulmonary fibrosis compared to nodules occurring in areas of non-fibrotic lung (34% vs 15%, p < 0.01). Adenocarcinoma was the most common cell type in both groups but more frequent in cancers occurring in non-fibrotic tissue. In the non-fibrotic lung, 1 of 126 (0.8%) of nodules measuring 1 to 6 mm were cancer. In contrast, 5 of 49 (10.2%) of nodules in fibrosis measuring 1 to 6 mm represented biopsy-proven cancer (p < 0.01). The doubling time for squamous cell cancer was shorter in the fibrotic lung compared to non-fibrotic lung, however, the difference was not statistically significant (p = 0.24). 15 incident lung nodules on second CT obtained ≤ 18 months after first CT scan was found in fibrotic lung and eight (53%) were diagnosed as cancer.

CONCLUSIONS

Nodules occurring in fibrotic lung tissue are more likely to be cancer than nodules in the nonfibrotic lung. Incident pulmonary nodules in pulmonary fibrosis have a high likelihood of being cancer.

Lung nodule malignancy classification with associated pulmonary fibrosis using 3D attention-gated convolutional network with CT scans

BACKGROUND

Chest Computed tomography (CT) scans detect lung nodules and assess pulmonary fibrosis. While pulmonary fibrosis indicates increased lung cancer risk, current clinical practice characterizes nodule risk of malignancy based on nodule size and smoking history; little consideration is given to the fibrotic microenvironment.

PURPOSE

To evaluate the effect of incorporating fibrotic microenvironment into classifying malignancy of lung nodules in chest CT images using deep learning techniques.

MATERIALS AND METHODS

We developed a visualizable 3D classification model trained with in-house CT dataset for the nodule malignancy classification task. Three slightly-modified datasets were created: (1) nodule alone (microenvironment removed); (2) nodule with surrounding lung microenvironment; and (3) nodule in microenvironment with semantic fibrosis metadata. For each of the models, tenfold cross-validation was performed. Results were evaluated using quantitative measures, such as accuracy, sensitivity, specificity, and area-under-curve (AUC), as well as qualitative assessments, such as attention maps and class activation maps (CAM).

RESULTS

The classification model trained with nodule alone achieved 75.61% accuracy, 50.00% sensitivity, 88.46% specificity, and 0.78 AUC; the model trained with nodule and microenvironment achieved 79.03% accuracy, 65.46% sensitivity, 85.86% specificity, and 0.84 AUC. The model trained with additional semantic fibrosis metadata achieved 80.84% accuracy, 74.67% sensitivity, 84.95% specificity, and 0.89 AUC. Our visual evaluation of attention maps and CAM suggested that both the nodules and the microenvironment contributed to the task.

CONCLUSION

The nodule malignancy classification performance was found to be improving with microenvironment data. Further improvement was found when incorporating semantic fibrosis information.

Diffusion tensor imaging of the physis: the ABC's

The physis, or growth plate, is the primary structure responsible for longitudinal growth of the long bones. Diffusion tensor imaging (DTI) is a technique that depicts the anisotropic motion of water molecules, or diffusion. When diffusion is limited by cellular membranes, information on tissue microstructure can be acquired. Tractography, the visual display of the direction and magnitude of water diffusion, provides qualitative visualization of complex cellular architecture as well as quantitative diffusion metrics that appear to indirectly reflect physeal activity. In the growing bones, DTI depicts the columns of cartilage and new bone in the physeal-metaphyseal complex. In this "How I do It", we will highlight the value of DTI as a clinical tool by presenting DTI tractography of the physeal-metaphyseal complex of children and adolescents during normal growth, illustrating variation in qualitative and quantitative tractography metrics with age and skeletal location. In addition, we will present tractography from patients with physeal dysfunction caused by growth hormone deficiency and physeal injury due to trauma, chemotherapy, and radiation therapy. Furthermore, we will delineate our process, or "DTI pipeline," from image acquisition to data interpretation.

Deep Learning Approaches with Digital Mammography for Evaluating Breast Cancer Risk, a Narrative Review

Breast cancer remains the leading cause of cancer-related deaths in women worldwide. Current screening regimens and clinical breast cancer risk assessment models use risk factors such as demographics and patient history to guide policy and assess risk. Applications of artificial intelligence methods (AI) such as deep learning (DL) and convolutional neural networks (CNNs) to evaluate individual patient information and imaging showed promise as personalized risk models. We reviewed the current literature for studies related to deep learning and convolutional neural networks with digital mammography for assessing breast cancer risk. We discussed the literature and examined the ongoing and future applications of deep learning techniques in breast cancer risk modeling.

Chronic convection-enhanced delivery of topotecan for patients with recurrent glioblastoma: a first-in-patient, single-centre, single-arm, phase 1b trial

BACKGROUND

Topotecan is cytotoxic to glioma cells but is clinically ineffective because of drug delivery limitations. Systemic delivery is limited by toxicity and insufficient brain penetrance, and, to date, convection-enhanced delivery (CED) has been restricted to a single treatment of restricted duration. To address this problem, we engineered a subcutaneously implanted catheter-pump system capable of repeated, chronic (prolonged, pulsatile) CED of topotecan into the brain and tested its safety and biological effects in patients with recurrent glioblastoma.

METHODS

We did a single-centre, open-label, single-arm, phase 1b clinical trial at Columbia University Irving Medical Center (New York, NY, USA). Eligible patients were at least 18 years of age with solitary, histologically confirmed recurrent glioblastoma showing radiographic progression after surgery, radiotherapy, and chemotherapy, and a Karnofsky Performance Status of at least 70. Five patients had catheters stereotactically implanted into the glioma-infiltrated peritumoural brain and connected to subcutaneously implanted pumps that infused 146 μM topotecan 200 μL/h for 48 h, followed by a 5-7-day washout period before the next infusion, with four total infusions. After the fourth infusion, the pump was removed and the tumour was resected. The primary endpoint of the study was safety of the treatment regimen as defined by presence of serious adverse events. Analyses were done in all treated patients. The trial is closed, and is registered with ClinicalTrials.gov, NCT03154996.

FINDINGS

Between Jan 22, 2018, and July 8, 2019, chronic CED of topotecan was successfully completed safely in all five patients, and was well tolerated without substantial complications. The only grade 3 adverse event related to treatment was intraoperative supplemental motor area syndrome (one [20%] of five patients in the treatment group), and there were no grade 4 adverse events. Other serious adverse events were related to surgical resection and not the study treatment. Median follow-up was 12 months (IQR 10-17) from pump explant. Post-treatment tissue analysis showed that topotecan significantly reduced proliferating tumour cells in all five patients.

INTERPRETATION

In this small patient cohort, we showed that chronic CED of topotecan is a potentially safe and active therapy for recurrent glioblastoma. Our analysis provided a unique tissue-based assessment of treatment response without the need for large patient numbers. This novel delivery of topotecan overcomes limitations in delivery and treatment response assessment for patients with glioblastoma and could be applicable for other anti-glioma drugs or other CNS diseases. Further studies are warranted to determine the effect of this drug delivery approach on clinical outcomes.

FUNDING

US National Institutes of Health, The William Rhodes and Louise Tilzer Rhodes Center for Glioblastoma, the Michael Weiner Glioblastoma Research Into Treatment Fund, the Gary and Yael Fegel Foundation, and The Khatib Foundation.

Deep learning prediction of axillary lymph node status using ultrasound images

OBJECTIVE

To investigate the ability of our convolutional neural network (CNN) to predict axillary lymph node metastasis using primary breast cancer ultrasound (US) images.

METHODS

In this IRB-approved study, 338 US images (two orthogonal images) from 169 patients from 1/2014-12/2016 were used. Suspicious lymph nodes were seen on US and patients subsequently underwent core-biopsy. 64 patients had metastatic lymph nodes. A custom CNN was utilized on 248 US images from 124 patients in the training dataset and tested on 90 US images from 45 patients. The CNN was implemented entirely of 3 × 3 convolutional kernels and linear layers. The 9 convolutional kernels consisted of 6 residual layers, totaling 12 convolutional layers. Feature maps were down-sampled using strided convolutions. Dropout with a 0.5 keep probability and L2 normalization was utilized. Training was implemented by using the Adam optimizer and a final SoftMax score threshold of 0.5 from the average of raw logits from each pixel was used for two class classification (metastasis or not).

RESULTS

Our CNN achieved an AUC of 0.72 (SD ± 0.08) in predicting axillary lymph node metastasis from US images in the testing dataset. The model had an accuracy of 72.6% (SD ± 8.4) with a sensitivity and specificity of 65.5% (SD ± 28.6) and 78.9% (SD ± 15.1) respectively. Our algorithm is available to be shared for research use. (https://github.com/stmutasa/MetUS).

CONCLUSION

It's feasible to predict axillary lymph node metastasis from US images using a deep learning technique. This can potentially aid nodal staging in patients with breast cancer.

Dosimetric assessment of patient dose calculation on a deep learning-based synthesized computed tomography image for adaptive radiotherapy

Purpose

Dose computation using cone beam computed tomography (CBCT) images is inaccurate for the purpose of adaptive treatment planning. The main goal of this study is to assess the dosimetric accuracy of synthetic computed tomography (CT)‐based calculation for adaptive planning in the upper abdominal region. We hypothesized that deep learning‐based synthetically generated CT images will produce comparable results to a deformed CT (CTdef) in terms of dose calculation, while displaying a more accurate representation of the daily anatomy and therefore superior dosimetric accuracy.

Methods

We have implemented a cycle‐consistent generative adversarial networks (CycleGANs) architecture to synthesize CT images from the daily acquired CBCT image with minimal error. CBCT and CT images from 17 liver stereotactic body radiation therapy (SBRT) patients were used to train, test, and validate the algorithm.

Results

The synthetically generated images showed increased signal‐to‐noise ratio, contrast resolution, and reduced root mean square error, mean absolute error, noise, and artifact severity. Superior edge matching, sharpness, and preservation of anatomical structures from the CBCT images were observed for the synthetic images when compared to the CTdef registration method. Three verification plans (CBCT, CTdef, and synthetic) were created from the original treatment plan and dose volume histogram (DVH) statistics were calculated. The synthetic‐based calculation shows comparatively similar results to the CTdef‐based calculation with a maximum mean deviation of 1.5%.

Conclusions

Our findings show that CycleGANs can produce reliable synthetic images for the adaptive delivery framework. Dose calculations can be performed on synthetic images with minimal error. Additionally, enhanced image quality should translate into better daily alignment, increasing treatment delivery accuracy.

Weakly Supervised Deep Learning Approach to Breast MRI Assessment

RATIONALE AND OBJECTIVES

To evaluate a weakly supervised deep learning approach to breast Magnetic Resonance Imaging (MRI) assessment without pixel level segmentation in order to improve the specificity of breast MRI lesion classification.

MATERIALS AND METHODS

In this IRB approved study, the dataset consisted of 278,685 image slices from 438 patients. The weakly supervised network was based on the Resnet-101 architecture. Training was implemented using the Adam optimizer and a final SoftMax score threshold of 0.5 was used for two class classification (malignant or benign). 278,685 image slices were combined into 92,895 3-channel images. 79,871 (85%) images were used for training and validation while 13,024 (15%) images were separated for testing. Of the testing dataset, 11,498 (88%) were benign and 1531 (12%) were malignant. Model performance was assessed.

RESULTS

The weakly supervised network achieved an AUC of 0.92 (SD ± 0.03) in distinguishing malignant from benign images. The model had an accuracy of 94.2% (SD ± 3.4) with a sensitivity and specificity of 74.4% (SD ± 8.5) and 95.3% (SD ± 3.3) respectively.

CONCLUSION

It is feasible to use a weakly supervised deep learning approach to assess breast MRI images without the need for pixel-by-pixel segmentation yielding a high degree of specificity in lesion classification.

COVID-19 neuropathology at Columbia University Irving Medical Center/New York Presbyterian Hospital

Many patients with SARS-CoV-2 infection develop neurological signs and symptoms; although, to date, little evidence exists that primary infection of the brain is a significant contributing factor. We present the clinical, neuropathological and molecular findings of 41 consecutive patients with SARS-CoV-2 infections who died and underwent autopsy in our medical centre. The mean age was 74 years (38-97 years), 27 patients (66%) were male and 34 (83%) were of Hispanic/Latinx ethnicity. Twenty-four patients (59%) were admitted to the intensive care unit. Hospital-associated complications were common, including eight patients (20%) with deep vein thrombosis/pulmonary embolism, seven (17%) with acute kidney injury requiring dialysis and 10 (24%) with positive blood cultures during admission. Eight (20%) patients died within 24 h of hospital admission, while 11 (27%) died more than 4 weeks after hospital admission. Neuropathological examination of 20-30 areas from each brain revealed hypoxic/ischaemic changes in all brains, both global and focal; large and small infarcts, many of which appeared haemorrhagic; and microglial activation with microglial nodules accompanied by neuronophagia, most prominently in the brainstem. We observed sparse T lymphocyte accumulation in either perivascular regions or in the brain parenchyma. Many brains contained atherosclerosis of large arteries and arteriolosclerosis, although none showed evidence of vasculitis. Eighteen patients (44%) exhibited pathologies of neurodegenerative diseases, which was not unexpected given the age range of our patients. We examined multiple fresh frozen and fixed tissues from 28 brains for the presence of viral RNA and protein, using quantitative reverse-transcriptase PCR, RNAscope® and immunocytochemistry with primers, probes and antibodies directed against the spike and nucleocapsid regions. The PCR analysis revealed low to very low, but detectable, viral RNA levels in the majority of brains, although they were far lower than those in the nasal epithelia. RNAscope® and immunocytochemistry failed to detect viral RNA or protein in brains. Our findings indicate that the levels of detectable virus in coronavirus disease 2019 brains are very low and do not correlate with the histopathological alterations. These findings suggest that microglial activation, microglial nodules and neuronophagia, observed in the majority of brains, do not result from direct viral infection of brain parenchyma, but more likely from systemic inflammation, perhaps with synergistic contribution from hypoxia/ischaemia. Further studies are needed to define whether these pathologies, if present in patients who survive coronavirus disease 2019, might contribute to chronic neurological problems.

3D Isotropic Super-resolution Prostate MRI Using Generative Adversarial Networks and Unpaired Multiplane Slices

We developed a deep learning-based super-resolution model for prostate MRI. 2D T2-weighted turbo spin echo (T2w-TSE) images are the core anatomical sequences in a multiparametric MRI (mpMRI) protocol. These images have coarse through-plane resolution, are non-isotropic, and have long acquisition times (approximately 10-15 min). The model we developed aims to preserve high-frequency details that are normally lost after 3D reconstruction. We propose a novel framework for generating isotropic volumes using generative adversarial networks (GAN) from anisotropic 2D T2w-TSE and single-shot fast spin echo (ssFSE) images. The CycleGAN model used in this study allows the unpaired dataset mapping to reconstruct super-resolution (SR) volumes. Fivefold cross-validation was performed. The improvements from patch-to-volume reconstruction (PVR) to SR are 80.17%, 63.77%, and 186% for perceptual index (PI), RMSE, and SSIM, respectively; the improvements from slice-to-volume reconstruction (SVR) to SR are 72.41%, 17.44%, and 7.5% for PI, RMSE, and SSIM, respectively. Five ssFSE cases were used to test for generalizability; the perceptual quality of SR images surpasses the in-plane ssFSE images by 37.5%, with 3.26% improvement in SSIM and a higher RMSE by 7.92%. SR images were quantitatively assessed with radiologist Likert scores. Our isotropic SR volumes are able to reproduce high-frequency detail, maintaining comparable image quality to in-plane TSE images in all planes without sacrificing perceptual accuracy. The SR reconstruction networks were also successfully applied to the ssFSE images, demonstrating that high-quality isotropic volume achieved from ultra-fast acquisition is feasible.

Information Bottleneck Attribution for Visual Explanations of Diagnosis and Prognosis

Visual explanation methods have an important role in the prognosis of the patients where the annotated data is limited or unavailable. There have been several attempts to use gradient-based attribution methods to localize pathology from medical scans without using segmentation labels. This research direction has been impeded by the lack of robustness and reliability. These methods are highly sensitive to the network parameters. In this study, we introduce a robust visual explanation method to address this problem for medical applications. We provide an innovative visual explanation algorithm for general purpose and as an example application we demonstrate its effectiveness for quantifying lesions in the lungs caused by the Covid-19 with high accuracy and robustness without using dense segmentation labels. This approach overcomes the drawbacks of commonly used Grad-CAM and its extended versions. The premise behind our proposed strategy is that the information flow is minimized while ensuring the classifier prediction stays similar. Our findings indicate that the bottleneck condition provides a more stable severity estimation than the similar attribution methods. The source code will be publicly available upon publication.

The International Workshop on Osteoarthritis Imaging Knee MRI Segmentation Challenge: A Multi-Institute Evaluation and Analysis Framework on a Standardized Dataset

PURPOSE

To organize a multi-institute knee MRI segmentation challenge for characterizing the semantic and clinical efficacy of automatic segmentation methods relevant for monitoring osteoarthritis progression.

MATERIALS AND METHODS

A dataset partition consisting of three-dimensional knee MRI from 88 retrospective patients at two time points (baseline and 1-year follow-up) with ground truth articular (femoral, tibial, and patellar) cartilage and meniscus segmentations was standardized. Challenge submissions and a majority-vote ensemble were evaluated against ground truth segmentations using Dice score, average symmetric surface distance, volumetric overlap error, and coefficient of variation on a holdout test set. Similarities in automated segmentations were measured using pairwise Dice coefficient correlations. Articular cartilage thickness was computed longitudinally and with scans. Correlation between thickness error and segmentation metrics was measured using the Pearson correlation coefficient. Two empirical upper bounds for ensemble performance were computed using combinations of model outputs that consolidated true positives and true negatives.

RESULTS

Six teams (T 1-T 6) submitted entries for the challenge. No differences were observed across any segmentation metrics for any tissues (P = .99) among the four top-performing networks (T 2, T 3, T 4, T 6). Dice coefficient correlations between network pairs were high (> 0.85). Per-scan thickness errors were negligible among networks T 1-T 4 (P = .99), and longitudinal changes showed minimal bias (< 0.03 mm). Low correlations (ρ < 0.41) were observed between segmentation metrics and thickness error. The majority-vote ensemble was comparable to top-performing networks (P = .99). Empirical upper-bound performances were similar for both combinations (P = .99).

CONCLUSION

Diverse networks learned to segment the knee similarly, where high segmentation accuracy did not correlate with cartilage thickness accuracy and voting ensembles did not exceed individual network performance.See also the commentary by Elhalawani and Mak in this issue.Keywords: Cartilage, Knee, MR-Imaging, Segmentation © RSNA, 2020Supplemental material is available for this article.

Integrating Eye Tracking and Speech Recognition Accurately Annotates MR Brain Images for Deep Learning: Proof of Principle

PURPOSE

To generate and assess an algorithm combining eye tracking and speech recognition to extract brain lesion location labels automatically for deep learning (DL).

MATERIALS AND METHODS

In this retrospective study, 700 two-dimensional brain tumor MRI scans from the Brain Tumor Segmentation database were clinically interpreted. For each image, a single radiologist dictated a standard phrase describing the lesion into a microphone, simulating clinical interpretation. Eye-tracking data were recorded simultaneously. Using speech recognition, gaze points corresponding to each lesion were obtained. Lesion locations were used to train a keypoint detection convolutional neural network to find new lesions. A network was trained to localize lesions for an independent test set of 85 images. The statistical measure to evaluate our method was percent accuracy.

RESULTS

Eye tracking with speech recognition was 92% accurate in labeling lesion locations from the training dataset, thereby demonstrating that fully simulated interpretation can yield reliable tumor location labels. These labels became those that were used to train the DL network. The detection network trained on these labels predicted lesion location of a separate testing set with 85% accuracy.

CONCLUSION

The DL network was able to locate brain tumors on the basis of training data that were labeled automatically from simulated clinical image interpretation.© RSNA, 2020.

Prior to Initiation of Chemotherapy, Can We Predict Breast Tumor Response? Deep Learning Convolutional Neural Networks Approach Using a Breast MRI Tumor Dataset

We hypothesize that convolutional neural networks (CNN) can be used to predict neoadjuvant chemotherapy (NAC) response using a breast MRI tumor dataset prior to initiation of chemotherapy. An institutional review board-approved retrospective review of our database from January 2009 to June 2016 identified 141 locally advanced breast cancer patients who (1) underwent breast MRI prior to the initiation of NAC, (2) successfully completed adriamycin/taxane-based NAC, and (3) underwent surgical resection with available final surgical pathology data. Patients were classified into three groups based on their NAC response confirmed on final surgical pathology: complete (group 1), partial (group 2), and no response/progression (group 3). A total of 3107 volumetric slices of 141 tumors were evaluated. Breast tumor was identified on first T1 postcontrast dynamic images and underwent 3D segmentation. CNN consisted of ten convolutional layers, four max-pooling layers, and dropout of 50% after a fully connected layer. Dropout, augmentation, and L2 regularization were implemented to prevent overfitting of data. Non-linear functions were modeled by a rectified linear unit (ReLU). Batch normalization was used between the convolutional and ReLU layers to limit drift of layer activations during training. A three-class neoadjuvant prediction model was evaluated (group 1, group 2, or group 3). The CNN achieved an overall accuracy of 88% in three-class prediction of neoadjuvant treatment response. Three-class prediction discriminating one group from the other two was analyzed. Group 1 had a specificity of 95.1% ± 3.1%, sensitivity of 73.9% ± 4.5%, and accuracy of 87.7% ± 0.6%. Group 2 (partial response) had a specificity of 91.6% ± 1.3%, sensitivity of 82.4% ± 2.7%, and accuracy of 87.7% ± 0.6%. Group 3 (no response/progression) had a specificity of 93.4% ± 2.9%, sensitivity of 76.8% ± 5.7%, and accuracy of 87.8% ± 0.6%. It is feasible for current deep CNN architectures to be trained to predict NAC treatment response using a breast MRI dataset obtained prior to initiation of chemotherapy. Larger dataset will likely improve our prediction model.

Eye Tracking for Deep Learning Segmentation Using Convolutional Neural Networks

Deep learning with convolutional neural networks (CNNs) has experienced tremendous growth in multiple healthcare applications and has been shown to have high accuracy in semantic segmentation of medical (e.g., radiology and pathology) images. However, a key barrier in the required training of CNNs is obtaining large-scale and precisely annotated imaging data. We sought to address the lack of annotated data with eye tracking technology. As a proof of principle, our hypothesis was that segmentation masks generated with the help of eye tracking (ET) would be very similar to those rendered by hand annotation (HA). Additionally, our goal was to show that a CNN trained on ET masks would be equivalent to one trained on HA masks, the latter being the current standard approach. Step 1: Screen captures of 19 publicly available radiologic images of assorted structures within various modalities were analyzed. ET and HA masks for all regions of interest (ROIs) were generated from these image datasets. Step 2: Utilizing a similar approach, ET and HA masks for 356 publicly available T1-weighted postcontrast meningioma images were generated. Three hundred six of these image + mask pairs were used to train a CNN with U-net-based architecture. The remaining 50 images were used as the independent test set. Step 1: ET and HA masks for the nonneurological images had an average Dice similarity coefficient (DSC) of 0.86 between each other. Step 2: Meningioma ET and HA masks had an average DSC of 0.85 between each other. After separate training using both approaches, the ET approach performed virtually identically to HA on the test set of 50 images. The former had an area under the curve (AUC) of 0.88, while the latter had AUC of 0.87. ET and HA predictions had trimmed mean DSCs compared to the original HA maps of 0.73 and 0.74, respectively. These trimmed DSCs between ET and HA were found to be statistically equivalent with a p value of 0.015. We have demonstrated that ET can create segmentation masks suitable for deep learning semantic segmentation. Future work will integrate ET to produce masks in a faster, more natural manner that distracts less from typical radiology clinical workflow.

A novel CNN algorithm for pathological complete response prediction using an I-SPY TRIAL breast MRI database

PURPOSE

To apply our convolutional neural network (CNN) algorithm to predict neoadjuvant chemotherapy (NAC) response using the I-SPY TRIAL breast MRI dataset.

METHODS

From the I-SPY TRIAL breast MRI database, 131 patients from 9 institutions were successfully downloaded for analysis. First post-contrast MRI images were used for 3D segmentation using 3D slicer. Our CNN was implemented entirely of 3 × 3 convolutional kernels and linear layers. The convolutional kernels consisted of 6 residual layers, totaling 12 convolutional layers. Dropout with a 0.5 keep probability and L2 normalization was utilized. Training was implemented by using the Adam optimizer. A 5-fold cross validation was used for performance evaluation. Software code was written in Python using the TensorFlow module on a Linux workstation with one NVidia Titan X GPU.

RESULTS

Of 131 patients, 40 patients achieved pCR following NAC (group 1) and 91 patients did not achieve pCR following NAC (group 2). Diagnostic accuracy of our CNN two classification model distinguishing patients with pCR vs non-pCR was 72.5 (SD ± 8.4), with sensitivity 65.5% (SD ± 28.1) and specificity of 78.9% (SD ± 15.2). The area under a ROC Curve (AUC) was 0.72 (SD ± 0.08).

CONCLUSION

It is feasible to use our CNN algorithm to predict NAC response in patients using a multi-institution dataset.

Surface Point Cloud Ultrasound with Transcranial Doppler: Coregistration of Surface Point Cloud Ultrasound with Magnetic Resonance Angiography for Improved Reproducibility, Visualization, and Navigation in Transcranial Doppler Ultrasound

Transcranial Doppler (TCD) ultrasound is a standard tool used in the setting of recent sub-arachnoid hemorrhage (SAH). By tracking velocity in the circle-of-Willis vessels, vasospasm can be detected as interval velocity increase. For this disease process, repeated TCD velocity measurements over many days is the basis for its usefulness. However, a key limitation to TCD is its user dependence, which is itself largely due to the fact that exact information about probe positioning is lost between subsequent scans. Surface point cloud ultrasound (SPC-US) was recently introduced as a general approach combining ultrasound and three-dimensional surface imaging of patient + probe. In the present proof-of-principle demonstration, we have applied SPC-US to TCD and co-registered the skin surface with that from MRA images to provide a roadmap of the vasculature in 3D space for better speed, accuracy, reproducibility, and potential semi-automation of TCD. Collating the acronyms, we call the combined approach SPC-US-TCD. TCD of the M1 was obtained while three-dimensional photographic images were obtained with the Structure Sensor camera. MRA imaging was also obtained. SPC-US-TCD and corresponding MRA 3D reconstruction images were co-registered in MeshMixer using the skin surfaces for alignment. A cylinder the width of the TCD probe was placed over the fused images and aligned with the direction and orientation of the TCD probe to demonstrate the acoustic beam. In the fused images, the acoustic beam intersects the right M1 segment of the middle cerebral artery (MCA). The angle of insonation is well demonstrated and measurable in various planes. Distance measurements made in Blender localized the TCD probe position based on three skin surface landmarks, and tabulated orientation based on three angles along the corresponding directions. SPC-US-TCD provides valuable information that is otherwise not present in TCD studies. By co-registering SPC-US-TCD data with that from cross sectional vessel imaging, precise probe location relative to external skin surface landmarks as well as 3D vessel location relative to TCD probe placement offers the potential to provide a roadmap that improves exam reproducibility, speed of acquisition, and accuracy. The goal of future work is to demonstrate this improvement statistically by application to multiple patients and scans.

Radiomics of MRI for pretreatment prediction of pathologic complete response, tumor regression grade, and neoadjuvant rectal score in patients with locally advanced rectal cancer undergoing neoadjuvant chemoradiation: an international multicenter study

OBJECTIVE

To investigate whether pretreatment MRI-based radiomics of locally advanced rectal cancer (LARC) and/or the surrounding mesorectal compartment (MC) can predict pathologic complete response (pCR), neoadjuvant rectal (NAR) score, and tumor regression grade (TRG).

METHODS

One hundred thirty-two consecutive patients with LARC who underwent neoadjuvant chemoradiation and total mesorectal excision (TME) were retrospectively collected from 2 centers in the USA and Italy. The primary tumor and surrounding MC were segmented on the best available T2-weighted sequence (axial, coronal, or sagittal). Three thousand one hundred ninety radiomic features were extracted using a python package. The most salient radiomic features as well as MRI parameter and clinical-based features were selected using recursive feature elimination. A logistic regression classifier was built to distinguish between any 2 binned categories in the considered endpoints: pCR, NAR, and TRG. Repeated k-fold validation was performed and AUCs calculated.

RESULTS

There were 24, 87, and 21 T4, T3, and T2 LARCs, respectively (median age 63 years, 32 to 86). For NAR and TRG, the best classification performance was obtained using both the tumor and MC segmentations. The AUCs for classifying NAR 0 versus 2, pCR, and TRG 0/1 versus 2/3 were 0.66 (95% CI, 0.60-0.71), 0.80 (95% CI, 0.74-0.85), and 0.80 (95% CI, 0.77-0.82), respectively.

CONCLUSION

Radiomics of pretreatment MRIs can predict pCR, TRG, and NAR score in patients with LARC undergoing neoadjuvant treatment and TME with moderate accuracy despite extremely heterogenous image data. Both the tumor and MC contain important prognostic information.

KEY POINTS

• Machine learning of rectal cancer on images from the pretreatment MRI can predict important patient outcomes with moderate accuracy. • The tumor and the tissue around it both contain important prognostic information.

Potential Role of Convolutional Neural Network Based Algorithm in Patient Selection for DCIS Observation Trials Using a Mammogram Dataset

RATIONALE AND OBJECTIVES

We investigated the feasibility of utilizing convolutional neural network (CNN) for predicting patients with pure Ductal Carcinoma In Situ (DCIS) versus DCIS with invasion using mammographic images.

MATERIALS AND METHODS

An IRB-approved retrospective study was performed. 246 unique images from 123 patients were used for our CNN algorithm. In total, 164 images in 82 patients diagnosed with DCIS by stereotactic-guided biopsy of calcifications without any upgrade at the time of surgical excision (pure DCIS group). A total of 82 images in 41 patients with mammographic calcifications yielding occult invasive carcinoma as the final upgraded diagnosis on surgery (occult invasive group). Two standard mammographic magnification views (CC and ML/LM) of the calcifications were used for analysis. Calcifications were segmented using an open source software platform 3D Slicer and resized to fit a 128 × 128 pixel bounding box. A 15 hidden layer topology was used to implement the neural network. The network architecture contained five residual layers and dropout of 0.25 after each convolution. Five-fold cross validation was performed using training set (80%) and validation set (20%). Code was implemented in open source software Keras with TensorFlow on a Linux workstation with NVIDIA GTX 1070 Pascal GPU.

RESULTS

Our CNN algorithm for predicting patients with pure DCIS achieved an overall diagnostic accuracy of 74.6% (95% CI, ±5) with area under the ROC curve of 0.71 (95% CI, ±0.04), specificity of 91.6% (95% CI, ±5%) and sensitivity of 49.4% (95% CI, ±6%).

CONCLUSION

It's feasible to apply CNN to distinguish pure DCIS from DCIS with invasion with high specificity using mammographic images.

Channel width optimized neural networks for liver and vessel segmentation in liver iron quantification

INTRODUCTION

MRI T2* relaxometry protocols are often used for Liver Iron Quantification in patients with hemochromatosis. Several methods exist to semi-automatically segment parenchyma and exclude vessels for this calculation.

PURPOSE

To determine if inclusion of multiple echoes inputs to Convolutional Neural Networks (CNN) improves automated liver and vessel segmentation in MRI T2* relaxometry protocols and to determine if the resultant segmentations agree with manual segmentations for liver iron quantification analysis.

METHODS

Multi echo Gradient Recalled Echo (GRE) MRI sequence for T2* relaxometry was performed for 79 exams on 31 patients with hemochromatosis for iron quantification analysis. 275 axial liver slices were manually segmented as ground truth masks. A batch normalized U-Net with variable input width to incorporate multiple echoes is used for segmentation, using DICE as the accuracy metric. ANOVA is used to evaluate significance of channel width changes in segmentation accuracy. Linear regression is used to model the relationship of channel width on segmentation accuracy. Liver segmentations are applied to relaxometry data to calculate liver T2* yielding liver iron concentration(LIC) derived from literature based calibration curves. Manual and CNN based LIC values are compared with Pearson correlation. Bland altman plots are used to visualize differences between manual and CNN based LIC values.

RESULTS

Performance metrics are tested on 55 hold out slices. Linear regression indicates that there is a monotonic increase of DICE with increasing channel depth (p = 0.001) with a slope of 3.61e-3. ANOVA indicates a significant increase segmentation accuracy over single channel starting at 3 channels. Incorporation of all channels results in an average DICE of 0.86, an average increase of 0.07 over single channel. The calculated LIC from CNN segmented livers agrees well with manual segmentation (R = 0.998, slope = 0.914, p«0.001), with an average absolute difference 0.27 ± 0.99 mg Fe/g or 1.34 ± 4.3%.

CONCLUSION

More input echoes yields higher model accuracy until the noise floor. Echos beyond the first three echo times in GRE based T2* relaxometry do not contribute significant information for segmentation of liver for LIC calculation. Deep learning models with three channel width allow for generalization of model to protocols of more than three echoes, effectively a universal requirement for relaxometry. Deep learning segmentations achieve a good accuracy compared with manual segmentations with minimal preprocessing. Liver iron values calculated from hand segmented liver and Neural network segmented liver were not statistically different from each other.

Fully Automated Convolutional Neural Network Method for Quantification of Breast MRI Fibroglandular Tissue and Background Parenchymal Enhancement

The aim of this study is to develop a fully automated convolutional neural network (CNN) method for quantification of breast MRI fibroglandular tissue (FGT) and background parenchymal enhancement (BPE). An institutional review board-approved retrospective study evaluated 1114 breast volumes in 137 patients using T1 precontrast, T1 postcontrast, and T1 subtraction images. First, using our previously published method of quantification, we manually segmented and calculated the amount of FGT and BPE to establish ground truth parameters. Then, a novel 3D CNN modified from the standard 2D U-Net architecture was developed and implemented for voxel-wise prediction whole breast and FGT margins. In the collapsing arm of the network, a series of 3D convolutional filters of size 3 × 3 × 3 are applied for standard CNN hierarchical feature extraction. To reduce feature map dimensionality, a 3 × 3 × 3 convolutional filter with stride 2 in all directions is applied; a total of 4 such operations are used. In the expanding arm of the network, a series of convolutional transpose filters of size 3 × 3 × 3 are used to up-sample each intermediate layer. To synthesize features at multiple resolutions, connections are introduced between the collapsing and expanding arms of the network. L2 regularization was implemented to prevent over-fitting. Cases were separated into training (80%) and test sets (20%). Fivefold cross-validation was performed. Software code was written in Python using the TensorFlow module on a Linux workstation with NVIDIA GTX Titan X GPU. In the test set, the fully automated CNN method for quantifying the amount of FGT yielded accuracy of 0.813 (cross-validation Dice score coefficient) and Pearson correlation of 0.975. For quantifying the amount of BPE, the CNN method yielded accuracy of 0.829 and Pearson correlation of 0.955. Our CNN network was able to quantify FGT and BPE within an average of 0.42 s per MRI case. A fully automated CNN method can be utilized to quantify MRI FGT and BPE. Larger dataset will likely improve our model.

Repeatability of Quantitative Diffusion-Weighted Imaging Metrics in Phantoms, Head-and-Neck and Thyroid Cancers: Preliminary Findings

The aim of this study was to establish the repeatability measures of quantitative Gaussian and non-Gaussian diffusion metrics using diffusion-weighted imaging (DWI) data from phantoms and patients with head-and-neck and papillary thyroid cancers. The Quantitative Imaging Biomarker Alliance (QIBA) DWI phantom and a novel isotropic diffusion kurtosis imaging phantom were scanned at 3 different sites, on 1.5T and 3T magnetic resonance imaging systems, using standardized multiple b-value DWI acquisition protocol. In the clinical component of this study, a total of 60 multiple b-value DWI data sets were analyzed for test–retest, obtained from 14 patients (9 head-and-neck squamous cell carcinoma and 5 papillary thyroid cancers). Repeatability of quantitative DWI measurements was assessed by within-subject coefficient of variation (wCV%) and Bland–Altman analysis. In isotropic diffusion kurtosis imaging phantom vial with 2% ceteryl alcohol and behentrimonium chloride solution, the mean apparent diffusion (D_app × 10⁻³ mm²/s) and kurtosis (K_app, unitless) coefficient values were 1.02 and 1.68 respectively, capturing in vivo tumor cellularity and tissue microstructure. For the same vial, D_app and K_app mean wCVs (%) were ≤1.41% and ≤0.43% for 1.5T and 3T across 3 sites. For pretreatment head-and-neck squamous cell carcinoma, apparent diffusion coefficient, D, D*, K, and f mean wCVs (%) were 2.38%, 3.55%, 3.88%, 8.0%, and 9.92%, respectively; wCVs exhibited a higher trend for papillary thyroid cancers. Knowledge of technical precision and bias of quantitative imaging metrics enables investigators to properly design and power clinical trials and better discern between measurement variability versus biological change.

A threshold-based method to predict thyroid nodules on scintigraphy scans

Nuclear Medicine imaging is an important modality to follow up abnormalities of thyroid function tests and to uncover and characterize thyroid nodules either de novo or as previously seen on other imaging modalities, namely ultrasound. In general, the hypofunctioning 'cold' nodules pose a higher malignancy potential than hyperfunctioning 'hot' nodules, for which the risk is <1%. Hot nodules are detected by the radiologist as a region of focal increased radiotracer uptake, which appears as a density of pixels that is higher than surrounding normal thyroid parenchyma. Similarly, cold nodules show decreased density of pixels, corresponding to their decreased uptake of radiotracer, and are photopenic. Partly because Nuclear Medicine images have poor resolution, these density variations can sometimes be subtle, and a second reader computer-aided detection (CAD) scheme that can highlight hot/cold nodules has the potential to reduce false negatives by bringing the radiologists' attention to the occasional overlooked nodules. Our approach subdivides thyroid images into small regions and employs a set of pixel density cutoffs, marking regions that fulfill density criteria. Thresholding is a fundamental tool in image processing. In nuclear medicine, scroll bars to adjust standardized uptake value cutoffs are already in wide commercial use in PET/CT display systems. A similar system could be used for planar thyroid images, whereby the user varies threshold and highlights suspect regions after an initial reader survey of the images. We hypothesized that a thresholding approach would accurately detect both hot and cold thyroid nodules relative to expert readers. Analyzing 22 nodules, half of them hot and the other half cold, we found good agreement between highlighted candidate nodules and the consensus selections of two expert readers, with nonzero overlap between expert and CAD selections in all cases.

Convolutional Neural Network Detection of Axillary Lymph Node Metastasis Using Standard Clinical Breast MRI

BACKGROUND

Axillary lymph node status is important for breast cancer staging and treatment planning as the majority of breast cancer metastasis spreads through the axillary lymph nodes. There is currently no reliable noninvasive imaging method to detect nodal metastasis associated with breast cancer.

MATERIALS AND METHODS

Magnetic resonance imaging (MRI) data were those from the peak contrast dynamic image from 1.5 Tesla MRI scanners at the pre-neoadjuvant chemotherapy stage. Data consisted of 66 abnormal nodes from 38 patients and 193 normal nodes from 61 patients. Abnormal nodes were those determined by expert radiologist based on ¹⁸Fluorodeoxyglucose positron emission tomography images. Normal nodes were those with negative diagnosis of breast cancer. The convolutional neural network consisted of 5 convolutional layers with filters from 16 to 128. Receiver operating characteristic analysis was performed to evaluate prediction performance. For comparison, an expert radiologist also scored the same nodes as normal or abnormal.

RESULTS

The convolutional neural network model yielded a specificity of 79.3% ± 5.1%, sensitivity of 92.1% ± 2.9%, positive predictive value of 76.9% ± 4.0%, negative predictive value of 93.3% ± 1.9%, accuracy of 84.8% ± 2.4%, and receiver operating characteristic area under the curve of 0.91 ± 0.02 for the validation data set. These results compared favorably with scoring by radiologists (accuracy of 78%).

CONCLUSION

The results are encouraging and suggest that this approach may prove useful for classifying lymph node status on MRI in clinical settings in patients with breast cancer, although additional studies are needed before routine clinical use can be realized. This approach has the potential to ultimately be a noninvasive alternative to lymph node biopsy.

Axillary Lymph Node Evaluation Utilizing Convolutional Neural Networks Using MRI Dataset

The aim of this study is to evaluate the role of convolutional neural network (CNN) in predicting axillary lymph node metastasis, using a breast MRI dataset. An institutional review board (IRB)-approved retrospective review of our database from 1/2013 to 6/2016 identified 275 axillary lymph nodes for this study. Biopsy-proven 133 metastatic axillary lymph nodes and 142 negative control lymph nodes were identified based on benign biopsies (100) and from healthy MRI screening patients (42) with at least 3 years of negative follow-up. For each breast MRI, axillary lymph node was identified on first T1 post contrast dynamic images and underwent 3D segmentation using an open source software platform 3D Slicer. A 32 × 32 patch was then extracted from the center slice of the segmented tumor data. A CNN was designed for lymph node prediction based on each of these cropped images. The CNN consisted of seven convolutional layers and max-pooling layers with 50% dropout applied in the linear layer. In addition, data augmentation and L2 regularization were performed to limit overfitting. Training was implemented using the Adam optimizer, an algorithm for first-order gradient-based optimization of stochastic objective functions, based on adaptive estimates of lower-order moments. Code for this study was written in Python using the TensorFlow module (1.0.0). Experiments and CNN training were done on a Linux workstation with NVIDIA GTX 1070 Pascal GPU. Two class axillary lymph node metastasis prediction models were evaluated. For each lymph node, a final softmax score threshold of 0.5 was used for classification. Based on this, CNN achieved a mean five-fold cross-validation accuracy of 84.3%. It is feasible for current deep CNN architectures to be trained to predict likelihood of axillary lymph node metastasis. Larger dataset will likely improve our prediction model and can potentially be a non-invasive alternative to core needle biopsy and even sentinel lymph node evaluation.

Convolutional Neural Networks for the Detection and Measurement of Cerebral Aneurysms on Magnetic Resonance Angiography

Aneurysm size correlates with rupture risk and is important for treatment planning. User annotation of aneurysm size is slow and tedious, particularly for large data sets. Geometric shortcuts to compute size have been shown to be inaccurate, particularly for nonstandard aneurysm geometries. To develop and train a convolutional neural network (CNN) to detect and measure cerebral aneurysms from magnetic resonance angiography (MRA) automatically and without geometric shortcuts. In step 1, a CNN based on the U-net architecture was trained on 250 MRA maximum intensity projection (MIP) images, then applied to a testing set. In step 2, the trained CNN was applied to a separate set of 14 basilar tip aneurysms for size prediction. Step 1-the CNN successfully identified aneurysms in 85/86 (98.8% of) testing set cases, with a receiver operating characteristic (ROC) area-under-the-curve of 0.87. Step 2-automated basilar tip aneurysm linear size differed from radiologist-traced aneurysm size on average by 2.01 mm, or 30%. The CNN aneurysm area differed from radiologist-derived area on average by 8.1 mm2 or 27%. CNN correctly predicted the area trend for the set of aneurysms. This approach is to our knowledge the first using CNNs to derive aneurysm size. In particular, we demonstrate the clinically pertinent application of computing maximal aneurysm one-dimensional size and two-dimensional area. We propose that future work can apply this to facilitate pre-treatment planning and possibly identify previously missed aneurysms in retrospective assessment.

Deep semantic lung segmentation for tracking potential pulmonary perfusion biomarkers in chronic obstructive pulmonary disease (COPD): The multi-ethnic study of atherosclerosis COPD study

BACKGROUND

Chronic obstructive pulmonary disease (COPD) is associated with high morbidity and mortality. Identification of imaging biomarkers for phenotyping is necessary for future treatment and therapy monitoring. However, translation of visual analytic pipelines into clinics or their use in large-scale studies is significantly slowed by time-consuming postprocessing steps.

PURPOSE

To implement an automated tool chain for regional quantification of pulmonary microvascular blood flow in order to reduce analysis time and user variability.

STUDY TYPE

Prospective.

POPULATION

In all, 90 MRI scans of 63 patients, of which 31 had a COPD with a mean Global Initiative for Chronic Obstructive Lung Disease status of 1.9 ± 0.64 (μ ± σ).

FIELD STRENGTH/SEQUENCE

1.5T dynamic gadolinium-enhanced MRI measurement using 4D dynamic contrast material-enhanced (DCE) time-resolved angiography acquired in a single breath-hold in inspiration. [Correction added on August 20, 2019, after first online publication: The field strength in the preceding sentence was corrected.] ASSESSMENT: We built a 3D convolutional neural network for semantic segmentation using 29 manually segmented perfusion maps. All five lobes of the lung are denoted, including the middle lobe. Evaluation was performed on 61 independent cases from two sites of the Multi-Ethnic Study of Arteriosclerosis (MESA)-COPD study. We publish our implementation of a model-free deconvolution filter according to Sourbron et al for 4D DCE MRI scans as open source.

STATISTICAL TEST

Cross-validation 29/61 (# training / # testing), intraclass correlation coefficient (ICC), Spearman ρ, Pearson r, Sørensen-Dice coefficient, and overlap.

RESULTS

Segmentations and derived clinical parameters were processed in ~90 seconds per case on a Xeon E5-2637v4 workstation with Tesla P40 GPUs. Clinical parameters and predicted segmentations exhibit high concordance with the ground truth regarding median perfusion for all lobes with an ICC of 0.99 and a Sørensen-Dice coefficient of 93.4 ± 2.8 (μ ± σ).

DATA CONCLUSION

We present a robust end-to-end pipeline that allows for the extraction of perfusion-based biomarkers for all lung lobes in 4D DCE MRI scans by combining model-free deconvolution with deep learning.

LEVEL OF EVIDENCE

3 Technical Efficacy: Stage 2 J. Magn. Reson. Imaging 2020;51:571-579.

Convolutional Neural Network Based Breast Cancer Risk Stratification Using a Mammographic Dataset

RATIONALE AND OBJECTIVES

We propose a novel convolutional neural network derived pixel-wise breast cancer risk model using mammographic dataset.

MATERIALS AND METHODS

An institutional review board approved retrospective case-control study of 1474 mammographic images was performed in average risk women. First, 210 patients with new incidence of breast cancer were identified. Mammograms from these patients prior to developing breast cancer were identified and made up the case group [420 bilateral craniocaudal mammograms]. The control group consisted of 527 patients without breast cancer from the same time period. Prior mammograms from these patients made up the control group [1054 bilateral craniocaudal mammograms]. A convolutional neural network (CNN) architecture was designed for pixel-wise breast cancer risk prediction. Briefly, each mammogram was normalized as a map of z-scores and resized to an input image size of 256 × 256. Then a contracting and expanding fully convolutional CNN architecture was composed entirely of 3 × 3 convolutions, a total of four strided convolutions instead of pooling layers, and symmetric residual connections. L2 regularization and augmentation methods were implemented to prevent overfitting. Cases were separated into training (80%) and test sets (20%). A 5-fold cross validation was performed. Software code was written in Python using the TensorFlow module on a Linux workstation with NVIDIA GTX 1070 Pascal GPU.

RESULTS

The average age of patients between the case and the control groups was not statistically different [case: 57.4years (SD, 10.4) and control: 58.2years (SD, 10.9), p = 0.33]. Breast Density (BD) was significantly higher in the case group [2.39 (SD, 0.7)] than the control group [1.98 (SD, 0.75), p < 0.0001]. On multivariate logistic regression analysis, both CNN pixel-wise mammographic risk model and BD were significant independent predictors of breast cancer risk (p < 0.0001). The CNN risk model showed greater predictive potential [OR = 4.42 (95% CI, 3.4-5.7] compared to BD [OR = 1.67 (95% CI, 1.4-1.9). The CNN risk model achieved an overall accuracy of 72% (95%CI, 69.8-74.4) in predicting patients in the case group.

CONCLUSION

Novel pixel-wise mammographic breast evaluation using a CNN architecture can stratify breast cancer risk, independent of the BD. Larger dataset will likely improve our model.

Quantitative imaging biomarkers alliance (QIBA) recommendations for improved precision of DWI and DCE-MRI derived biomarkers in multicenter oncology trials

Physiological properties of tumors can be measured both in vivo and noninvasively by diffusion-weighted imaging and dynamic contrast-enhanced magnetic resonance imaging. Although these techniques have been used for more than two decades to study tumor diffusion, perfusion, and/or permeability, the methods and studies on how to reduce measurement error and bias in the derived imaging metrics is still lacking in the literature. This is of paramount importance because the objective is to translate these quantitative imaging biomarkers (QIBs) into clinical trials, and ultimately in clinical practice. Standardization of the image acquisition using appropriate phantoms is the first step from a technical performance standpoint. The next step is to assess whether the imaging metrics have clinical value and meet the requirements for being a QIB as defined by the Radiological Society of North America's Quantitative Imaging Biomarkers Alliance (QIBA). The goal and mission of QIBA and the National Cancer Institute Quantitative Imaging Network (QIN) initiatives are to provide technical performance standards (QIBA profiles) and QIN tools for producing reliable QIBs for use in the clinical imaging community. Some of QIBA's development of quantitative diffusion-weighted imaging and dynamic contrast-enhanced QIB profiles has been hampered by the lack of literature for repeatability and reproducibility of the derived QIBs. The available research on this topic is scant and is not in sync with improvements or upgrades in MRI technology over the years. This review focuses on the need for QIBs in oncology applications and emphasizes the importance of the assessment of their reproducibility and repeatability. Level of Evidence: 5 Technical Efficacy Stage: 1 J. Magn. Reson. Imaging 2019;49:e101-e121.

Convolutional Neural Network Using a Breast MRI Tumor Dataset Can Predict Oncotype Dx Recurrence Score

BACKGROUND

Oncotype Dx is a validated genetic analysis that provides a recurrence score (RS) to quantitatively predict outcomes in patients who meet the criteria of estrogen receptor positive / human epidermal growth factor receptor-2 negative (ER+/HER2-)/node negative invasive breast carcinoma. Although effective, the test is invasive and expensive, which has motivated this investigation to determine the potential role of radiomics.

HYPOTHESIS

We hypothesized that convolutional neural network (CNN) can be used to predict Oncotype Dx RS using an MRI dataset.

STUDY TYPE

Institutional Review Board (IRB)-approved retrospective study from January 2010 to June 2016.

POPULATION

In all, 134 patients with ER+/HER2- invasive ductal carcinoma who underwent both breast MRI and Oncotype Dx RS evaluation. Patients were classified into three groups: low risk (group 1, RS <18), intermediate risk (group 2, RS 18-30), and high risk (group 3, RS >30).

FIELD STRENGTH/SEQUENCE

1.5T and 3.0T. Breast MRI, T₁ postcontrast.

ASSESSMENT

Each breast tumor underwent 3D segmentation. In all, 1649 volumetric slices in 134 tumors (mean 12.3 slices/tumor) were evaluated. A CNN consisted of four convolutional layers and max-pooling layers. Dropout at 50% was applied to the second to last fully connected layer to prevent overfitting. Three-class prediction (group 1 vs. group 2 vs. group 3) and two-class prediction (group 1 vs. group 2/3) models were performed.

STATISTICAL TESTS

A 5-fold crossvalidation test was performed using 80% training and 20% testing. Diagnostic accuracy, sensitivity, specificity, and receiver operating characteristic (ROC) area under the curve (AUC) were evaluated.

RESULTS

The CNN achieved an overall accuracy of 81% (95% confidence interval [CI] ± 4%) in three-class prediction with specificity 90% (95% CI ± 5%), sensitivity 60% (95% CI ± 6%), and the area under the ROC curve was 0.92 (SD, 0.01). The CNN achieved an overall accuracy of 84% (95% CI ± 5%) in two-class prediction with specificity 81% (95% CI ± 4%), sensitivity 87% (95% CI ± 5%), and the area under the ROC curve was 0.92 (SD, 0.01).

DATA CONCLUSION

It is feasible for current deep CNN architecture to be trained to predict Oncotype DX RS.

LEVEL OF EVIDENCE

4 Technical Efficacy: Stage 2 J. Magn. Reson. Imaging 2019;49:518-524.

Predicting Post Neoadjuvant Axillary Response Using a Novel Convolutional Neural Network Algorithm

OBJECTIVES

In the postneoadjuvant chemotherapy (NAC) setting, conventional radiographic complete response (rCR) is a poor predictor of pathologic complete response (pCR) of the axilla. We developed a convolutional neural network (CNN) algorithm to better predict post-NAC axillary response using a breast MRI dataset.

METHODS

An institutional review board-approved retrospective study from January 2009 to June 2016 identified 127 breast cancer patients who: (1) underwent breast MRI before the initiation of NAC; (2) successfully completed Adriamycin/Taxane-based NAC; and (3) underwent surgery, including sentinel lymph node evaluation/axillary lymph node dissection with final surgical pathology data. Patients were classified into pathologic complete response (pCR) of the axilla group and non-pCR group based on surgical pathology. Breast MRI performed before NAC was used. Tumor was identified on first T1 postcontrast images underwent 3D segmentation. A total of 2811 volumetric slices of 127 tumors were evaluated. CNN consisted of 10 convolutional layers, 4 max-pooling layers. Dropout, augmentation and L2 regularization were implemented to prevent overfitting of data.

RESULTS

On final surgical pathology, 38.6% (49/127) of the patients achieved pCR of the axilla (group 1), and 61.4% (78/127) of the patients did not with residual metastasis detected (group 2). For predicting axillary pCR, our CNN algorithm achieved an overall accuracy of 83% (95% confidence interval [CI] ± 5) with sensitivity of 93% (95% CI ± 6) and specificity of 77% (95% CI ± 4). Area under the ROC curve (0.93, 95% CI ± 0.04).

CONCLUSIONS

It is feasible to use CNN architecture to predict post NAC axillary pCR. Larger data set will likely improve our prediction model.

Calibration and error analysis of metal-oxide-semiconductor field-effect transistor dosimeters for computed tomography radiation dosimetry

PURPOSE

Metal-oxide-semiconductor field-effect transistors (MOSFETs) serve as a helpful tool for organ radiation dosimetry and their use has grown in computed tomography (CT). While different approaches have been used for MOSFET calibration, those using the commonly available 100 mm pencil ionization chamber have not incorporated measurements performed throughout its length, and moreover, no previous work has rigorously evaluated the multiple sources of error involved in MOSFET calibration. In this paper, we propose a new MOSFET calibration approach to translate MOSFET voltage measurements into absorbed dose from CT, based on serial measurements performed throughout the length of a 100-mm ionization chamber, and perform an analysis of the errors of MOSFET voltage measurements and four sources of error in calibration.

METHODS

MOSFET calibration was performed at two sites, to determine single calibration factors for tube potentials of 80, 100, and 120 kVp, using a 100-mm-long pencil ion chamber and a cylindrical computed tomography dose index (CTDI) phantom of 32 cm diameter. The dose profile along the 100-mm ion chamber axis was sampled in 5 mm intervals by nine MOSFETs in the nine holes of the CTDI phantom. Variance of the absorbed dose was modeled as a sum of the MOSFET voltage measurement variance and the calibration factor variance, the latter being comprised of three main subcomponents: ionization chamber reading variance, MOSFET-to-MOSFET variation and a contribution related to the fact that the average calibration factor of a few MOSFETs was used as an estimate for the average value of all MOSFETs. MOSFET voltage measurement error was estimated based on sets of repeated measurements. The calibration factor overall voltage measurement error was calculated from the above analysis.

RESULTS

Calibration factors determined were close to those reported in the literature and by the manufacturer (~3 mV/mGy), ranging from 2.87 to 3.13 mV/mGy. The error σ_V of a MOSFET voltage measurement was shown to be proportional to the square root of the voltage V: σV=cV where c = 0.11 mV. A main contributor to the error in the calibration factor was the ionization chamber reading error with 5% error. The usage of a single calibration factor for all MOSFETs introduced an additional error of about 5-7%, depending on the number of MOSFETs that were used to determine the single calibration factor. The expected overall error in a high-dose region (~30 mGy) was estimated to be about 8%, compared to 6% when an individual MOSFET calibration was performed. For a low-dose region (~3 mGy), these values were 13% and 12%.

CONCLUSIONS

A MOSFET calibration method was developed using a 100-mm pencil ion chamber and a CTDI phantom, accompanied by an absorbed dose error analysis reflecting multiple sources of measurement error. When using a single calibration factor, per tube potential, for different MOSFETs, only a small error was introduced into absorbed dose determinations, thus supporting the use of a single calibration factor for experiments involving many MOSFETs, such as those required to accurately estimate radiation effective dose.

Can diffusion-weighted imaging serve as a biomarker of fibrosis in pancreatic adenocarcinoma?

PURPOSE

To assess the relationship between diffusion-weighted imaging (DWI) and intravoxel incoherent motion (IVIM)-derived quantitative parameters (apparent diffusion coefficient [ADC], perfusion fraction [f], Dslow , diffusion coefficient [D], and Dfast , pseudodiffusion coefficient [D*]) and histopathology in pancreatic adenocarcinoma (PAC).

MATERIALS AND METHODS

Subjects with suspected surgically resectable PAC were prospectively enrolled in this Health Insurance Portability and Accountability Act (HIPAA)-compliant, Institutional Review Board-approved study. Imaging was performed at 1.5T with a respiratory-triggered echo planar DWI sequence using 10 b values. Two readers drew regions of interest (ROIs) over the tumor and adjacent nontumoral tissue. Monoexponential and biexponential fits were used to derive ADC2b , ADCall , f, D, and D*, which were compared to quantitative histopathology of fibrosis, mean vascular density, and cellularity. Two biexponential IVIM models were investigated and compared: 1) nonlinear least-square fitting based on the Levenberg-Marquardt algorithm, and 2) linear fit using a fixed D* (20 mm2 /s). Statistical analysis included Student's t-test, Pearson correlation (P < 0.05 was considered significant), intraclass correlation, and coefficients of variance.

RESULTS

Twenty subjects with PAC were included in the final cohort. Negative correlation between D and fibrosis (Reader 2: r = -0.57 P = 0.01; pooled P = -0.46, P = 0.04) was observed with a trend toward positive correlation between f and fibrosis (r = 0.44, P = 0.05). ADC2b was significantly lower in PAC with dense fibrosis than with loose fibrosis ADC2b (P = 0.03). Inter- and intrareader agreement was excellent for ADC, D, and f.

CONCLUSION

In PAC, D negatively correlates with fibrosis, with a trend toward positive correlation with f suggesting both perfusion and diffusion effects contribute to stromal desmoplasia. ADC2b is significantly lower in tumors with dense fibrosis and may serve as a biomarker of fibrosis architecture.

LEVEL OF EVIDENCE

1 Technical Efficacy: Stage 2 J. MAGN. RESON. IMAGING 2017;46:393-402.

Estimating Effective Dose of Radiation From Pediatric Cardiac CT Angiography Using a 64-MDCT Scanner: New Conversion Factors Relating Dose-Length Product to Effective Dose

OBJECTIVE

The purpose of this study is to determine the conversion factors that enable accurate estimation of the effective dose (ED) used for cardiac 64-MDCT angiography performed for children.

MATERIALS AND METHODS

Anthropomorphic phantoms representative of 1- and 10-year-old children, with 50 metal oxide semiconductor field-effect transistor dosimeters placed in organs, underwent scanning performed using a 64-MDCT scanner with different routine clinical cardiac scan modes and x-ray tube potentials. Organ doses were used to calculate the ED on the basis of weighting factors published in 1991 in International Commission on Radiological Protection (ICRP) publication 60 and in 2007 in ICRP publication 103. The EDs and the scanner-reported dose-length products were used to determine conversion factors for each scan mode. The effect of infant heart rate on the ED and the conversion factors was also assessed.

RESULTS

The mean conversion factors calculated using the current definition of ED that appeared in ICRP publication 103 were as follows: 0.099 mSv · mGy-1 · cm-1, for the 1-year-old phantom, and 0.049 mSv · mGy-1 · cm-1, for the 10-year-old phantom. These conversion factors were a mean of 37% higher than the corresponding conversion factors calculated using the older definition of ED that appeared in ICRP publication 60. Varying the heart rate did not influence the ED or the conversion factors.

CONCLUSION

Conversion factors determined using the definition of ED in ICRP publication 103 and cardiac, rather than chest, scan coverage suggest that the radiation doses that children receive from cardiac CT performed using a contemporary 64-MDCT scanner are higher than the radiation doses previously reported when older chest conversion factors were used. Additional up-to-date pediatric cardiac CT conversion factors are required for use with other contemporary CT scanners and patients of different age ranges.

Pharmacokinetic analysis and drug delivery efficiency of the focused ultrasound-induced blood-brain barrier opening in non-human primates

PURPOSE

Focused Ultrasound (FUS) in conjunction with systemically administered microbubbles has been shown to open the Blood-Brain Barrier (BBB) locally, non-invasively and reversibly in rodents and non-human primates (NHP), suggesting the immense potential of this technique. The objective of this study entailed the investigation of the physiologic changes in the brain following the FUS-induced BBB opening and their relationship with the underlying anatomy.

MATERIALS AND METHODS

Pharmacokinetic analysis was implemented in NHP's that received FUS at various acoustic pressures. Relaxivity mapping enabled the robust quantitative detection of the BBB opening as well as grey and white matter segmentation. Drug delivery efficiency was measured for pre-clinical validation of the technique.

RESULTS

Based on our results, the opening volume and the amount of the gadolinium delivered were found mostly contained in the grey matter, while FUS-induced permeability and drug concentration varied depending upon the underlying brain inhomogeneity, and increased with the acoustic pressure.

CONCLUSIONS

Overall, apart from the in vivo protocols for BBB analysis developed here, this study also suggests the important role that FUS can have in efficient drug delivery via localized and transient BBB opening.

Investigating the mechanical function of the cervix during pregnancy using finite element models derived from high-resolution 3D MRI

Preterm birth is a strong contributor to perinatal mortality, and preterm infants that survive are at risk for long-term morbidities. During most of pregnancy, appropriate mechanical function of the cervix is required to maintain the developing fetus in utero. Premature cervical softening and subsequent cervical shortening are hypothesized to cause preterm birth. Presently, there is a lack of understanding of the structural and material factors that influence the mechanical function of the cervix during pregnancy. In this study we build finite element models of the pregnant uterus, cervix, and fetal membrane based on magnetic resonance imagining data in order to examine the mechanical function of the cervix under the physiologic loading conditions of pregnancy. We calculate the mechanical loading state of the cervix for two pregnant patients: 22 weeks gestational age with a normal cervical length and 28 weeks with a short cervix. We investigate the influence of (1) anatomical geometry, (2) cervical material properties, and (3) fetal membrane material properties, including its adhesion properties, on the mechanical loading state of the cervix under physiologically relevant intrauterine pressures. Our study demonstrates that membrane-uterus interaction, cervical material modeling, and membrane mechanical properties are factors that must be deliberately and carefully handled in order to construct a high quality mechanical simulation of pregnancy.

Diffusion tensor imaging of peripheral nerves

Magnetic resonance diffusion tensor imaging (DTI) allows the directional dependence of water diffusion to be studied. Analysis of the resulting image data allows for the determination of fractional anisotropy (FA), apparent diffusion coefficient (ADC), as well as allowing three-dimensional visualization of the fiber tract (tractography). We visualized the ulnar nerve of ten healthy volunteers with DTI. We found FA to be 0.752 ± 0.067 and the ADC to be 0.96 ± 0.13 × 10(-3) mm(2)/s. A nuts-and-bolts description of the physical aspects of DTI is provided as an educational process for readers.

Dynamic infrared imaging for the detection of malignancy

The potential for malignancy detection using dynamic infrared imaging (DIRI) has been investigated in an animal model of human malignancy. Malignancy was apparent in images formed at the vasomotor and cardiogenic frequencies of tumour bearing mice. The observation of malignancy was removed by the administration of an agent that blocks vasodilation caused by nitric oxide (NO). Image patterns similar to those that characterize malignancy could be mimicked in normal mice using an NO producing agent. Apparently DIRI allows for cancer detection in this model through vasodilation caused by malignancy generated NO. Dynamic infrared detection of vasomotor and cardiogenic surface perfusion was validated in human subjects by a comparison with laser Doppler flowmetry (LDF). Dynamic infrared imaging technology was then applied to breast cancer detection. It is shown that dynamic infrared images formed at the vasomotor and cardiogenic frequencies of the normal and malignant breast have image pattern differences, which may allow for breast cancer detection.