Deep learning-based harmonization of trabecular bone microstructures between high- and low-resolution CT imaging

BACKGROUND

Osteoporosis is a bone disease related to increased bone loss and fracture-risk. The variability in bone strength is partially explained by bone mineral density (BMD), and the remainder is contributed by bone microstructure. Recently, clinical CT has emerged as a viable option for in vivo bone microstructural imaging. Wide variations in spatial-resolution and other imaging features among different CT scanners add inconsistency to derived bone microstructural metrics, urging the need for harmonization of image data from different scanners.

PURPOSE

This paper presents a new deep learning (DL) method for the harmonization of bone microstructural images derived from low- and high-resolution CT scanners and evaluates the method's performance at the levels of image data as well as derived microstructural metrics.

METHODS

We generalized a three-dimensional (3D) version of GAN-CIRCLE that applies two generative adversarial networks (GANs) constrained by the identical, residual, and cycle learning ensemble (CIRCLE). Two GAN modules simultaneously learn to map low-resolution CT (LRCT) to high-resolution CT (HRCT) and vice versa. Twenty volunteers were recruited. LRCT and HRCT scans of the distal tibia of their left legs were acquired. Five-hundred pairs of LRCT and HRCT image blocks of64 × 64 × 64 $64 \times 64 \times 64 $ voxels were sampled for each of the twelve volunteers and used for training in supervised as well as unsupervised setups. LRCT and HRCT images of the remaining eight volunteers were used for evaluation. LRCT blocks were sampled at 32 voxel intervals in each coordinate direction and predicted HRCT blocks were stitched to generate a predicted HRCT image.

RESULTS

Mean ± standard deviation of structural similarity (SSIM) values between predicted and true HRCT using both 3DGAN-CIRCLE-based supervised (0.84 ± 0.03) and unsupervised (0.83 ± 0.04) methods were significantly (p < 0.001) higher than the mean SSIM value between LRCT and true HRCT (0.75 ± 0.03). All Tb measures derived from predicted HRCT by the supervised 3DGAN-CIRCLE showed higher agreement (CCC  ∈ $ \in $ [0.956 0.991]) with the reference values from true HRCT as compared to LRCT-derived values (CCC  ∈ $ \in $ [0.732 0.989]). For all Tb measures, except Tb plate-width (CCC = 0.866), the unsupervised 3DGAN-CIRCLE showed high agreement (CCC  ∈ $ \in $ [0.920 0.964]) with the true HRCT-derived reference measures. Moreover, Bland-Altman plots showed that supervised 3DGAN-CIRCLE predicted HRCT reduces bias and variability in residual values of different Tb measures as compared to LRCT and unsupervised 3DGAN-CIRCLE predicted HRCT. The supervised 3DGAN-CIRCLE method produced significantly improved performance (p < 0.001) for all Tb measures as compared to the two DL-based supervised methods available in the literature.

CONCLUSIONS

3DGAN-CIRCLE, trained in either unsupervised or supervised fashion, generates HRCT images with high structural similarity to the reference true HRCT images. The supervised 3DGAN-CIRCLE improves agreements of computed Tb microstructural measures with their reference values and outperforms the unsupervised 3DGAN-CIRCLE. 3DGAN-CIRCLE offers a viable DL solution to retrospectively improve image resolution, which may aid in data harmonization in multi-site longitudinal studies where scanner mismatch is unavoidable.

Synergistic multi-spectral CT reconstruction with directional total variation

This work considers synergistic multi-spectral CT reconstruction where information from all available energy channels is combined to improve the reconstruction of each individual channel. We propose to fuse these available data (represented by a single sinogram) to obtain a polyenergetic image which keeps structural information shared by the energy channels with increased signal-to-noise ratio. This new image is used as prior information during a channel-by-channel minimization process through the directional total variation. We analyse the use of directional total variation within variational regularization and iterative regularization. Our numerical results on simulated and experimental data show improvements in terms of image quality and in computational speed. This article is part of the theme issue 'Synergistic tomographic image reconstruction: part 2'.

Enhanced spatial resolution for snapshot hyperspectral imaging of blood perfusion and melanin information within human tissue

We report a reconstruction method to achieve high spatial resolution for hyperspectral imaging of chromophore features in skin in vivo. The method utilizes an established structure-adaptive normalized convolution algorithm to reconstruct high spatial resolution of hyperspectral images from snapshot low-resolution hyperspectral image sequences captured by a snapshot spectral camera. The reconstructed images at chromophore-sensitive wavebands are used to map the skin features of interest. We demonstrate the method experimentally by mapping the blood perfusion and melanin features (moles) on the facial skin. The method relaxes the constrains of the relatively low spatial resolution in the snapshot hyperspectral camera, making it more usable in imaging applications.

Deep Learning Based High-Resolution Reconstruction of Trabecular Bone Microstructures from Low-Resolution CT Scans using GAN-CIRCLE

Osteoporosis is a common age-related disease characterized by reduced bone density and increased fracture-risk. Microstructural quality of trabecular bone (Tb), commonly found at axial skeletal sites and at the end of long bones, is an important determinant of bone-strength and fracture-risk. High-resolution emerging CT scanners enable in vivo measurement of Tb microstructures at peripheral sites. However, resolution-dependence of microstructural measures and wide resolution-discrepancies among various CT scanners together with rapid upgrades in technology warrant data harmonization in CT-based cross-sectional and longitudinal bone studies. This paper presents a deep learning-based method for high-resolution reconstruction of Tb microstructures from low-resolution CT scans using GAN-CIRCLE. A network was developed and evaluated using post-registered ankle CT scans of nineteen volunteers on both low- and high-resolution CT scanners. 9,000 matching pairs of low- and high-resolution patches of size 64×64 were randomly harvested from ten volunteers for training and validation. Another 5,000 matching pairs of patches from nine other volunteers were used for evaluation. Quantitative comparison shows that predicted high-resolution scans have significantly improved structural similarity index (p < 0.01) with true high-resolution scans as compared to the same metric for low-resolution data. Different Tb microstructural measures such as thickness, spacing, and network area density are also computed from low- and predicted high-resolution images, and compared with the values derived from true high-resolution scans. Thickness and network area measures from predicted images showed higher agreement with true high-resolution CT (CCC = [0.95, 0.91]) derived values than the same measures from low-resolution images (CCC = [0.72, 0.88]).

Detecting Pulmonary Nodules in Lung Cancer Patients Using Whole Body FDG PET/CT, High-resolution Lung Reformat of FDG PET/CT, or Diagnostic Breath Hold Chest CT

RATIONALE AND OBJECTIVES

Pulmonary nodules can be missed on the non-breath hold computed tomography (CT) portion of 18F-fluorodeoxyglucose positron emission tomography/computed tomography (FDG PET/CT), and for this reason prior studies have advocated for routinely performing dedicated breath hold CT of the chest in addition to PET/CT for routine staging of malignancy. We evaluated the rate of pulmonary nodule detection on standard CT images from whole body PET/CT studies (WB-PET/CT), high-resolution lung reconstruction CT images from PET/CT studies (HR-PET/CT), and diagnostic breath hold chest CT (BH-CT).

MATERIALS AND METHODS

A cohort of 25 patients was identified who had a history of lung cancer as well as a PET/CT staging or restaging scan and BH-CT within 30 days of each other. All PET/CTs included a set of CT images using a soft tissue algorithm filter and 3.75- to 5-mm slice thickness, as well as high-resolution reformats with a sharp reconstruction filter and 2-mm slice thickness. The CT images from WB-PET/CT, HR-PET/CT, and BH-CT were reviewed by three radiologists. Significance was analyzed by two-way repeated measures analysis of variance.

RESULTS

There were 2.84 nodules found per patient with WB-PET/CT, 3.85 nodules with HR-PET/CT, and 3.91 nodules with BH-CT. When only nodules less than or equal to 8 mm in size were considered, WB-PET/CT also demonstrated significantly fewer nodules (1.98) compared to the HR-PET/CT (2.94) or a BH-CT (2.86) (P < 0.001). No difference in detection rate was noted between the two higher resolution modalities.

CONCLUSIONS

More pulmonary nodules are detected on the CT portion of PET/CT studies when high-resolution reformatted images are created and reviewed. The ability to detect nodules with the reformatted images was indistinguishable from dedicated BH-CT. Overall, high-resolution reformats of PET/CT images of the lungs can increase the sensitivity for pulmonary nodule detection, approaching that of dedicated BH-CT. These data suggest that if HR-PET/CT reformats are used, additional dedicated BH-CT is unnecessary for routine staging of lung cancer.