Vision 20/20: Simultaneous CT-MRI--Next chapter of multimodality imaging

Achieving accurate prostate auto-segmentation on CT in the absence of MR imaging

BACKGROUND

Magnetic resonance imaging (MRI) is considered the gold standard for prostate segmentation. Computed tomography (CT)-based segmentation is prone to observer bias, potentially overestimating the prostate volume by ∼ 30 % compared to MRI. However, MRI accessibility is challenging for patients with contraindications or in rural areas globally with limited clinical resources.

PURPOSE

This study investigates the possibility of achieving MRI-level prostate auto-segmentation accuracy using CT-only input via a deep learning (DL) model trained with CT-MRI registered segmentation.

METHODS AND MATERIALS

A cohort of 111 definitive prostate radiotherapy patients with both CT and MRI images was retrospectively grouped into training (n = 37) and validation (n = 20) (where reference contours were derived from CT-MRI registration), and testing (n = 54) sets. Two commercial DL models were benchmarked against the reference contours in the training and validation sets. A custom DL model was incrementally retrained using the training dataset, quantitatively evaluated on the validation dataset, and qualitatively assessed by two different physician groups on the validation and testing datasets. A contour quality assurance (QA) model, established from the proposed model on the validation dataset, was applied to the test group to identify potential errors, confirmed by human visual inspection.

RESULTS

Two commercial models exhibited large deviations in the prostate apex with CT-only input (median: 0.77/0.78 for Dice similarity coefficient (DSC), and 0.80 cm/0.83 cm for 95 % directed Hausdorff Distance (HD95), respectively). The proposed model demonstrated superior geometric similarity compared to commercial models, particularly in the apex region, with improvements of 0.05/0.17 cm and 0.06/0.25 cm in median DSC/HD95, respectively. Physician evaluation on MRI-CT registration data rated 69 %-78 % of the proposed model's contours as clinically acceptable without modifications. Additionally, 73 % of cases flagged by the contour quality assurance (QA) model were confirmed via visual inspection.

CONCLUSIONS

The proposed incremental learning strategy based on CT-MRI registration information enhances prostate segmentation accuracy when MRI availability is limited clinically.

Multi-Sensor Learning Enables Information Transfer Across Different Sensory Data and Augments Multi-Modality Imaging

Multi-modality imaging is widely used in clinical practice and biomedical research to gain a comprehensive understanding of an imaging subject. Currently, multi-modality imaging is accomplished by post hoc fusion of independently reconstructed images under the guidance of mutual information or spatially registered hardware, which limits the accuracy and utility of multi-modality imaging. Here, we investigate a data-driven multi-modality imaging (DMI) strategy for synergetic imaging of CT and MRI. We reveal two distinct types of features in multi-modality imaging, namely intra- and inter-modality features, and present a multi-sensor learning (MSL) framework to utilize the crossover inter-modality features for augmented multi-modality imaging. The MSL imaging approach breaks down the boundaries of traditional imaging modalities and allows for optimal hybridization of CT and MRI, which maximizes the use of sensory data. We showcase the effectiveness of our DMI strategy through synergetic CT-MRI brain imaging. The principle of DMI is quite general and holds enormous potential for various DMI applications across disciplines.

Near Lossless Compression for 3D Radiological Images Using Optimal Multilinear Singular Value Decomposition (3D-VOI-OMLSVD)

Storage and transmission of high-compression 3D radiological images that create high-quality reconstruction upon decompression are critical necessities for effective and efficient teleradiology. To cater to this need, we propose a near lossless 3D image volume compression method based on optimal multilinear singular value decomposition called "3D-VOI-OMLSVD." The proposed strategy first eliminates any blank 2D image slices from the 3D image volume and uses the selective bounding volume (SBV) to identify and extract the volume of Interest (VOI). Following this, the VOI is decomposed with an optimal multilinear singular value decomposition (OMLSVD) to obtain the corresponding core tensor, factor matrices, and singular values that are compressed with adaptive binary range coder (ABRC), integrated as an entropy encoder. The compressed file can be transferred or transmitted and then decompressed in order to reconstruct the original image. The resultant decompressed VOI is acquired by reversing the above process and then fusing it with the background, using the bound volume coordinates associated with the compressed 3D image. The proposed method performance was tested on a variety of 3D radiological images with different imaging modalities and dimensions using quantitative evaluation metrics such as the compression rate (CR), bit rate (BR), peak signal to noise ratio (PSNR), and structural similarity index (SSIM). Furthermore, we also investigate the impact of VOI extraction on the model performance, before comparing it with two popular compression methods, namely JPEG and JPEG2000. Our proposed method, 3D-VOI-OMLSVD, displayed a high CR value, with a maximum of 37.31, and a low BR, with the lowest reported to be 0.21. The SSIM score was consistently high, with an average performance of 0.9868, while using < 1 second for decoding the image. We observe that with VOI extraction, the compression rate increases manifold, and bit rate drops significantly, and thus reduces the encoding and decoding time to a great extent. Compared to JPEG and JPEG2000, our method consistently performs better in terms of higher CR and lower BR. The results indicate that the proposed compression methodology performs consistently to create high-quality image compressions, and overall gives a better outcome when compared against two state-of-the-art and widely used methods, JPEG and JPEG2000.

Evaluation of beam hardening artifacts around dental implants: CT study on bovine ribs

Background/Aim: The aim of this study was to evaluate beam hardening artifacts generated by Grade 4 and Grade 5 dental implants on computed tomography (CT) images at low and high kilovoltage peaks (kVp). Material and Methods: A total of 16 implants, 8 of which were Grade 4 and 8 were Grade 5, were inserted into bovine ribs. CT images of bovine ribs were acquired using two different exposure protocol: low kVp and high kVp. Beam hardening artifacts generated by Grade 4 and Grade 5 dental implants were calculated by the mean Hounsfield unit (HU) within a standardized region-of-interest (ROI). Results: Artifact in Grade 4 implants were greater than that in Grade 5 implants. Also, artifacts at the high kVp were lower than that at the low kVp. Conclusions: CT scans providing HU values can be used to evaluate the beam hardening artifact. Beam hardening artifacts decreased in the CT images with high kVp. Grade 5 dental implants have an advantage by producing less severe beam hardening artifacts.

Ultrafast Fabrication of Iron/Manganese Co-Doped Bismuth Trimetallic Nanoparticles: A Thermally Aided Chemodynamic/Radio-Nanoplatform for Low-Dose Radioresistance

Low-dose radioresistance continues to be one of the major limitations for clinical curative treatment of cancer. Luckily, nanotechnology mediated by multifunctional nanomaterials provides potential opportunity to relieve the radioresistance via increasing the radiosensitivity of cancer cells. Herein, an ultrafast fabrication strategy is reported to prepare iron/manganese co-doped bismuth trimetallic nanoparticles (pFMBi NPs) as a multifunctional radiosensitizer for combined therapy. The bismuth matrix provides the intrinsic radiosensitization effect under the low and safe radiation dose via Auger electrons, photoelectrons, and Rayleigh scattering. Meanwhile, co-doping of iron and manganese ions endows pFMBi NPs with both the Fenton reaction property for reactive oxygen species (ROS) generation and photothermal conversion performance for heat production. Additional ROS generation enhances the radiosensitization effect by collaborating with Rayleigh scattering-mediated water radiolysis, and endogenous heat production under near-infrared 808 nm laser irradiation makes DNA more sensitive to radiation and ROS damage. Both in vitro and in vivo evaluations demonstrate the effective antitumor and radiosensitization effects via thermally aided chemodynamic/radiotreatment with a low radiation dose (6 Gy). Therefore, this work provides a potential strategy for overcoming the low-dose radioresistance in cancer therapy.

Multi-Contrast Super-Resolution MRI Through a Progressive Network

Magnetic resonance imaging (MRI) is widely used for screening, diagnosis, image-guided therapy, and scientific research. A significant advantage of MRI over other imaging modalities such as computed tomography (CT) and nuclear imaging is that it clearly shows soft tissues in multi-contrasts. Compared with other medical image super-resolution methods that are in a single contrast, multi-contrast super-resolution studies can synergize multiple contrast images to achieve better super-resolution results. In this paper, we propose a one-level non-progressive neural network for low up-sampling multi-contrast super-resolution and a two-level progressive network for high up-sampling multi-contrast super-resolution. The proposed networks integrate multi-contrast information in a high-level feature space and optimize the imaging performance by minimizing a composite loss function, which includes mean-squared-error, adversarial loss, perceptual loss, and textural loss. Our experimental results demonstrate that 1) the proposed networks can produce MRI super-resolution images with good image quality and outperform other multi-contrast super-resolution methods in terms of structural similarity and peak signal-to-noise ratio; 2) combining multi-contrast information in a high-level feature space leads to a significantly improved result than a combination in the low-level pixel space; and 3) the progressive network produces a better super-resolution image quality than the non-progressive network, even if the original low-resolution images were highly down-sampled.

Multi-Modal Deep Guided Filtering for Comprehensible Medical Image Processing

Deep learning-based image processing is capable of creating highly appealing results. However, it is still widely considered as a "blackbox" transformation. In medical imaging, this lack of comprehensibility of the results is a sensitive issue. The integration of known operators into the deep learning environment has proven to be advantageous for the comprehensibility and reliability of the computations. Consequently, we propose the use of the locally linear guided filter in combination with a learned guidance map for general purpose medical image processing. The output images are only processed by the guided filter while the guidance map can be trained to be task-optimal in an end-to-end fashion. We investigate the performance based on two popular tasks: image super resolution and denoising. The evaluation is conducted based on pairs of multi-modal magnetic resonance imaging and cross-modal computed tomography and magnetic resonance imaging datasets. For both tasks, the proposed approach is on par with state-of-the-art approaches. Additionally, we can show that the input image's content is almost unchanged after the processing which is not the case for conventional deep learning approaches. On top, the proposed pipeline offers increased robustness against degraded input as well as adversarial attacks.

Projection-to-Projection Translation for Hybrid X-ray and Magnetic Resonance Imaging

Hybrid X-ray and magnetic resonance (MR) imaging promises large potential in interventional medical imaging applications due to the broad variety of contrast of MRI combined with fast imaging of X-ray-based modalities. To fully utilize the potential of the vast amount of existing image enhancement techniques, the corresponding information from both modalities must be present in the same domain. For image-guided interventional procedures, X-ray fluoroscopy has proven to be the modality of choice. Synthesizing one modality from another in this case is an ill-posed problem due to ambiguous signal and overlapping structures in projective geometry. To take on these challenges, we present a learning-based solution to MR to X-ray projection-to-projection translation. We propose an image generator network that focuses on high representation capacity in higher resolution layers to allow for accurate synthesis of fine details in the projection images. Additionally, a weighting scheme in the loss computation that favors high-frequency structures is proposed to focus on the important details and contours in projection imaging. The proposed extensions prove valuable in generating X-ray projection images with natural appearance. Our approach achieves a deviation from the ground truth of only 6% and structural similarity measure of 0.913 ± 0.005. In particular the high frequency weighting assists in generating projection images with sharp appearance and reduces erroneously synthesized fine details.

Simultaneous Emission-Transmission Tomography in an MRI Hardware Framework

Multi-modality imaging is essential for diagnosis and therapy in challenging cases. A Holy Grail of medical imaging is a hybrid imaging system combining computed tomography (CT), nuclear imaging, and magnetic resonance imaging (MRI) to deliver registered morphological, functional, and cellular/molecular information simultaneously and quantitatively for precision medicine. Recently, a unique imaging approach was demonstrated that combines nuclear imaging with polarized radiotracers and MRI-based spatial encoding. The detection scheme exploits the directional preference of γ-rays emitted from the polarized nuclei, and the result is a concentration image with resolution that can outperform standard nuclear imaging at a sensitivity significantly higher than that of MRI. However, the method does not calculate the attenuation image. Here we propose to obtain MRI-modulated γ-ray data for simultaneous image reconstruction of emission and transmission parameters, which could serve as a stepping stone toward simultaneous CT-SPECT-MRI. This method acquires synchronized datasets to provide insight into morphological features and molecular activities with accurate spatiotemporal registration. We present a complete overview of the system design and the formulation for tomographic reconstruction when the distribution of polarized radiotracers is either global or limited to a region of interest (ROI). Numerical results support the feasibility of our approach and suggest further research topics.

Generalized backpropagation algorithm for training second-order neural networks

The artificial neural network is a popular framework in machine learning. To empower individual neurons, we recently suggested that the current type of neurons could be upgraded to second-order counterparts, in which the linear operation between inputs to a neuron and the associated weights is replaced with a nonlinear quadratic operation. A single second-order neurons already have a strong nonlinear modeling ability, such as implementing basic fuzzy logic operations. In this paper, we develop a general backpropagation algorithm to train the network consisting of second-order neurons. The numerical studies are performed to verify the generalized backpropagation algorithm.

Wavelet-based joint CT-MRI reconstruction

Since their inceptions, the multimodal imaging techniques have received a great deal of attention for achieving enhanced imaging performance. In this work, a novel joint reconstruction framework using sparse computed tomography (CT) and magnetic resonance imaging (MRI) data is developed and evaluated. CT and MRI images are synchronously acquired and registered from a hybrid CT-MRI platform. Because image data are highly undersampled, analytic methods are unable to generate decent image quality. To overcome this drawback, we resort to the compressed sensing (CS) techniques, which employ sparse priors that result from an application of a wavelet transform. To utilize multimodal information, projection distance is introduced and is tuned to tailor the texture and pattern of the final images. Specifically, CT and MRI images are alternately reconstructed using the updated multimodal results that are calculated at the latest step of the iterative optimization algorithm. The method exploits the structural similarities shared by the CT and MRI images to achieve better reconstruction quality. The good performance of the proposed approach is demonstrated on a pair of undersampled CT and MRI body images. Clinical CT and MRI images are tested with the joint reconstruction, the analytic reconstruction, and the independent reconstruction which does not uses multimodal imaging information. Results show that the proposed method improves about 5dB in signal to noise ratio (SNR) and nearly 10% in structural similarity measure comparing to independent reconstruction. It offers similar quality with fully sampled analytic reconstruction with only 20% sampling rate for CT and 40% for MRI. Structural similarities and correlations residing in images from different modalities are useful to mutually promote the quality of image reconstruction.

In vivo pentamodal tomographic imaging for small animals

Multimodality molecular imaging emerges as a powerful strategy for correlating multimodal information. We developed a pentamodal imaging system which can perform positron emission tomography, bioluminescence tomography, fluorescence molecular tomography, Cerenkov luminescence tomography and X-ray computed tomography successively. Performance of sub-systems corresponding to different modalities were characterized. In vivo multimodal imaging of an orthotopic hepatocellular carcinoma xenograft mouse model was performed, and acquired multimodal images were fused. The feasibility of pentamodal tomographic imaging system was successfully validated with the imaging application on the mouse model. The ability of integrating anatomical, metabolic, and pharmacokinetic information promises applications of multimodality molecular imaging in precise medicine.

High-kVp Assisted Metal Artifact Reduction for X-ray Computed Tomography

In X-ray computed tomography (CT), the presence of metallic parts in patients causes serious artifacts and degrades image quality. Many algorithms were published for metal artifact reduction (MAR) over the past decades with various degrees of success but without a perfect solution. Some MAR algorithms are based on the assumption that metal artifacts are due only to strong beam hardening and may fail in the case of serious photon starvation. Iterative methods handle photon starvation by discarding or underweighting corrupted data, but the results are not always stable and they come with high computational cost. In this paper, we propose a high-kVp-assisted CT scan mode combining a standard CT scan with a few projection views at a high-kVp value to obtain critical projection information near the metal parts. This method only requires minor hardware modifications on a modern CT scanner. Two MAR algorithms are proposed: dual-energy normalized MAR (DNMAR) and high-energy embedded MAR (HEMAR), aiming at situations without and with photon starvation respectively. Simulation results obtained with the CT simulator CatSim demonstrate that the proposed DNMAR and HEMAR methods can eliminate metal artifacts effectively.