Multimodality image registration with software: state-of-the-art

Comparative Analysis of Color Space and Channel, Detector, and Descriptor for Feature-Based Image Registration

The current study aimed to quantify the value of color spaces and channels as a potential superior replacement for standard grayscale images, as well as the relative performance of open-source detectors and descriptors for general feature-based image registration purposes, based on a large benchmark dataset. The public dataset UDIS-D, with 1106 diverse image pairs, was selected. In total, 21 color spaces or channels including RGB, XYZ, Y'CrCb, HLS, L*a*b* and their corresponding channels in addition to grayscale, nine feature detectors including AKAZE, BRISK, CSE, FAST, HL, KAZE, ORB, SIFT, and TBMR, and 11 feature descriptors including AKAZE, BB, BRIEF, BRISK, DAISY, FREAK, KAZE, LATCH, ORB, SIFT, and VGG were evaluated according to reprojection error (RE), root mean square error (RMSE), structural similarity index measure (SSIM), registration failure rate, and feature number, based on 1,950,984 image registrations. No meaningful benefits from color space or channel were observed, although XYZ, RGB color space and L* color channel were able to outperform grayscale by a very minor margin. Per the dataset, the best-performing color space or channel, detector, and descriptor were XYZ/RGB, SIFT/FAST, and AKAZE. The most robust color space or channel, detector, and descriptor were L*a*b*, TBMR, and VGG. The color channel, detector, and descriptor with the most initial detector features and final homography features were Z/L*, FAST, and KAZE. In terms of the best overall unfailing combinations, XYZ/RGB+SIFT/FAST+VGG/SIFT seemed to provide the highest image registration quality, while Z+FAST+VGG provided the most image features.

Data Science for Health Image Alignment: A User-Friendly Open-Source ImageJ/Fiji Plugin for Aligning Multimodality/Immunohistochemistry/Immunofluorescence 2D Microscopy Images

Most of the time, the deep analysis of a biological sample requires the acquisition of images at different time points, using different modalities and/or different stainings. This information gives morphological, functional, and physiological insights, but the acquired images must be aligned to be able to proceed with the co-localisation analysis. Practically speaking, according to Aristotle's principle, "The whole is greater than the sum of its parts", multi-modal image registration is a challenging task that involves fusing complementary signals. In the past few years, several methods for image registration have been described in the literature, but unfortunately, there is not one method that works for all applications. In addition, there is currently no user-friendly solution for aligning images that does not require any computer skills. In this work, DS4H Image Alignment (DS4H-IA), an open-source ImageJ/Fiji plugin for aligning multimodality, immunohistochemistry (IHC), and/or immunofluorescence (IF) 2D microscopy images, designed with the goal of being extremely easy to use, is described. All of the available solutions for aligning 2D microscopy images have also been revised. The DS4H-IA source code; standalone applications for MAC, Linux, and Windows; video tutorials; manual documentation; and sample datasets are publicly available.

Multi-modality cardiac image computing: A survey

Multi-modality cardiac imaging plays a key role in the management of patients with cardiovascular diseases. It allows a combination of complementary anatomical, morphological and functional information, increases diagnosis accuracy, and improves the efficacy of cardiovascular interventions and clinical outcomes. Fully-automated processing and quantitative analysis of multi-modality cardiac images could have a direct impact on clinical research and evidence-based patient management. However, these require overcoming significant challenges including inter-modality misalignment and finding optimal methods to integrate information from different modalities. This paper aims to provide a comprehensive review of multi-modality imaging in cardiology, the computing methods, the validation strategies, the related clinical workflows and future perspectives. For the computing methodologies, we have a favored focus on the three tasks, i.e., registration, fusion and segmentation, which generally involve multi-modality imaging data, either combining information from different modalities or transferring information across modalities. The review highlights that multi-modality cardiac imaging data has the potential of wide applicability in the clinic, such as trans-aortic valve implantation guidance, myocardial viability assessment, and catheter ablation therapy and its patient selection. Nevertheless, many challenges remain unsolved, such as missing modality, modality selection, combination of imaging and non-imaging data, and uniform analysis and representation of different modalities. There is also work to do in defining how the well-developed techniques fit in clinical workflows and how much additional and relevant information they introduce. These problems are likely to continue to be an active field of research and the questions to be answered in the future.

Co-Registration of pre- and post-Operative images after cochlear Implantation: A proposed technique to Improve cochlear visualization and localization of cochlear electrodes

BACKGROUND AND PURPOSE

After cochlear implantation, metallic artifact can obscure nearby structures on CT images, which is problematic in patients with facial nerve stimulation (FNS). This study evaluated the usefulness of co-registered pre- and post-operative examinations to evaluate the cochlear implant and adjacent structures.

MATERIALS AND METHODS

A retrospective review was completed of consecutive patients that underwent CT imaging of the temporal bone before and after placement of a cochlear implant. Two blinded neuroradiologists independently reviewed all available examinations. All examinations were assessed for the presence or absence of dehiscence of the osseous ridge between the cochlea and facial nerve canal (FNC). Pre-operative and fused pre- and post-operative examinations were compared in their ability to visualize the osseous ridge using a 5-point Likert scale (ranging from 1 = unfused images were substantially superior to 5 = fused images were substantially superior). The electrode closest to the FNC were noted.

RESULTS

Of 34 included patients, 13 (38.2%) were female and 21 (61.8%) were male; average age was 72.2. Seven patients (20.6%) had frank dehiscence between the cochlea and FNC. Fused images were superior to the post-operative study alone for assessing the integrity of the osseous partition between the cochlea and FNC and for reducing artifact from the electrode array (average Likert scores for both reviewers were 4.4 and 4.7). There was good agreement between reviewers in noting electrode closest to the FNC (concordance correlation coefficient=0.82).

CONCLUSIONS

Following cochlear implantation, co-registered pre- and post-operative CT images are superior to conventional images in assessing the anatomic relationship between the cochlea and FNC.

Web-Based Application for Biomedical Image Registry, Analysis, and Translation (BiRAT)

Imaging has become an invaluable tool in preclinical research for its capability to non-invasively detect and monitor disease and assess treatment response. With the increased use of preclinical imaging, large volumes of image data are being generated requiring critical data management tools. Due to proprietary issues and continuous technology development, preclinical images, unlike DICOM-based images, are often stored in an unstructured data file in company-specific proprietary formats. This limits the available DICOM-based image management database to be effectively used for preclinical applications. A centralized image registry and management tool is essential for advances in preclinical imaging research. Specifically, such tools may have a high impact in generating large image datasets for the evolving artificial intelligence applications and performing retrospective analyses of previously acquired images. In this study, a web-based server application is developed to address some of these issues. The application is designed to reflect the actual experimentation workflow maintaining detailed records of both individual images and experimental data relevant to specific studies and/or projects. The application also includes a web-based 3D/4D image viewer to easily and quickly view and evaluate images. This paper briefly describes the initial implementation of the web-based application.

Improved EEG source localization with Bayesian uncertainty modelling of unknown skull conductivity

Electroencephalography (EEG) source imaging is an ill-posed inverse problem that requires accurate conductivity modelling of the head tissues, especially the skull. Unfortunately, the conductivity values are difficult to determine in vivo. In this paper, we show that the exact knowledge of the skull conductivity is not always necessary when the Bayesian approximation error (BAE) approach is exploited. In BAE, we first postulate a probability distribution for the skull conductivity that describes our (lack of) knowledge on its value, and model the effects of this uncertainty on EEG recordings with the help of an additive error term in the observation model. Before the Bayesian inference, the likelihood is marginalized over this error term. Thus, in the inversion we estimate only our primary unknown, the source distribution. We quantified the improvements in the source localization when the proposed Bayesian modelling was used in the presence of different skull conductivity errors and levels of measurement noise. Based on the results, BAE was able to improve the source localization accuracy, particularly when the unknown (true) skull conductivity was much lower than the expected standard conductivity value. The source locations that gained the highest improvements were shallow and originally exhibited the largest localization errors. In our case study, the benefits of BAE became negligible when the signal-to-noise ratio dropped to 20 dB.

Survey of Non-Rigid Registration Tools in Medicine

We catalogue available software solutions for non-rigid image registration to support scientists in selecting suitable tools for specific medical registration purposes. Registration tools were identified using non-systematic search in Pubmed, Web of Science, IEEE Xplore® Digital Library, Google Scholar, and through references in identified sources (n = 22). Exclusions are due to unavailability or inappropriateness. The remaining (n = 18) tools were classified by (i) access and technology, (ii) interfaces and application, (iii) living community, (iv) supported file formats, and (v) types of registration methodologies emphasizing the similarity measures implemented. Out of the 18 tools, (i) 12 are open source, 8 are released under a permissive free license, which imposes the least restrictions on the use and further development of the tool, 8 provide graphical processing unit (GPU) support; (ii) 7 are built on software platforms, 5 were developed for brain image registration; (iii) 6 are under active development but only 3 have had their last update in 2015 or 2016; (iv) 16 support the Analyze format, while 7 file formats can be read with only one of the tools; and (v) 6 provide multiple registration methods and 6 provide landmark-based registration methods. Based on open source, licensing, GPU support, active community, several file formats, algorithms, and similarity measures, the tools Elastics and Plastimatch are chosen for the platform ITK and without platform requirements, respectively. Researchers in medical image analysis already have a large choice of registration tools freely available. However, the most recently published algorithms may not be included in the tools, yet.

Multimodality Image Fusion for Coronary Artery Disease Detection: Concepts and Latest Developments

The debate on the role of anatomy and function in the assessment of coronary artery disease has been progressing for decades. While each imaging modality brings its own strengths and weaknesses, a multimodality image fusion approach combining an anatomical acquisition with a functional one has the potential of providing all the complementary information necessary to select the proper treatment. The technology has been available to physicians for a decade, but the recent introduction of positron emission tomography-derived absolute myocardial blood flow has further advanced the case for an image fusion diagnostic approach.

Anatomical accuracy of abdominal lesion localization

Software-based image registration can improve the diagnostic value of imaging procedures and is an alternative to hybrid scanners. The aim of this study was to evaluate the anatomical accuracy of automatic rigid image registration of independently acquired datasets of positron emission tomography with 18F-deoxyglucose and abdominal magnetic resonance imaging. Patients, methods: Analyses were performed on 28 abdominal lesions from 20 patients. The PET data were obtained using a stand-alone PET camera in 14 cases and a hybrid PET/CT scanner in 9 cases. The abdominal T1- and T2-weighted MRI scans were acquired on 1.5 T MRI scanners. The mean time interval between MRI and PET was 7.3 days (0–28 days). Automatic rigid registration was carried out using a self-developed registration tool integrated into commercial available software (InSpace for Siemens Syngo). Distances between the centres of gravity of 28 manually delineated neoplastic lesions represented in PET and MRI were measured in X-, Y-, and Z-direction. The intra- (intraclass correlation 0.94) and inter- (intraclass correlation 0.86) observer repeatability were high. Results: The average distance in all MRI sequences was 5.2 ± 7.6 mm in X-direction, 4.0 ± 3.7 mm in Y-direction and 6.1 ± 5.1 mm in Z-direction. There was a significantly higher misalignment in Z-direction (p < 0.05). The misalignment was not significantly different for the registration of T1- and T2- weighted sequences (p = 0.7). Conclusion: The misalignment between FDG-PET and abdominal MRI registered using an automated rigid registration tool was comparable to data reported for software-based fusion between PET and CT. Although this imprecision may not affect diagnostic accuracy, it is not sufficient to allow for pixel-wise integration of MRI and PET information.

Multimodality fusion imaging in abdominal and pelvic malignancies: current applications and future perspectives

Medicine is evolving toward personalized care and this development entails the integration, amalgamation, and synchronized analysis of data from multiple sources. Multimodality fusion imaging refers to the simultaneous visualization of spatially aligned and juxtaposed medical images obtained by two or more image modalities. PET/MRI scanners and MMFI platforms are able to improve the diagnostic workflow in oncologic patients and provide exquisite images that aid physicians in the molecular profiling and characterization of tissues. Advanced navigation platforms involving real-time ultrasound are promising tools for guiding personalized and tailored mini-invasive interventional procedures on technically challenging targets. The main objective of the present essay was to describe the current applications and future perspectives of multimodality fusion imaging for both diagnostic and interventional purposes in the field of abdominal and pelvic malignancies. We also outlined the technical differences between fusion imaging achieved by means of simultaneous bimodal acquisition (i.e., integrated PET/MRI scanners), retrospective co-registration, and multimodality fusion imaging involving ultrafast or real-time imaging modalities.

Navigational Tools for Interventional Radiology and Interventional Oncology Applications

The interventional radiologist is increasingly called upon to successfully access challenging biopsy and ablation targets, which may be difficult based on poor visualization, small size, or the proximity of vulnerable regional anatomy. Complex therapeutic procedures, including tumor ablation and transarterial oncologic therapies, can be associated with procedural risk, significant procedure time, and measurable radiation time. Navigation tools, including electromagnetic, optical, laser, and robotic guidance systems, as well as image fusion platforms, have the potential to facilitate these complex interventions with the potential to improve lesion targeting, reduce procedure time, and radiation dose, and thus potentially improve patient outcomes. This review will provide an overview of currently available navigational tools and their application to interventional radiology and oncology. A summary of the pertinent literature on the use of these tools to improve safety and efficacy of interventional procedures compared with conventional techniques will be presented.

Clinical applications of PET/MRI: current status and future perspectives

Fully integrated positron emission tomography (PET)/magnetic resonance imaging (MRI) scanners have been available for a few years. Since then, the number of scanner installations and published studies have been growing. While feasibility of integrated PET/MRI has been demonstrated for many clinical and preclinical imaging applications, now those applications where PET/MRI provides a clear benefit in comparison to the established reference standards need to be identified. The current data show that those particular applications demanding multiparametric imaging capabilities, high soft tissue contrast and/or lower radiation dose seem to benefit from this novel hybrid modality. Promising results have been obtained in whole-body cancer staging in non-small cell lung cancer and multiparametric tumor imaging. Furthermore, integrated PET/MRI appears to have added value in oncologic applications requiring high soft tissue contrast such as assessment of liver metastases of neuroendocrine tumors or prostate cancer imaging. Potential benefit of integrated PET/MRI has also been demonstrated for cardiac (i.e., myocardial viability, cardiac sarcoidosis) and brain (i.e., glioma grading, Alzheimer's disease) imaging, where MRI is the predominant modality. The lower radiation dose compared to PET/computed tomography will be particularly valuable in the imaging of young patients with potentially curable diseases.However, further clinical studies and technical innovation on scanner hard- and software are needed. Also, agreements on adequate refunding of PET/MRI examinations need to be reached. Finally, the translation of new PET tracers from preclinical evaluation into clinical applications is expected to foster the entire field of hybrid PET imaging, including PET/MRI.

EANM procedural guidelines for radionuclide myocardial perfusion imaging with SPECT and SPECT/CT: 2015 revision

Since the publication of the European Association of Nuclear Medicine (EANM) procedural guidelines for radionuclide myocardial perfusion imaging (MPI) in 2005, many small and some larger steps of progress have been made, improving MPI procedures. In this paper, the major changes from the updated 2015 procedural guidelines are highlighted, focusing on the important changes related to new instrumentation with improved image information and the possibility to reduce radiation exposure, which is further discussed in relation to the recent developments of new International Commission on Radiological Protection (ICRP) models. Introduction of the selective coronary vasodilator regadenoson and the use of coronary CT-contrast agents for hybrid imaging with SPECT/CT angiography are other important areas for nuclear cardiology that were not included in the previous guidelines. A large number of minor changes have been described in more detail in the fully revised version available at the EANM home page: http://eanm.org/publications/guidelines/2015_07_EANM_FINAL_myocardial_perfusion_guideline.pdf.

Diffusion-weighted and PET/MR Imaging after Radiation Therapy for Malignant Head and Neck Tumors

Interpreting imaging studies of the irradiated neck constitutes a challenge because of radiation therapy-induced tissue alterations, the variable appearances of recurrent tumors, and functional and metabolic phenomena that mimic disease. Therefore, morphologic magnetic resonance (MR) imaging, diffusion-weighted (DW) imaging, positron emission tomography with computed tomography (PET/CT), and software fusion of PET and MR imaging data sets are increasingly used to facilitate diagnosis in clinical practice. Because MR imaging and PET often yield complementary information, PET/MR imaging holds promise to facilitate differentiation of tumor recurrence from radiation therapy-induced changes and complications. This review focuses on clinical applications of DW and PET/MR imaging in the irradiated neck and discusses the added value of multiparametric imaging to solve diagnostic dilemmas. Radiologists should understand key features of radiation therapy-induced tissue alterations and potential complications seen at DW and PET/MR imaging, including edema, fibrosis, scar tissue, soft-tissue necrosis, bone and cartilage necrosis, cranial nerve palsy, and radiation therapy-induced arteriosclerosis, brain necrosis, and thyroid disorders. DW and PET/MR imaging also play a complementary role in detection of residual and recurrent disease. Interpretation pitfalls due to technical, functional, and metabolic phenomena should be recognized and avoided. Familiarity with DW and PET/MR imaging features of expected findings, potential complications, and treatment failure after radiation therapy increases diagnostic confidence when interpreting images of the irradiated neck. Online supplemental material is available for this article.

Anatomy-guided brain PET imaging incorporating a joint prior model

We proposed a maximum a posterior (MAP) framework for incorporating information from co-registered anatomical images into PET image reconstruction through a novel anato-functional joint prior. The characteristic of the utilized hyperbolic potential function is determinate by the voxel intensity differences within the anatomical image, while the penalization is computed based on voxel intensity differences in reconstructed PET images. Using realistic simulated (18)FDG PET scan data, we optimized the performance of the proposed MAP reconstruction with the joint prior (JP-MAP) and compared its performance with conventional 3D MLEM and 3D MAP reconstructions. The proposed JP-MAP reconstruction algorithm resulted in quantitatively enhanced reconstructed images, as demonstrated in extensive FDG PET simulation study. The proposed method was also tested on a 20 min Florbetapir patient study performed on the high-resolution research tomograph. It was shown to outperform conventional methods in visual as well as quantitative accuracy assessment (in terms of regional noise versus activity value performance). The JP-MAP method was also compared with another MR-guided MAP reconstruction method, utilizing the Bowsher prior and was seen to result in some quantitative enhancements, especially in the case of MR-PET mis-registrations, and a definitive improvement in computational performance.

Image coregistration: quantitative processing framework for the assessment of brain lesions

The quantitative, multiparametric assessment of brain lesions requires coregistering different parameters derived from MRI sequences. This will be followed by analysis of the voxel values of the ROI within the sequences and calculated parametric maps, and deriving multiparametric models to classify imaging data. There is a need for an intuitive, automated quantitative processing framework that is generalized and adaptable to different clinical and research questions. As such flexible frameworks have not been previously described, we proceeded to construct a quantitative post-processing framework with commonly available software components. Matlab was chosen as the programming/integration environment, and SPM was chosen as the coregistration component. Matlab routines were created to extract and concatenate the coregistration transforms, take the coregistered MRI sequences as inputs to the process, allow specification of the ROI, and store the voxel values to the database for statistical analysis. The functionality of the framework was validated using brain tumor MRI cases. The implementation of this quantitative post-processing framework enables intuitive creation of multiple parameters for each voxel, facilitating near real-time in-depth voxel-wise analysis. Our initial empirical evaluation of the framework is an increased usage of analysis requiring post-processing and increased number of simultaneous research activities by clinicians and researchers with non-technical backgrounds. We show that common software components can be utilized to implement an intuitive real-time quantitative post-processing framework, resulting in improved scalability and increased adoption of post-processing needed to answer important diagnostic questions.

The utility of deformable image registration for small artery visualisation in contrast-enhanced whole body MR angiography

PURPOSE

An investigation was carried out into the effect of three image registration techniques on the diagnostic image quality of contrast-enhanced magnetic resonance angiography (CE-MRA) images.

METHODS

Whole-body CE-MRA data from the lower legs of 27 patients recruited onto a study of asymptomatic atherosclerosis were processed using three deformable image registration algorithms. The resultant diagnostic image quality was evaluated qualitatively in a clinical evaluation by four expert observers, and quantitatively by measuring contrast-to-noise ratios and volumes of blood vessels, and assessing the techniques' ability to correct for varying degrees of motion.

RESULTS

The first registration algorithm ('AIR') introduced significant stenosis-mimicking artefacts into the blood vessels' appearance, observed both qualitatively (clinical evaluation) and quantitatively (vessel volume measurements). The two other algorithms ('Slicer' and 'SEMI'), based on the normalised mutual information (NMI) concept and designed specifically to deal with variations in signal intensity as found in contrast-enhanced image data, did not suffer from this serious issue but were rather found to significantly improve the diagnostic image quality both qualitatively and quantitatively, and demonstrated a significantly improved ability to deal with the common problem of patient motion.

CONCLUSIONS

This work highlights both the significant benefits to be gained through the use of suitable registration algorithms and the deleterious effects of an inappropriate choice of algorithm for contrast-enhanced MRI data. The maximum benefit was found in the lower legs, where the small arterial vessel diameters and propensity for leg movement during image acquisitions posed considerable problems in making accurate diagnoses from the un-registered images.

Application of tolerance limits to the characterization of image registration performance

Deformable image registration is used increasingly in image-guided interventions and other applications. However, validation and characterization of registration performance remain areas that require further study. We propose an analysis methodology for deriving tolerance limits on the initial conditions for deformable registration that reliably lead to a successful registration. This approach results in a concise summary of the probability of registration failure, while accounting for the variability in the test data. The (β, γ) tolerance limit can be interpreted as a value of the input parameter that leads to successful registration outcome in at least 100β% of cases with the 100γ% confidence. The utility of the methodology is illustrated by summarizing the performance of a deformable registration algorithm evaluated in three different experimental setups of increasing complexity. Our examples are based on clinical data collected during MRI-guided prostate biopsy registered using publicly available deformable registration tool. The results indicate that the proposed methodology can be used to generate concise graphical summaries of the experiments, as well as a probabilistic estimate of the registration outcome for a future sample. Its use may facilitate improved objective assessment, comparison and retrospective stress-testing of deformable.

Influence of rigid coregistration of PET and CT data on metabolic volumetry: a user's perspective

Background

While non-rigid fusion is by definition expected to alter the information of positron emission tomography (PET) data, we assessed whether rigid transformation also influences metabolic tumor volume (MTV) determination.

Methods

The PET/computed tomography (CT) data of 28 solid pulmonary lesions of 20 tumor patients examined with ¹⁸ F-Fluordeoxyglucose (FDG) was retrospectively analyzed. The original (OR) hardware-coregistered PET images were fused with contrast-enhanced diagnostic CT (CT1, 1 mm slices) and low dose CT (CT5, 5 mm slices). After automatic rigid transformation (Mirada Fusion7D) using two algorithms (rigid fast (RF), rigid slow (RS)), MTV and maximal standardized uptake value (SUVmax) were determined applying four different segmentation methods with either fixed or background-adapted thresholding and compared to OR-PET data.

Results

Relative differences in SUVmax compared to OR data revealed no significant differences for RF (median, −0.1%; interquartile range (IQR), −1.1% to 0.9%; p = 0.75) and RS (median, 0.5%; IQR, −0.6% to 1.3%; p = 0.19) in CT1, whereas in CT5 significant deviations were observed for RF (median, −9.0%; IQR, −10.9 to −6.1; p < 0.001) and RS (median, −8.4%; IQR, −11.1 to −5.6; p < 0.001). Relative MTV differences were 0.7% (IQR, −3.0% to 2.7%; p = 0.76) for RF and −1.3% (IQR, −3.6% to 0.9%; p = 0.12) for RS in CT1. Coregistration led to significant MTV differences in RF (median, 10.4%; IQR, 7.4% to 16.7%; p < 0.001) and RS (median, 10.6%; IQR, 5.4% to 17.7%; p < 0.001) in CT5.

Conclusions

Rigid coregistration of PET data allows a quantitative evaluation with reasonable accuracy in most cases. However, in some cases, it can result in substantial deviations of MTV and SUVmax. Therefore, it is recommended to perform quantitative evaluation in the original PET data rather than in coregistered PET data.

Multimodality image fusion for diagnosing coronary artery disease

Coronary artery disease (CAD) is one of the leading causes of death in the US and a substantial health-care burden in all industrialized societies. In recent years we have witnessed a constant strive towards the development and the clinical application of novel or improved detection methods as well as therapies. Particularly, noninvasive imaging is a decisive component in the cardiovascular field. Image fusion is the ability of combining into a single integrated display the anatomical as well as the physiological data retrieved by separated modalities. Clinical evidence suggests that it represents a promising strategy in CAD assessment and risk stratification by significantly improving the diagnostic power of each modality independently considered and of the traditional side-by-side interpretation. Numerous techniques and approaches taken from the image registration field have been implemented and validated in the context of CAD assessment and management. Although its diagnostic power is widely accepted, additional technical developments are still needed to become a routinely used clinical tool.

Diagnosing coronary artery disease with hybrid PET/CT: it takes two to tango

The noninvasive diagnosis of coronary artery disease (CAD) is a challenging task. Although a large armamentarium of imaging modalities is available to evaluate the functional consequences of the extent and severity of CAD, cardiac perfusion positron emission tomography (PET) is considered the gold standard for this purpose. Alternatively, noninvasive anatomical imaging of coronary atherosclerosis with coronary computed tomography angiography (CCTA) has recently been successfully implemented in clinical practice. Although each of these diagnostic approaches has its own merits and caveats, functional and morphological imaging techniques provide fundamentally different insights into the disease process and should be considered to be complementary rather than overlapping. Hybrid imaging with PET/CT offers the possibility to evaluate both aspects nearly simultaneously, and studies have demonstrated that such a comprehensive assessment results in superior diagnostic accuracy, better prognostication, and helps in guiding clinical patient management. The aim of this review is to discuss the value of stand-alone CCTA and PET in CAD, and to summarize the available data on the surplus value of hybrid PET/CT including its strengths and limitations.

Image fusion during vascular and nonvascular image-guided procedures

Image fusion may be useful in any procedure where previous imaging such as positron emission tomography, magnetic resonance imaging, or contrast-enhanced computed tomography (CT) defines information that is referenced to the procedural imaging, to the needle or catheter, or to an ultrasound transducer. Fusion of prior and intraoperative imaging provides real-time feedback on tumor location or margin, metabolic activity, device location, or vessel location. Multimodality image fusion in interventional radiology was initially introduced for biopsies and ablations, especially for lesions only seen on arterial phase CT, magnetic resonance imaging, or positron emission tomography/CT but has more recently been applied to other vascular and nonvascular procedures. Two different types of platforms are commonly used for image fusion and navigation: (1) electromagnetic tracking and (2) cone-beam CT. Both technologies would be reviewed as well as their strengths and weaknesses, indications, when to use one vs the other, tips and guidance to streamline use, and early evidence defining clinical benefits of these rapidly evolving, commercially available and emerging techniques.

PET/MR imaging: technical aspects and potential clinical applications

Instruments that combine positron emission tomography (PET) and magnetic resonance (MR) imaging have recently been assembled for use in humans, and may have diagnostic performance superior to that of PET/computed tomography (CT) for particular clinical and research applications. MR imaging has major strengths compared with CT, including superior soft-tissue contrast resolution, multiplanar image acquisition, and functional imaging capability through specialized techniques such as diffusion-tensor imaging, diffusion-weighted (DW) imaging, functional MR imaging, MR elastography, MR spectroscopy, perfusion-weighted imaging, MR imaging with very short echo times, and the availability of some targeted MR imaging contrast agents. Furthermore, the lack of ionizing radiation from MR imaging is highly appealing, particularly when pediatric, young adult, or pregnant patients are to be imaged, and the safety profile of MR imaging contrast agents compares very favorably with iodinated CT contrast agents. MR imaging also can be used to guide PET image reconstruction, partial volume correction, and motion compensation for more accurate disease quantification and can improve anatomic localization of sites of radiotracer uptake, improve diagnostic performance, and provide for comprehensive regional and global structural, functional, and molecular assessment of various clinical disorders. In this review, we discuss the historical development, software-based registration, instrumentation and design, quantification issues, potential clinical applications, potential clinical roles of image segmentation and global disease assessment, and challenges related to PET/MR imaging.

SUPPLEMENTAL MATERIAL

http://radiology.rsna.org/lookup/suppl/doi:10.1148/radiol.13121038/-/DC1.

Registration of PET and CT images based on multiresolution gradient of mutual information demons algorithm for positioning esophageal cancer patients

Accurate registration of  18F−FDG PET (positron emission tomography) and CT (computed tomography) images has important clinical significance in radiation oncology. PET and CT images are acquired from  18F−FDG PET/CT scanner, but the two acquisition processes are separate and take a long time. As a result, there are position errors in global and deformable errors in local caused by respiratory movement or organ peristalsis. The purpose of this work was to implement and validate a deformable CT to PET image registration method in esophageal cancer to eventually facilitate accurate positioning the tumor target on CT, and improve the accuracy of radiation therapy. Global registration was firstly utilized to preprocess position errors between PET and CT images, achieving the purpose of aligning these two images on the whole. Demons algorithm, based on optical flow field, has the features of fast process speed and high accuracy, and the gradient of mutual information‐based demons (GMI demons) algorithm adds an additional external force based on the gradient of mutual information (GMI) between two images, which is suitable for multimodality images registration. In this paper, GMI demons algorithm was used to achieve local deformable registration of PET and CT images, which can effectively reduce errors between internal organs. In addition, to speed up the registration process, maintain its robustness, and avoid the local extremum, multiresolution image pyramid structure was used before deformable registration. By quantitatively and qualitatively analyzing cases with esophageal cancer, the registration scheme proposed in this paper can improve registration accuracy and speed, which is helpful for precisely positioning tumor target and developing the radiation treatment planning in clinical radiation therapy application.

PACS numbers: 87.57.nj, 87.57.Q‐, 87.57.uk

Discriminative segmentation-based evaluation through shape dissimilarity

Segmentation-based scores play an important role in the evaluation of computational tools in medical image analysis. These scores evaluate the quality of various tasks, such as image registration and segmentation, by measuring the similarity between two binary label maps. Commonly these measurements blend two aspects of the similarity: pose misalignments and shape discrepancies. Not being able to distinguish between these two aspects, these scores often yield similar results to a widely varying range of different segmentation pairs. Consequently, the comparisons and analysis achieved by interpreting these scores become questionable. In this paper, we address this problem by exploring a new segmentation-based score, called normalized Weighted Spectral Distance (nWSD), that measures only shape discrepancies using the spectrum of the Laplace operator. Through experiments on synthetic and real data we demonstrate that nWSD provides additional information for evaluating differences between segmentations, which is not captured by other commonly used scores. Our results demonstrate that when jointly used with other scores, such as Dice's similarity coefficient, the additional information provided by nWSD allows richer, more discriminative evaluations. We show for the task of registration that through this addition we can distinguish different types of registration errors. This allows us to identify the source of errors and discriminate registration results which so far had to be treated as being of similar quality in previous evaluation studies.

Attenuation Correction Strategies for Positron Emission Tomography/Computed Tomography and 4-Dimensional Positron Emission Tomography/Computed Tomography

This article discusses attenuation correction strategies in positron emission tomography/computed tomography (PET/CT) and 4-dimensional PET/CT imaging. Average CT scan derived from averaging the high temporal resolution CT images is effective in improving the registration of the CT and the PET images and quantification of the PET data. It underscores list-mode data acquisition in 4-dimensional PET, and introduces 4-dimensional CT, popular in thoracic treatment planning, to 4-dimensional PET/CT.

Novel Quantitative Techniques in Hybrid (PET-MR) Imaging of Brain Tumors

Brain tumors are a collection of heterogeneous intracranial neoplasms. Molecular PET and anatomic MR imaging together can provide reliable quantitative information on tumor characterization, and help in treatment planning and monitoring therapeutic evaluation, noninvasively. Coregistration of MRI and PET images have been successfully used to improve diagnostic accuracy and in evaluating treatment response. Whole-body PET-MR scanners capable of assessing morphologic, metabolic, and functional information simultaneously are now commercially available. Early clinical studies speculate that PET-MR will be useful in several clinical specialties. In this report, we highlight the advances and applications of hybrid PET-MR in quantitative brain tumor imaging.

Hybrid imaging in planar scintigraphy: new implementations and historical precedents

Fusion of tomographic radionuclide studies with anatomical examinations has become standard practice in positron emission tomography (PET) and single photon emission computed tomography (SPECT) imaging. Nonetheless, fusion of planar scintigraphic images with an anatomical modality remains distinctly uncommon, although methods to do so have appeared sporadically in the literature during the past 2 decades. In this article we review several techniques that have been used to combine planar scintigraphic images with radiographs and visual (photographic) images. Rigid or affine transformations have been performed to co-register the planar images with each other using custom, commercial, or public domain software. Display of the hybrid images has been achieved primarily with nonselective color-fusion methods. Promising efforts are underway to develop a technique of fusing planar lymphoscintigraphic images with CT topograms (scout images) obtained on the SPECT-CT camera in a manner that compensates for position-dependent variation in magnification that affects the CT scout. An advantage of this approach is that both of the component images are acquired on the same gantry, without need for repositioning of the patient. It is instructive to note that techniques of fusing rectilinear scans with radiographic and visual images were first developed more than 50 years ago. The revisiting of these methods after many decades reflects a fundamental need for spatial orientation in nuclear medicine that fusion imaging can also bring to planar scintigraphic studies.

Automatic alignment of myocardial perfusion PET and 64-slice coronary CT angiography on hybrid PET/CT

BACKGROUND

Hybrid PET/CT allows for acquisition of cardiac PET and coronary CT angiography (CCTA) in one session. However, PET and CCTA are acquired with differing breathing protocols and require software registration. We aimed to validate automatic correction for breathing misalignment between PET and CCTA acquired on hybrid scanner.

METHODS

Single-session hybrid PET/CT studies of rest/stress (13)N-ammonia PET and CCTA in 32 consecutive patients were considered. Automated registration of PET left ventricular (LV) surfaces with CCTA volumes was evaluated by comparing with expert manual alignment by two observers.

RESULTS

The average initial misalignments between the position of LV on PET and CCTA were 27.2 ± 11.8, 13.3 ± 11.5, and 14.3 ± 9.1 mm in x, y, and z axes on rest, and 26.3 ± 10.2, 11.1 ± 9.5, and 11.7 ± 7.1 mm in x, y, and z axes on stress, respectively. The automated PET-CCTA co-registration had 95% agreement as judged visually. Compared with expert manual alignment, the translation errors of the algorithm were 5.3 ± 2.8 mm (rest) and 6.0 ± 3.5 mm (stress). 3D visualization of combined coronary vessel anatomy and hypoperfusion from PET could be made without further manual adjustments.

CONCLUSION

Software co-registration of CCTA and PET myocardial perfusion imaging on hybrid PET/CT scanners is necessary, but can be performed automatically, facilitating integrated 3D display on PET/CT.

Medical image registration: a review

This paper presents a review of automated image registration methodologies that have been used in the medical field. The aim of this paper is to be an introduction to the field, provide knowledge on the work that has been developed and to be a suitable reference for those who are looking for registration methods for a specific application. The registration methodologies under review are classified into intensity or feature based. The main steps of these methodologies, the common geometric transformations, the similarity measures and accuracy assessment techniques are introduced and described.

Evaluation of three MRI-based anatomical priors for quantitative PET brain imaging

In emission tomography, image reconstruction and therefore also tracer development and diagnosis may benefit from the use of anatomical side information obtained with other imaging modalities in the same subject, as it helps to correct for the partial volume effect. One way to implement this, is to use the anatomical image for defining the a priori distribution in a maximum-a-posteriori (MAP) reconstruction algorithm. In this contribution, we use the PET-SORTEO Monte Carlo simulator to evaluate the quantitative accuracy reached by three different anatomical priors when reconstructing positron emission tomography (PET) brain images, using volumetric magnetic resonance imaging (MRI) to provide the anatomical information. The priors are: 1) a prior especially developed for FDG PET brain imaging, which relies on a segmentation of the MR-image (Baete , 2004); 2) the joint entropy-prior (Nuyts, 2007); 3) a prior that encourages smoothness within a position dependent neighborhood, computed from the MR-image. The latter prior was recently proposed by our group in (Vunckx and Nuyts, 2010), and was based on the prior presented by Bowsher (2004). The two latter priors do not rely on an explicit segmentation, which makes them more generally applicable than a segmentation-based prior. All three priors produced a compromise between noise and bias that was clearly better than that obtained with postsmoothed maximum likelihood expectation maximization (MLEM) or MAP with a relative difference prior. The performance of the joint entropy prior was slightly worse than that of the other two priors. The performance of the segmentation-based prior is quite sensitive to the accuracy of the segmentation. In contrast to the joint entropy-prior, the Bowsher-prior is easily tuned and does not suffer from convergence problems.

Rigid registration of medical images using 1D and 2D binary projections

Image registration is a necessary procedure in everyday clinical practice. Several techniques for rigid and non-rigid registration have been developed and tested and the state-of-the-art is evolving from the research setting to incorporate image registration techniques into clinically useful tools. In this paper, we develop a novel rigid medical image registration technique which incorporates binary projections. This technique is tested and compared to the standard mutual information (MI) methods. Results show that the method is significantly more accurate and robust compared to MI methods. The accuracy is well below 0.5° and 0.5 mm. This method introduces two more improvements over MI methods: (1)for 2D registration with the use of 1D binary projections, we use minimal interpolation; and (2) for 3D registration with the use of 2D binary projections the method converges to stable final positions, independent of the initial misregistration.

Hybrid cardiac imaging: SPECT/CT and PET/CT. A joint position statement by the European Association of Nuclear Medicine (EANM), the European Society of Cardiac Radiology (ESCR) and the European Council of Nuclear Cardiology (ECNC)

Improvements in software and hardware have enabled the integration of dual imaging modalities into hybrid systems, which allow combined acquisition of the different data sets. Integration of positron emission tomography (PET) and computed tomography (CT) scanners into PET/CT systems has shown improvement in the management of patients with cancer over stand-alone acquired CT and PET images. Hybrid cardiac imaging either with single photon emission computed tomography (SPECT) or PET combined with CT depicts cardiac and vascular anatomical abnormalities and their physiologic consequences in a single setting and appears to offer superior information compared with either stand-alone or side-by-side interpretation of the data sets in patients with known or suspected coronary artery disease (CAD). Hybrid systems are also advantageous for the patient because of the single short dual data acquisition. However, hybrid cardiac imaging has also generated controversy with regard to which patients should undergo such integrated examination for clinical effectiveness and minimization of costs and radiation dose, and if software-based fusion of images obtained separately would be a useful alternative. The European Association of Nuclear Medicine (EANM), the European Society of Cardiac Radiology (ESCR) and the European Council of Nuclear Cardiology (ECNC) in this paper want to present a position statement of the institutions on the current roles of SPECT/CT and PET/CT hybrid cardiac imaging in patients with known or suspected CAD.

Real-time co-registration using novel ultrasound technology: ex vivo validation and in vivo applications

OBJECTIVE

The study objective was to evaluate whether a novel global position system (GPS)-like position-sensing technology will enable accurate co-registration of images between imaging modalities. Co-registration of images obtained by different imaging modalities will allow for comparison and fusion between imaging modalities, and therefore has significant clinical and research implications. We compared ultrasound (US) and magnetic resonance imaging (MRI) scans of carotid endarterectomy (CEA) specimens using a novel position-sensing technology that uses an electromagnetic (EM) transmitter and sensors mounted on a US transducer. We then evaluated in vivo US-US and US-MRI co-registration.

METHODS

Thirteen CEA specimens underwent 3.0 Tesla MRI, after which images were uploaded to a LOGIQ E9 3D (GE Healthcare, Wauwatosa, WI) US system and registered by identifying two to three common points. A similar method was used to evaluate US-MRI co-registration in patients with carotid atherosclerosis. For carotid intima-media thickness (C-IMT) measurements, 10 volunteers underwent bilateral carotid US scans co-registered to three-dimensional US maps created on the initial visit, with a repeat scan 2 days later.

RESULTS

For the CEA specimens, there was a mean of 20 (standard error [SE] 2.0) frames per MRI slice. The mean frame difference, over 33 registration markers, between MRI and US scans for readers 1 and 2 was -2.82 ± 19.32 and 2.09 ± 14.68 (mean ± 95% CI) frames, respectively. The US-MRI intraclass correlation coefficients (ICCs) for the first and second readers were 0.995 and 0.997, respectively. For patients with carotid atherosclerosis, the mean US frames per MRI slice (9 [SE 2.3]) was within range of that observed with CEA specimens. Inter-visit, intra-reader, and inter-reader reproducibility of C-IMT measurements were consistently high (side-averaged ICC >0.9).

CONCLUSION

Accurate co-registration between US and other modalities is feasible with a GPS-like technology, which has significant clinical and research applicability.

The future of hybrid imaging-part 1: hybrid imaging technologies and SPECT/CT

Since the 1990s, hybrid imaging by means of software and hardware image fusion alike allows the intrinsic combination of functional and anatomical image information. This review summarises in three parts the state-of-the-art of dual-technique imaging, with a focus on clinical applications. We will attempt to highlight selected areas of potential improvement of combined imaging technologies and new applications. In this first part, we briefly review the origins of hybrid imaging and comment on the status and future development of single photon emission tomography (SPECT)/computed tomography (CT). In short, we could predict that, within 10 years, we may see all existing dual-technique imaging systems, including SPECT/CT, in clinical routine use worldwide. SPECT/CT, in particular, may evolve into a whole-body imaging technique with supplementary use in dosimetry applications.

Co-registration of Sequential Multidetector Computed Tomography Studies for the Evaluation of Surgical Instrumentation following Resection of Spinal Tumors

Surgical resection of spinal tumors involves complex reconstructive procedures. The stability and integrity of the surgical construct are evaluated with multidetector computed tomography (MDCT). As coregistration, or fusion, of different imaging modalities, especially positron emission tomography/computed tomography (PET/CT), is common practice, we sought to determine if this technique could be applied to sequential, postoperative MDCT studies of the spine. Herein, we demonstrate that by utilizing the Hermes workstation, co-registration of MDCT spine studies can be performed. This technique allows sequential MDCT examinations of the post-operative spine to be viewed together as one study and may aid in evaluation of the position and integrity of the surgical construct over time. Further study and refinement of this technique will be necessary before clinical implementation.