3D Multi-Modality Medical Imaging: Combining Anatomical and Infrared Thermal Images for 3D Reconstruction

Medical thermography provides an overview of the human body with two-dimensional (2D) information that assists the identification of temperature changes, based on the analysis of surface distribution. However, this approach lacks spatial depth information, which can be enhanced by adding multiple images or three-dimensional (3D) systems. Therefore, the methodology applied for this paper generates a 3D point cloud (from thermal infrared images), a 3D geometry model (from CT images), and the segmented inner anatomical structures. Thus, the following computational processing was employed: Structure from Motion (SfM), image registration, and alignment (affine transformation) between the 3D models obtained to combine and unify them. This paper presents the 3D reconstruction and visualization of the respective geometry of the neck/bust and inner anatomical structures (thyroid, trachea, veins, and arteries). Additionally, it shows the whole 3D thermal geometry in different anatomical sections (i.e., coronal, sagittal, and axial), allowing it to be further examined by a medical team, improving pathological assessments. The generation of 3D thermal anatomy models allows for a combined visualization, i.e., functional and anatomical images of the neck region, achieving encouraging results. These 3D models bring correlation of the inner and outer regions, which could improve biomedical applications and future diagnosis with such a methodology.

Modeling of Respiratory Motion to Support the Minimally Invasive Destruction of Liver Tumors

OBJECTIVE

Respiratory movements are a significant factor that may hinder the use of image navigation systems during minimally invasive procedures used to destroy focal lesions in the liver. This article aims to present a method of estimating the displacement of the target point due to respiratory movements during the procedure, working in real time.

METHOD

The real-time method using skin markers and non-rigid registration algorithms has been implemented and tested for various classes of transformation. The method was validated using clinical data from 21 patients diagnosed with liver tumors. For each patient, each marker was treated as a target and the remaining markers as target position predictors, resulting in 162 configurations and 1095 respiratory cycles analyzed. In addition, the possibility of estimating the respiratory phase signal directly from intraoperative US images and the possibility of synchronization with the 4D CT respiratory sequence are also presented, based on ten patients.

RESULTS

The median value of the target registration error (TRE) was 3.47 for the non-rigid registration method using the combination of rigid transformation and elastic body spline curves, and an adaptation of the assessing quality using image registration circuits (AQUIRC) method. The average maximum distance was 3.4 (minimum: 1.6, maximum 6.8) mm.

CONCLUSIONS

The proposed method obtained promising real-time TRE values. It also allowed for the estimation of the TRE at a given geometric margin level to determine the estimated target position. Directions for further quantitative research and the practical possibility of combining both methods are also presented.

Robust Multiscale Identification of Apparent Elastic Properties at Mesoscale for Random Heterogeneous Materials with Multiscale Field Measurements

The aim of this work is to efficiently and robustly solve the statistical inverse problem related to the identification of the elastic properties at both macroscopic and mesoscopic scales of heterogeneous anisotropic materials with a complex microstructure that usually cannot be properly described in terms of their mechanical constituents at microscale. Within the context of linear elasticity theory, the apparent elasticity tensor field at a given mesoscale is modeled by a prior non-Gaussian tensor-valued random field. A general methodology using multiscale displacement field measurements simultaneously made at both macroscale and mesoscale has been recently proposed for the identification the hyperparameters of such a prior stochastic model by solving a multiscale statistical inverse problem using a stochastic computational model and some information from displacement fields at both macroscale and mesoscale. This paper contributes to the improvement of the computational efficiency, accuracy and robustness of such a method by introducing (i) a mesoscopic numerical indicator related to the spatial correlation length(s) of kinematic fields, allowing the time-consuming global optimization algorithm (genetic algorithm) used in a previous work to be replaced with a more efficient algorithm and (ii) an ad hoc stochastic representation of the hyperparameters involved in the prior stochastic model in order to enhance both the robustness and the precision of the statistical inverse identification method. Finally, the proposed improved method is first validated on in silico materials within the framework of 2D plane stress and 3D linear elasticity (using multiscale simulated data obtained through numerical computations) and then exemplified on a real heterogeneous biological material (beef cortical bone) within the framework of 2D plane stress linear elasticity (using multiscale experimental data obtained through mechanical testing monitored by digital image correlation).

Deep Evolutionary Networks with Expedited Genetic Algorithms for Medical Image Denoising

Deep convolutional neural networks offer state-of-the-art performance for medical image analysis. However, their architectures are manually designed for particular problems. On the one hand, a manual designing process requires many trials to tune a large number of hyperparameters and is thus quite a time-consuming task. On the other hand, the fittest hyperparameters that can adapt to source data properties (e.g., sparsity, noisy features) are not able to be quickly identified for target data properties. For instance, the realistic noise in medical images is usually mixed and complicated, and sometimes unknown, leading to challenges in applying existing methods directly and creating effective denoising neural networks easily. In this paper, we present a Genetic Algorithm (GA)-based network evolution approach to search for the fittest genes to optimize network structures automatically. We expedite the evolutionary process through an experience-based greedy exploration strategy and transfer learning. Our evolutionary algorithm procedure has flexibility, which allows taking advantage of current state-of-the-art modules (e.g., residual blocks) to search for promising neural networks. We evaluate our framework on a classic medical image analysis task: denoising. The experimental results on computed tomography perfusion (CTP) image denoising demonstrate the capability of the method to select the fittest genes for building high-performance networks, named EvoNets. Our results outperform state-of-the-art methods consistently at various noise levels.

Evaluation of a novel elastic registration algorithm for spinal imaging data: A pilot clinical study

BACKGROUND

Rigid image coregistration is an established technique that allows spatial aligning. However, rigid fusion is prone to deformation of the imaged anatomies. In this work, a novel fully automated elastic image registration method is evaluated.

METHODS

Cervical CT and MRI data of 10 patients were evaluated. The MRI was acquired with the patient in neutral, flexed, and rotated head position. Vertebrawise rigid fusions were performed to transfer bony landmarks for each vertebra from the CT to the MRI space serving as a reference.

RESULTS

Elastic fusion of 3D MRI data showed the highest image registration accuracy (target registration error of 3.26 mm with 95% confidence). Further, an elastic fusion of 2D axial MRI data (<4.75 mm with 95% c.) was more reliable than for 2D sagittal sequences (<6.02 mm with 95% c.).

CONCLUSIONS

The novel method enables elastic MRI-to-CT image coregistration for cervical indications with changes of the head position.

Integrating Morphological Edge Detection and Mutual Information for Nonrigid Registration of Medical Images

Background:

Medical images are widely used within healthcare and medical research. There is an increased interest in precisely correlating information in these images through registration techniques for investigative and therapeutic purposes. This work proposes and evaluates an improved measure function for registration of carotid ultrasound and magnetic resonance images (MRI) taken at different times.

Methods:

To achieve this, a morphological edge detection operator has been designed to extract the vital edge information from images which is integrated with the Mutual Information (MI) to carry out the registration process. The improved performance of proposed registration measure function is demonstrated using four quality metrics: Correlation Coefficient (CC), Structural Similarity Index (SSIM), Visual Information Fidelity (VIF) and Gradient Magnitude Similarity Deviation (GMSD). The qualitative validation has also been done through visual inspection of the registered image pairs by clinical radiologists.

Results:

The experimental results showed that the proposed method outperformed the existing method (based on integrated MI and standard edge detection) for both ultrasound and MR images in terms of CC by about 4.67%, SSIM by 3.21%, VIF by 18.5%, and decreased GMSD by 37.01%. Whereas, in comparison to the standard MI based method, the proposed method has increased CC by 16.29%, SSIM by 16.13%, VIF by 52.56% and decreased GMSD by 66.06%, approximately.

Conclusion:

Thus, the proposed method improves the registration accuracy when the original images are corrupted by noise, have low intensity values or missing data.

Multi-Modal Medical Image Registration with Full or Partial Data: A Manifold Learning Approach

Multi-modal image registration is the primary step in integrating information stored in two or more images, which are captured using multiple imaging modalities. In addition to intensity variations and structural differences between images, they may have partial or full overlap, which adds an extra hurdle to the success of registration process. In this contribution, we propose a multi-modal to mono-modal transformation method that facilitates direct application of well-founded mono-modal registration methods in order to obtain accurate alignment of multi-modal images in both cases, with complete (full) and incomplete (partial) overlap. The proposed transformation facilitates recovering strong scales, rotations, and translations. We explain the method thoroughly and discuss the choice of parameters. For evaluation purposes, the effectiveness of the proposed method is examined and compared with widely used information theory-based techniques using simulated and clinical human brain images with full data. Using RIRE dataset, mean absolute error of 1.37, 1.00, and 1.41 mm are obtained for registering CT images with PD-, T1-, and T2-MRIs, respectively. In the end, we empirically investigate the efficacy of the proposed transformation in registering multi-modal partially overlapped images.

Comparison of different registration landmarks for MRI-CT fusion in radiotherapy for lung cancer with post-obstructive lobar collapse

The registration of the two sets of images based on the spine and pulmonary artery landmarks and the geometric center difference of the mean displacement in the X, Y, and Z directions (X, Y, and Z represent the directions of the body from left to right, superior to inferior, and anterior to posterior) between their MRI-CT fusions were compared, respectively. Fifty-five lung cancer patients with post-obstructive lobar collapse were enrolled in this study. Before radiation, two sets of simulating images according to the spine and the pulmonary artery registrations were obtained for each patient using MRI-CT fusion. The differences of mean displacement in the X, Y, and Z directions based on spine and pulmonary artery landmarks were of -0.29, 0.25, and 0.18 cm, respectively. The mean displacements of the pulmonary artery based images in the three directions were smaller than that in the spine registration images (P < 0.05). By the method of pulmonary artery landmark, MRI-CT has better registration accuracy and can better help confirm the target volume.

Registration methods in radiotherapy

Purpose

The aim of this study is to present a short and comprehensive review of the methods of medical image registration, their conditions and applications in radiotherapy. A particular focus was placed on the methods of deformable image registration.

Methods

To structure and deepen the knowledge on medical image registration in radiotherapy, a medical literature analysis was made using the Google Scholar browser and the medical database of the PubMed library.

Results

Chronological review of image registration methods in radiotherapy based on 34 selected articles. A particular attention was given to show: (i) potential regions of the application of different methods of registration, (ii) mathematical basis of the deformable methods and (iii) the methods of quality control for the registration process.

Conclusions

The primary aim of the medical image registration process is to connect the contents of images. What we want to achieve is a complementary or extended knowledge that can be used for more precise localisation of pathogenic lesions and continuous improvement of patient treatment. Therefore, the choice of imaging mode is dependent on the type of clinical study. It is impossible to visualise all anatomical details or functional changes using a single modality machine. Therefore, fusion of various modality images is of great clinical relevance. A natural problem in analysing the fusion of medical images is geographical errors related to displacement. The registered images are performed not at the same time and, very often, at different respiratory phases.

Survey of Non-Rigid Registration Tools in Medicine

We catalogue available software solutions for non-rigid image registration to support scientists in selecting suitable tools for specific medical registration purposes. Registration tools were identified using non-systematic search in Pubmed, Web of Science, IEEE Xplore® Digital Library, Google Scholar, and through references in identified sources (n = 22). Exclusions are due to unavailability or inappropriateness. The remaining (n = 18) tools were classified by (i) access and technology, (ii) interfaces and application, (iii) living community, (iv) supported file formats, and (v) types of registration methodologies emphasizing the similarity measures implemented. Out of the 18 tools, (i) 12 are open source, 8 are released under a permissive free license, which imposes the least restrictions on the use and further development of the tool, 8 provide graphical processing unit (GPU) support; (ii) 7 are built on software platforms, 5 were developed for brain image registration; (iii) 6 are under active development but only 3 have had their last update in 2015 or 2016; (iv) 16 support the Analyze format, while 7 file formats can be read with only one of the tools; and (v) 6 provide multiple registration methods and 6 provide landmark-based registration methods. Based on open source, licensing, GPU support, active community, several file formats, algorithms, and similarity measures, the tools Elastics and Plastimatch are chosen for the platform ITK and without platform requirements, respectively. Researchers in medical image analysis already have a large choice of registration tools freely available. However, the most recently published algorithms may not be included in the tools, yet.

Use of image registration and fusion algorithms and techniques in radiotherapy: Report of the AAPM Radiation Therapy Committee Task Group No. 132

Image registration and fusion algorithms exist in almost every software system that creates or uses images in radiotherapy. Most treatment planning systems support some form of image registration and fusion to allow the use of multimodality and time-series image data and even anatomical atlases to assist in target volume and normal tissue delineation. Treatment delivery systems perform registration and fusion between the planning images and the in-room images acquired during the treatment to assist patient positioning. Advanced applications are beginning to support daily dose assessment and enable adaptive radiotherapy using image registration and fusion to propagate contours and accumulate dose between image data taken over the course of therapy to provide up-to-date estimates of anatomical changes and delivered dose. This information aids in the detection of anatomical and functional changes that might elicit changes in the treatment plan or prescription. As the output of the image registration process is always used as the input of another process for planning or delivery, it is important to understand and communicate the uncertainty associated with the software in general and the result of a specific registration. Unfortunately, there is no standard mathematical formalism to perform this for real-world situations where noise, distortion, and complex anatomical variations can occur. Validation of the software systems performance is also complicated by the lack of documentation available from commercial systems leading to use of these systems in undesirable 'black-box' fashion. In view of this situation and the central role that image registration and fusion play in treatment planning and delivery, the Therapy Physics Committee of the American Association of Physicists in Medicine commissioned Task Group 132 to review current approaches and solutions for image registration (both rigid and deformable) in radiotherapy and to provide recommendations for quality assurance and quality control of these clinical processes.

Automatic Marker-free Longitudinal Infrared Image Registration by Shape Context Based Matching and Competitive Winner-guided Optimal Corresponding

Long-term comparisons of infrared image can facilitate the assessment of breast cancer tissue growth and early tumor detection, in which longitudinal infrared image registration is a necessary step. However, it is hard to keep markers attached on a body surface for weeks, and rather difficult to detect anatomic fiducial markers and match them in the infrared image during registration process. The proposed study, automatic longitudinal infrared registration algorithm, develops an automatic vascular intersection detection method and establishes feature descriptors by shape context to achieve robust matching, as well as to obtain control points for the deformation model. In addition, competitive winner-guided mechanism is developed for optimal corresponding. The proposed algorithm is evaluated in two ways. Results show that the algorithm can quickly lead to accurate image registration and that the effectiveness is superior to manual registration with a mean error being 0.91 pixels. These findings demonstrate that the proposed registration algorithm is reasonably accurate and provide a novel method of extracting a greater amount of useful data from infrared images.

Pacemaker Implantation in Elderly Patients: Safety of Various Regimens of Anticoagulant Therapy

OBJECTIVE

To study incidence of hemorrhagic complications after pacemaker implantation in elderly patients receiving antithrombotic therapy with warfarin or uninterrupted dabigatran.

METHODS

126 patients aged 83 [82; 85] years who receive continuous antithrombotic therapy after pacemaker implantation, were enrolled in the study. Adverse event data were collected during hospitalization and further 12 weeks.

RESULTS

95 subjects (75.4%) from general number of enrolled patients received elective anticoagulant warfarin therapy and 31 subjects (24.6%) were treated with dabigatran. All patients of dabigatran group received 220 mg/day skipping the last dose before a surgery and resumed the drug intake in 36-48 hours after it. Patients of warfarin group underwent surgery if INR was NMT 3; they didn't stop taking the drug for the duration of operation.No statistically significant differences of hematoma incidence were detected in dabigatran (incidence is 0.065, 95%CI (-0.02-0.15)) and warfarin (incidence is 0.05, 95%CI (0.006-0.01)) groups, p(Fisher)= 0.55. Three cases of nonfatal gastrointestinal bleeding (warfarin group) and 1 similar event in dabigatran group were detected during a follow-up (12 [6; 20] weeks): RR= 0.98 (warfarin group), p(Fisher)=0.68. No statistically significant difference of age, sex composition, history of IHD and diabetes was detected between groups by comparison of individual characteristics of patients whose surgeries were complicated/non-complicated by hematoma formation. Upon that, hematoma formation rate was significantly higher in patients with adjunctive pacemaker muscular fixation: 71.4% vs 31.9% (patients without hematomas), p(Fisher)= 0.045.

CONCLUSION

Incidence of hematoma formation after pacemaker implantation in patients > 75 years receiving warfarin or dabigatran, is the same as in general population of patients treated with anticoagulants. Adjunctive pacemaker muscular fixation is a significant risk factor of hematoma formation.

3D Breast Image Registration — A Review

Image registration is an important problem in breast imaging. It is used in a wide variety of applications that include better visualization of lesions on pre- and post-contrast breast MRI images, speckle tracking and image compounding in breast ultrasound images, alignment of positron emission, and standard mammography images on hybrid machines et cetera. It is a prerequisite to align images taken at different times to isolate small interval lesions. Image registration also has useful applications in monitoring cancer therapy. The field of breast image registration has gained considerable interest in recent years. While the primary focus of interest continues to be the registration of pre- and post-contrast breast MRI images, other areas like breast ultrasound registration have gained more attention in recent years. The focus of registration algorithms has also shifted from control point based semiautomated techniques, to more sophisticated voxel based automated techniques that use mutual information as a similarity measure. This paper visits the problem of breast image registration and provides an overview of the current state-of-the-art in this area.

Multimodal 7T Imaging of Thalamic Nuclei for Preclinical Deep Brain Stimulation Applications

Precise neurosurgical targeting of electrode arrays within the brain is essential to the successful treatment of a range of brain disorders with deep brain stimulation (DBS) therapy. Here, we describe a set of computational tools to generate in vivo, subject-specific atlases of individual thalamic nuclei thus improving the ability to visualize thalamic targets for preclinical DBS applications on a subject-specific basis. A sequential nonlinear atlas warping technique and a Bayesian estimation technique for probabilistic crossing fiber tractography were applied to high field (7T) susceptibility-weighted and diffusion-weighted imaging, respectively, in seven rhesus macaques. Image contrast, including contrast within thalamus from the susceptibility-weighted images, informed the atlas warping process and guided the seed point placement for fiber tractography. The susceptibility-weighted imaging resulted in relative hyperintensity of the intralaminar nuclei and relative hypointensity in the medial dorsal nucleus, pulvinar, and the medial/ventral border of the ventral posterior nuclei, providing context to demarcate borders of the ventral nuclei of thalamus, which are often targeted for DBS applications. Additionally, ascending fiber tractography of the medial lemniscus, superior cerebellar peduncle, and pallidofugal pathways into thalamus provided structural demarcation of the ventral nuclei of thalamus. The thalamic substructure boundaries were validated through in vivo electrophysiological recordings and post-mortem blockface tissue sectioning. Together, these imaging tools for visualizing and segmenting thalamus have the potential to improve the neurosurgical targeting of DBS implants and enhance the selection of stimulation settings through more accurate computational models of DBS.

MRI and CBCT image registration of temporomandibular joint: a systematic review

Purpose

The purpose of the present review is to systematically and critically analyze the available literature regarding the importance, applicability, and practicality of (MRI), computerized tomography (CT) or cone-beam CT (CBCT) image registration for TMJ anatomy and assessment.

Data sources

A systematic search of 4 databases; MEDLINE, EMBASE, EBM reviews and Scopus, was conducted by 2 reviewers. An additional manual search of the bibliography was performed.

Inclusion criteria

All articles discussing the magnetic resonance imaging MRI and CT or CBCT image registration for temporomandibular joint (TMJ) visualization or assessment were included.

Results and included articles’ characteristics

Only 3 articles satisfied the inclusion criteria. All included articles were published within the last 7 years. Two articles described MRI to CT multimodality image registration as a complementary tool to visualize TMJ. Both articles used images of one patient only to introduce the complementary concept of MRI-CT fused image. One article assessed the reliability of using MRI-CBCT registration to evaluate the TMJ disc position and osseous pathology for 10 temporomandibular disorder (TMD) patients.

Conclusion

There are very limited studies of MRI-CT/CBCT registration to reach a conclusion regarding its accuracy or clinical use in the temporomandibular joints.

Electronic supplementary material

The online version of this article (doi:10.1186/s40463-016-0144-4) contains supplementary material, which is available to authorized users.

Evaluation of Medical Image Registration Techniques Based on Nature and Domain of the Transformation

A lot of research has been done during the past 20 years in the area of medical image registration for obtaining detailed, important, and complementary information from two or more images and aligning them into a single, more informative image. Nature of the transformation and domain of the transformation are two important medical image registration techniques that deal with characters of objects (motions) in images. This article presents a detailed survey of the registration techniques that belong to both categories with detailed elaboration on their features, issues, and challenges. An investigation estimating similarity and dissimilarity measures and performance evaluation is the main objective of this work. This article also provides reference knowledge in a compact form for researchers and clinicians looking for the proper registration technique for a particular application.

Energy minimization in medical image analysis: Methodologies and applications

Energy minimization is of particular interest in medical image analysis. In the past two decades, a variety of optimization schemes have been developed. In this paper, we present a comprehensive survey of the state-of-the-art optimization approaches. These algorithms are mainly classified into two categories: continuous method and discrete method. The former includes Newton-Raphson method, gradient descent method, conjugate gradient method, proximal gradient method, coordinate descent method, and genetic algorithm-based method, while the latter covers graph cuts method, belief propagation method, tree-reweighted message passing method, linear programming method, maximum margin learning method, simulated annealing method, and iterated conditional modes method. We also discuss the minimal surface method, primal-dual method, and the multi-objective optimization method. In addition, we review several comparative studies that evaluate the performance of different minimization techniques in terms of accuracy, efficiency, or complexity. These optimization techniques are widely used in many medical applications, for example, image segmentation, registration, reconstruction, motion tracking, and compressed sensing. We thus give an overview on those applications as well.

A Stochastic Approach to Diffeomorphic Point Set Registration with Landmark Constraints

This work presents a deformable point set registration algorithm that seeks an optimal set of radial basis functions to describe the registration. A novel, global optimization approach is introduced composed of simulated annealing with a particle filter based generator function to perform the registration. It is shown how constraints can be incorporated into this framework. A constraint on the deformation is enforced whose role is to ensure physically meaningful fields (i.e., invertible). Further, examples in which landmark constraints serve to guide the registration are shown. Results on 2D and 3D data demonstrate the algorithm's robustness to noise and missing information.

A simple and efficient methodology to improve geometric accuracy in gamma knife radiation surgery: implementation in multiple brain metastases

PURPOSE

To propose, verify, and implement a simple and efficient methodology for the improvement of total geometric accuracy in multiple brain metastases gamma knife (GK) radiation surgery.

METHODS AND MATERIALS

The proposed methodology exploits the directional dependence of magnetic resonance imaging (MRI)-related spatial distortions stemming from background field inhomogeneities, also known as sequence-dependent distortions, with respect to the read-gradient polarity during MRI acquisition. First, an extra MRI pulse sequence is acquired with the same imaging parameters as those used for routine patient imaging, aside from a reversal in the read-gradient polarity. Then, "average" image data are compounded from data acquired from the 2 MRI sequences and are used for treatment planning purposes. The method was applied and verified in a polymer gel phantom irradiated with multiple shots in an extended region of the GK stereotactic space. Its clinical impact in dose delivery accuracy was assessed in 15 patients with a total of 96 relatively small (<2 cm) metastases treated with GK radiation surgery.

RESULTS

Phantom study results showed that use of average MR images eliminates the effect of sequence-dependent distortions, leading to a total spatial uncertainty of less than 0.3 mm, attributed mainly to gradient nonlinearities. In brain metastases patients, non-eliminated sequence-dependent distortions lead to target localization uncertainties of up to 1.3 mm (mean: 0.51 ± 0.37 mm) with respect to the corresponding target locations in the "average" MRI series. Due to these uncertainties, a considerable underdosage (5%-32% of the prescription dose) was found in 33% of the studied targets.

CONCLUSIONS

The proposed methodology is simple and straightforward in its implementation. Regarding multiple brain metastases applications, the suggested approach may substantially improve total GK dose delivery accuracy in smaller, outlying targets.

Validation of the CT-MRI image registration with a dedicated phantom

PURPOSE

The present study was aimed at verifying the automatic registration of the Focal (Elekta) platform with a dedicated phantom.

MATERIALS AND METHODS

A phantom that simulates the pelvis region in a stylised way and finalised to the registration of computed tomography-magnetic resonance images was designed and realised. After acquiring the two sets of images, the registration was performed both in automatic and manual mode to verify whether they were comparable. To test the repeatability of the automatic registration, some known rigid transformations were imposed to the original images. If the registration method works correctly, parameters which bring the images into alignment must always be the same.

RESULTS

Automatic registration performed by the software did not prove satisfactory, whereas if a specific tool [volume of interest (VOI) tool] allowing the calculation to be limited to the landmark region was used, the registration parameters were comparable with those of the manual registration. Regarding the repeatability of the automatic registration, the software brought the images in the correct alignment performing translations and rotations along the longitudinal axis up to 40°, while it was not satisfactory for rotations along the transverse axes.

CONCLUSION

The experimental results showed that in clinical application automatic registration is reliable if the VOI tool that includes visible landmarks in both studies is used. However, because the algorithm did not prove sensitive to rotations along the transverse axes, the position of the patient during the examinations plays a crucial role.

Geodesics-based image registration: applications to biological and medical images depicting concentric ring patterns

In many biological or medical applications, images that contain sequences of shapes are common. The existence of high inter-individual variability makes their interpretation complex. In this paper, we address the computer-assisted interpretation of such images and we investigate how we can remove or reduce these image variabilities. The proposed approach relies on the development of an efficient image registration technique. We first show the inadequacy of state-of-the-art intensity-based and feature-based registration techniques for the considered image datasets. Then, we propose a robust variational method which benefits from the geometrical information present in this type of images. In the proposed non-rigid geodesics-based registration, the successive shapes are represented by a level-set representation, which we rely on to carry out the registration. The successive level sets are regarded as elements in a shape space and the corresponding matching is that of the optimal geodesic path. The proposed registration scheme is tested on synthetic and real images. The comparison against results of state-of-the-art methods proves the relevance of the proposed method for this type of images.

Nonrigid image registration in digital subtraction angiography using multilevel B-spline

We address the problem of motion artifact reduction in digital subtraction angiography (DSA) using image registration techniques. Most of registration algorithms proposed for application in DSA, have been designed for peripheral and cerebral angiography images in which we mainly deal with global rigid motions. These algorithms did not yield good results when applied to coronary angiography images because of complex nonrigid motions that exist in this type of angiography images. Multiresolution and iterative algorithms are proposed to cope with this problem, but these algorithms are associated with high computational cost which makes them not acceptable for real-time clinical applications. In this paper we propose a nonrigid image registration algorithm for coronary angiography images that is significantly faster than multiresolution and iterative blocking methods and outperforms competing algorithms evaluated on the same data sets. This algorithm is based on a sparse set of matched feature point pairs and the elastic registration is performed by means of multilevel B-spline image warping. Experimental results with several clinical data sets demonstrate the effectiveness of our approach.

Evaluation of interpolation effects on upsampling and accuracy of cost functions-based optimized automatic image registration

Interpolation has become a default operation in image processing and medical imaging and is one of the important factors in the success of an intensity-based registration method. Interpolation is needed if the fractional unit of motion is not matched and located on the high resolution (HR) grid. The purpose of this work is to present a systematic evaluation of eight standard interpolation techniques (trilinear, nearest neighbor, cubic Lagrangian, quintic Lagrangian, hepatic Lagrangian, windowed Sinc, B-spline 3rd order, and B-spline 4th order) and to compare the effect of cost functions (least squares (LS), normalized mutual information (NMI), normalized cross correlation (NCC), and correlation ratio (CR)) for optimized automatic image registration (OAIR) on 3D spoiled gradient recalled (SPGR) magnetic resonance images (MRI) of the brain acquired using a 3T GE MR scanner. Subsampling was performed in the axial, sagittal, and coronal directions to emulate three low resolution datasets. Afterwards, the low resolution datasets were upsampled using different interpolation methods, and they were then compared to the high resolution data. The mean squared error, peak signal to noise, joint entropy, and cost functions were computed for quantitative assessment of the method. Magnetic resonance image scans and joint histogram were used for qualitative assessment of the method.

Emerging technologies for image guidance and device navigation in interventional radiology

Recent developments in image-guidance and device navigation, along with emerging robotic technologies, are rapidly transforming the landscape of interventional radiology (IR). Future state-of-the-art IR procedures may include real-time three-dimensional imaging that is capable of visualizing the target organ, interventional tools, and surrounding anatomy with high spatial and temporal resolution. Remote device actuation is becoming a reality with the introduction of novel magnetic-field enabled instruments and remote robotic steering systems. Robots offer several degrees of freedom and unprecedented accuracy, stability, and dexterity during device navigation, propulsion, and actuation. Optimization of tracking and navigation of interventional tools inside the human body will be critical in converting IR suites into the minimally invasive operating theaters of the future with increased safety and unsurpassed therapeutic efficacy. In the not too distant future, individual image guidance modalities and device tracking methods could merge into autonomous, multimodality, multiparametric platforms that offer real-time data of anatomy, morphology, function, and metabolism along with on-the-fly computational modeling and remote robotic actuation. The authors provide a concise overview of the latest developments in image guidance and device navigation, while critically envisioning what the future might hold for 2020 IR procedures.

Mono- and multimodal registration of optical breast images

Optical breast imaging offers the possibility of noninvasive, low cost, and high sensitivity imaging of breast cancers. Poor spatial resolution and a lack of anatomical landmarks in optical images of the breast make interpretation difficult and motivate registration and fusion of these data with subsequent optical images and other breast imaging modalities. Methods used for registration and fusion of optical breast images are reviewed. Imaging concerns relevant to the registration problem are first highlighted, followed by a focus on both monomodal and multimodal registration of optical breast imaging. Where relevant, methods pertaining to other imaging modalities or imaged anatomies are presented. The multimodal registration discussion concerns digital x-ray mammography, ultrasound, magnetic resonance imaging, and positron emission tomography.

Local analysis of trabecular bone fracture

Assessment of bone fracture risk is the first step in the prevention of traumatic events. In several previous study the use of bone mineral density and bone volume fraction was suggested for the identification of the failure zone, nonetheless the limits of this approach were also investigated, underling the need of other information to fully describe the failure event. In the present study, a comparison between fracture and non-fracture zones of trabecular bone is proposed with the aim of analyze the local structural differences attempting to identify the morphometrical parameters who best can describe the trabecular fracture zone. Eighteen trabecular specimens were extracted from the lower limb of two donors without skeletal disorders. All the specimens were scanned by means of a micro-CT and mechanically tested. After the mechanical compression every specimen was scanned again obtaining for every specimen two datasets: pre- and post-failure. An automatic registration scheme, comprising of a three-dimensional automatic registration method to define the differences between the two datasets, and the application of a criterion for defining "broken" or "unbroken" trabeculae, was applied for the identification of the full 3D fracture zone. The morphometrical analysis of fracture and non-fracture zone was performed by the study of several morphometrical parameters, such as bone volume fraction, off-axis angle, structural model index, connectivity density, etc. The results of the two different structures were compared by means of a Wilcoxon non-parametric test. Ten out of 12 morphometrical parameters were found statistically significantly different between fracture and non-fracture zones, underlining the strong structural difference between the two areas. Nonetheless, only three of them have shown differences superior to 30%, with a reduce overlapping of their distributions: off-axis angle, structural model index and connectivity density. On the other hand, bone volume fraction showed a smaller, even if significant, difference with great overlap of the distributions, in agreement with the limits already pointed out in the literature.

Medical image registration: a review

This paper presents a review of automated image registration methodologies that have been used in the medical field. The aim of this paper is to be an introduction to the field, provide knowledge on the work that has been developed and to be a suitable reference for those who are looking for registration methods for a specific application. The registration methodologies under review are classified into intensity or feature based. The main steps of these methodologies, the common geometric transformations, the similarity measures and accuracy assessment techniques are introduced and described.

Efficient registration of nonrigid 3-D bodies

We present a novel method to perform an accurate registration of 3-D nonrigid bodies by using phase-shift properties of the dual-tree complex wavelet transform [Formula: see text]. Since the phases of [Formula: see text] coefficients change approximately linearly with the amount of feature displacement in the spatial domain, motion can be estimated using the phase information from these coefficients. The motion estimation is performed iteratively: first by using coarser level complex coefficients to determine large motion components and then by employing finer level coefficients to refine the motion field. We use a parametric affine model to describe the motion, where the affine parameters are found locally by substituting into an optical flow model and by solving the resulting overdetermined set of equations. From the estimated affine parameters, the motion field between the sensed and the reference data sets can be generated, and the sensed data set then can be shifted and interpolated spatially to align with the reference data set.

RIGID MEDICAL IMAGE REGISTRATION AND ITS ASSOCIATION WITH MUTUAL INFORMATION

Image registration plays a crucial role in the computer vision and medical imaging field where it is used to develop a spatial mapping between different sets of data. These transformations can range from simple rigid registrations to complex nonrigid deformations. Mutual information (MI) is a popular entropy-based similarity measure which has recently experienced a prolific expansion in a number of image registration applications. Stemming from information theory, this measure generally outperforms most other intensity-based measures in multimodal applications as it only assumes a statistical dependence between images. This paper provides a thorough introduction to the MI measure and its use in rigid medical image registration. A look at the extensions proposed to the original measure will also be provided. These were developed to improve the robustness of the measure and to avoid certain cases when maximizing MI does not lead to the correct spatial alignment.

Microwave ablation assisted by a computerised tomography-ultrasonography fusion imaging system for liver lesions: an ex vivo experimental study

PURPOSE

To investigate the feasibility and validity of real-time guidance using a fusion imaging system that combines ultrasonography (US) and computerised tomography (CT) information in the targeting and subsequent microwave ablation of a liver target inconspicuous on US.

MATERIALS AND METHODS

The study was an experimental ex vivo study in calf livers with internal targets, simulating a focal liver lesion, focused on the accuracy of real-time US using a multimodality fusion imaging system in combination with 15 gauge (G) microwave antennae. US image and pre-procedural CT image were fused by the external markers registration procedure. Microwave antennae were inserted into the liver to ablate the target assisted by the CT-US fusion imaging system. Finally, a post-procedural CT with needles in situ and multiplanar reconstructions were performed to compare with pre-procedural CT information in order to calculate the accuracy of positioning (distance between the needle tip and the target).

RESULTS

Eight insertions were planned and eight ablations were performed in four calf livers. The calf livers were undertaken successfully on external markers registration procedure. The mean registration error in the four livers was 2.1 ± 0.1 mm, 2.8 ± 0.1 mm, 3.4 ± 0.1 mm and 2.3 ± 0.1 mm, respectively. The accuracy of the matched US-CT images was very satisfactory in the fact that it was found there was a mean discrepancy of 1.63 ± 1.06 mm.

CONCLUSION

Real-time registration and fusion of pre-procedural CT volume images with intraprocedural US is feasible and accurate for microwave (MW) ablation in experimental setting. Further studies are warranted to validate the system under clinical conditions.

Microwave ablation assisted by a real-time virtual navigation system for hepatocellular carcinoma undetectable by conventional ultrasonography

OBJECTIVES

To evaluate the efficiency and feasibility of microwave (MW) ablation assisted by a real-time virtual navigation system for hepatocellular carcinoma (HCC) undetectable by conventional ultrasonography.

METHODS

18 patients with 18 HCC nodules (undetectable on conventional US but detectable by intravenous contrast-enhanced CT or MRI) were enrolled in this study. Before MW ablation, US images and MRI or CT images were synchronized using the internal markers at the best timing of the inspiration. Thereafter, MW ablation was performed under real-time virtual navigation system guidance. Therapeutic efficacy was assessed by the result of contrast-enhanced imagings after the treatment.

RESULTS

The target HCC nodules could be detected with fusion images in all patients. The time required for image fusion was 8-30 min (mean, 13.3 ± 5.7 min). 17 nodules were successfully ablated according to the contrast enhanced imagings 1 month after ablation. The technique effectiveness rate was 94.44% (17/18). The follow-up time was 3-12 months (median, 6 months) in our study. No severe complications occurred. No local recurrence was observed in any patients.

CONCLUSIONS

MW ablation assisted by a real-time virtual navigation system is a feasible and efficient treatment of patients with HCC undetectable by conventional ultrasonography.

Algorithm for registration of full Scanning Laser Ophthalmoscope video sequences

Fluorescein angiography is an established technique for examining the functional integrity of the retinal microcirculation for early detection of changes due to retinopathy. This paper describes a new method for the registration of large Scanning Laser Ophthalmoscope sequences (SLO), where the patient has been injected with a fluorescent dye. This allows the measurement of parameters such as the arteriovenous passage time. Due to the long time needed to acquire these sequences, there will inevitably be eye movement, which must be corrected prior to the application of quantitative analysis. The algorithm described here combines mutual information-based registration and landmark-based registration. The former will allow the alignment of the darkest frames of the sequence, where the dye has not still arrived to the retina, because of its ability to work with images without a preprocessing or segmentation, while the latter uses relevant features (the vessels) extracted by means of a robust creaseness operator, to get a very fast and accurate registration. The algorithm only detects rigid transformations but proves to be robust against the slight alterations derived from the eye location perspective during acquisition. Results were validated by expert clinicians.

Inflammatory breast cancer: the disease, the biology, the treatment

Inflammatory breast cancer (IBC) is a rare and aggressive form of invasive breast cancer accounting for 2.5% of all breast cancer cases. It is characterized by rapid progression, local and distant metastases, younger age of onset, and lower overall survival compared with other breast cancers. Historically, IBC is a lethal disease with less than a 5% survival rate beyond 5 years when treated with surgery or radiation therapy. Because of its rarity, IBC is often misdiagnosed as mastitis or generalized dermatitis. This review examines IBC's unique clinical presentation, pathology, epidemiology, imaging, and biology and details current multidisciplinary management of the disease, which comprises systemic therapy, surgery, and radiation therapy.

Accurate Coregistration between Ultra-High-Resolution Micro-SPECT and Circular Cone-Beam Micro-CT Scanners

Introduction. Spatially registering SPECT with CT makes it possible to anatomically localize SPECT tracers. In this study, an accurate method for the coregistration of ultra-high-resolution SPECT volumes and multiple cone-beam CT volumes is developed and validated, which does not require markers during animal scanning. Methods. Transferable animal beds were developed with an accurate mounting interface. Simple calibration phantoms make it possible to obtain both the spatial transformation matrix for stitching multiple CT scans of different parts of the animal and to register SPECT and CT. The spatial transformation for image coregistration is calculated once using Horn's matching algorithm. Animal images can then be coregistered without using markers. Results. For mouse-sized objects, average coregistration errors between SPECT and CT in X, Y, and Z directions are within 0.04 mm, 0.10 mm, and 0.19 mm, respectively. For rat-sized objects, these numbers are 0.22 mm, 0.14 mm, and 0.28 mm. Average 3D coregistration errors were within 0.24 mm and 0.42 mm for mouse and rat imaging, respectively. Conclusion. Extending the field-of-view of cone-beam CT by stitching is improved by prior registration of the CT volumes. The accuracy of registration between SPECT and CT is typically better than the image resolution of current ultra-high-resolution SPECT.

Clinical evaluation of spatial accuracy of a fusion imaging technique combining previously acquired computed tomography and real-time ultrasound for imaging of liver metastases

PURPOSE

This study was designed to evaluate the spatial accuracy of matching volumetric computed tomography (CT) data of hepatic metastases with real-time ultrasound (US) using a fusion imaging system (VNav) according to different clinical settings.

METHODS

Twenty-four patients with one hepatic tumor identified on enhanced CT and US were prospectively enrolled. A set of three landmarks markers was chosen on CT and US for image registration. US and CT images were then superimposed using the fusion imaging display mode. The difference in spatial location between the tumor visible on the CT and the US on the overlay images (reviewer #1, comment #2) was measured in the lateral, anterior-posterior, and vertical axis. The maximum difference (Dmax) was evaluated for different predictive factors. CT performed 1-30 days before registration versus immediately before. Use of general anesthesia for CT and US versus no anesthesia. Anatomic landmarks versus landmarks that include at least one nonanatomic structure, such as a cyst or a calcification

RESULTS

Overall, Dmax was 11.53 ± 8.38 mm. Dmax was 6.55 ± 7.31 mm with CT performed immediately before VNav versus 17.4 ± 5.18 with CT performed 1-30 days before (p < 0.0001). Dmax was 7.05 ± 6.95 under general anesthesia and 16.81 ± 6.77 without anesthesia (p < 0.0015). Landmarks including at least one nonanatomic structure increase Dmax of 5.2 mm (p < 0.0001). The lowest Dmax (1.9 ± 1.4 mm) was obtained when CT and VNav were performed under general anesthesia, one immediately after the other.

CONCLUSIONS

VNav is accurate when adequate clinical setup is carefully selected. Only under these conditions (reviewer #2), liver tumors not identified on US can be accurately targeted for biopsy or radiofrequency ablation using fusion imaging.

Integration of multimodality imaging and surgical navigation in the management of patients with refractory epilepsy. A pilot study using a new minimally invasive reference and head-fixation system

OBJECTIVE

To review the experience with a new system (VBH system) for minimally invasive frameless stereotactic guidance, acting as a common platform to provide multimodal image integration and surgical navigation in a consecutive series of 25 patients who underwent surgery for drug-resistant seizures.

METHODS

The usefulness of the VBH system for integrating all images to produce one dataset and for intraoperative instrument guidance and navigation was judged semiquantitatively in a three-tiered scale (+, ++, +++). Seizure outcome was classified according to Engel.

RESULTS

The presurgical evaluation extended over 21.2 months (mean). A total of 141 registrations of images were performed (mean 5.6 per patient, range: 2 to 16). In 19 (76%) of 25 patients structural data fused with functional data were used for the presurgical workup. Six patients proceeded directly to navigated resection. Nineteen patients (76%) underwent invasive recording, of whom 13 underwent resective surgery. In seven patients (28%) the combination of multimodal image fusion and intra-operative stereotactic guidance was judged "essential" (+++) to remove the epileptogenic zone. Integration of all images to form one dataset was "essential" (+++) for decision making in 15 and "helpful" (++) in 4 patients (overall 76% of patients). Intraoperative use of frameless neuronavigation was "essential" (+++) in ten and "helpful" (++) in all remaining patients. Eighty percent of the patients achieved satisfactory seizure outcome after 1 year.

CONCLUSION

The VBH system is a safe and effective non-invasive tool for repetitive imaging, multimodal image fusion and frameless stereotactic surgical navigation in candidates for epilepsy surgery.