Automated segmentation of a motion mask to preserve sliding motion in deformable registration of thoracic CT

Improving hybrid image and structure-based deformable image registration for large internal deformations

Objective.Deformable image registration (DIR) is a widely used technique in radiotherapy. Complex deformations, resulting from large anatomical changes, are a regular challenge. DIR algorithms generally seek a balance between capturing large deformations and preserving a smooth deformation vector field (DVF). We propose a novel structure-based term that can enhance the registration efficacy while ensuring a smooth DVF.Approach.The proposed novel similarity metric for controlling structures was introduced as a new term into a commercially available algorithm. Its performance was compared to the original algorithm using a dataset of 46 patients who received pelvic re-irradiation, many of which exhibited complex deformations.Main results.The mean Dice Similarity Coefficient (DSC) under the improved algorithm was 0.96, 0.94, 0.76, and 0.91 for bladder, rectum, colon, and bone respectively, compared to 0.69, 0.89, 0.62, and 0.88 for the original algorithm. The improvement was more pronounced for complex deformations.Significance.With this work, we have demonstrated that the proposed term is able to improve registration accuracy for complex cases while maintaining realistic deformations.

Hierarchical anatomical structure-aware based thoracic CT images registration

Accurate thoracic CT image registration remains challenging due to complex joint deformations and different motion patterns in multiple organs/tissues during breathing. To combat this, we devise a hierarchical anatomical structure-aware based registration framework. It affords a coordination scheme necessary for constraining a general free-form deformation (FFD) during thoracic CT registration. The key is to integrate the deformations of different anatomical structures in a divide-and-conquer way. Specifically, a deformation ability-aware dissimilarity metric is proposed for complex joint deformations containing large-scale flexible deformation of the lung region, rigid displacement of the bone region, and small-scale flexible deformation of the rest region. Furthermore, a motion pattern-aware regularization is devised to handle different motion patterns, which contain sliding motion along the lung surface, almost no displacement of the spine and smooth deformation of other regions. Moreover, to accommodate large-scale deformation, a novel hierarchical strategy, wherein different anatomical structures are fused on the same control lattice, registers images from coarse to fine via elaborate Gaussian pyramids. Extensive experiments and comprehensive evaluations have been executed on the 4D-CT DIR and 3D DIR COPD datasets. It confirms that this newly proposed method is locally comparable to state-of-the-art registration methods specializing in local deformations, while guaranteeing overall accuracy. Additionally, in contrast to the current popular learning-based methods that typically require dozens of hours or more pre-training with powerful graphics cards, our method only takes an average of 63 s to register a case with an ordinary graphics card of RTX2080 SUPER, making our method still worth promoting. Our code is available at https://github.com/heluxixue/Structure_Aware_Registration/tree/master.

Improving motion-mask segmentation in thoracic CT with multiplanar U-nets

PURPOSE

Motion-mask segmentation from thoracic computed tomography (CT) images is the process of extracting the region that encompasses lungs and viscera, where large displacements occur during breathing. It has been shown to help image registration between different respiratory phases. This registration step is, for example, useful for radiotherapy planning or calculating local lung ventilation. Knowing the location of motion discontinuity, that is, sliding motion near the pleura, allows a better control of the registration preventing unrealistic estimates. Nevertheless, existing methods for motion-mask segmentation are not robust enough to be used in clinical routine. This article shows that it is feasible to overcome this lack of robustness by using a lightweight deep-learning approach usable on a standard computer, and this even without data augmentation or advanced model design.

METHODS

A convolutional neural-network architecture with three 2D U-nets for the three main orientations (sagittal, coronal, axial) was proposed. Predictions generated by the three U-nets were combined by majority voting to provide a single 3D segmentation of the motion mask. The networks were trained on a database of nonsmall cell lung cancer 4D CT images of 43 patients. Training and evaluation were done with a K-fold cross-validation strategy. Evaluation was based on a visual grading by two experts according to the appropriateness of the segmented motion mask for the registration task, and on a comparison with motion masks obtained by a baseline method using level sets. A second database (76 CT images of patients with early-stage COVID-19), unseen during training, was used to assess the generalizability of the trained neural network.

RESULTS

The proposed approach outperformed the baseline method in terms of quality and robustness: the success rate increased from 53 % to 79 % without producing any failure. It also achieved a speed-up factor of 60 with GPU, or 17 with CPU. The memory footprint was low: less than 5 GB GPU RAM for training and less than 1 GB GPU RAM for inference. When evaluated on a dataset with images differing by several characteristics (CT device, pathology, and field of view), the proposed method improved the success rate from 53 % to 83 % .

CONCLUSION

With 5-s processing time on a mid-range GPU and success rates around 80 % , the proposed approach seems fast and robust enough to be routinely used in clinical practice. The success rate can be further improved by incorporating more diversity in training data via data augmentation and additional annotated images from different scanners and diseases. The code and trained model are publicly available.

Projection-based dynamic tomography

Objective. This paper proposes a 4D dynamic tomography framework that allows a moving sample to be imaged in a tomograph as well as the associated space-time kinematics to be measured with nothing more than a single conventional x-ray scan.Approach. The method exploits the consistency of the projection/reconstruction operations through a multi-scale procedure. The procedure is composed of two main parts solved alternatively: a motion-compensated reconstruction algorithm and a projection-based measurement procedure that estimates the displacement field directly on each projection.Main results. The method is applied to two studies: a numerical simulation of breathing from chest computed tomography (4D-CT) and a clinical cone-beam CT of a breathing patient acquired for image guidance of radiotherapy. The reconstructed volume, initially blurred by the motion, is cleaned from motion artifacts.Significance. Applying the proposed approach results in an improved reconstruction quality showing sharper edges and finer details.

Detection of vessel bifurcations in CT scans for automatic objective assessment of deformable image registration accuracy

PURPOSE

Objective assessment of deformable image registration (DIR) accuracy often relies on the identification of anatomical landmarks in image pairs, a manual process known to be extremely time-expensive. The goal of this study is to propose a method to automatically detect vessel bifurcations in images and assess their use for the computation of target registration errors (TREs).

MATERIALS AND METHODS

Three image datasets were retrospectively analyzed. The first dataset included 10 pairs of inhale/exhale phases from lung 4DCTs and full inhale and exhale breath-hold CT scans from 10 patients presenting with chronic obstructive pulmonary disease, with 300 corresponding landmarks available for each case (DIR-Lab). The second dataset included six pairs of inhale/exhale phases from lung 4DCTs (POPI dataset), with 100 pairs of landmarks for each case. The third dataset included 28 pairs of pre/post-radiotherapy liver contrast-enhanced CT scans, each with five manually picked vessel bifurcation correspondences. For all images, the vasculature was autosegmented by computing and thresholding a vesselness image. Images of the vasculature centerline were computed, and bifurcations were detected based on centerline voxel neighbors' count. The vasculature segmentations were independently registered using a Demons algorithm between representations of their surface with distance maps. Detected bifurcations were considered as corresponding when distant by less than 5 mm after vasculature DIR. The selected pairs of bifurcations were used to calculate TRE after registration of the images considering three algorithms: rigid registration, Anaconda, and a Demons algorithm. For comparison with the ground truth, TRE values calculated using the automatically detected correspondences were interpolated in the whole organs to generate TRE maps. The performance of the method in automatically calculating TRE after image registration was quantified by measuring the correlation with the TRE obtained when using the ground truth landmarks.

RESULTS

The median Pearson correlation coefficients between ground truth TRE and corresponding values in the generated TRE maps were r = 0.81 and r = 0.67 for the lung and liver cases, respectively. The correlation coefficients between mean TRE for each case were r = 0.99 and r = 0.64 for the lung and liver cases, respectively.

CONCLUSION

For lungs or liver CT scans DIR, a strong correlation was obtained between TRE calculated using manually picked or landmarks automatically detected with the proposed method. This tool should be particularly useful in studies requiring assessing the reliability of a high number of DIRs.

A multivariable study of deformable image registration evaluation metrics in 4DCT of thoracic cancer patients

Deformable image registration (DIR) accuracy is often validated using manually identified landmarks or known deformations generated using digital or physical phantoms. In daily practice, the application of these approaches is limited since they are time-consuming or require additional equipment. An alternative is the use of metrics automatically derived from the registrations, but their interpretation is not straightforward. In this work we aim to determine the suitability of DIR-derived metrics to validate the accuracy of 4 commonly used DIR algorithms. First, we investigated the DIR accuracy using a landmark-based metric (target registration error (TRE)) and a digital phantom-based metric (known deformation recovery error (KDE)). 4DCT scans of 16 thoracic cancer patients along with corresponding pairwise anatomical landmarks (AL) locations were collected from two public databases. Digital phantoms with known deformations were generated by each DIR algorithm to test all other algorithms and compute KDE. TRE and KDE were evaluated at AL. KDE was additionally quantified in coordinates randomly sampled (RS) inside the lungs. Second, we investigated the associations of 5 DIR-derived metrics (distance discordance metric (DDM), inverse consistency error (ICE), transitivity (TE), spatial (SS) and temporal smoothness (TS)) with DIR accuracy through uni- and multivariable linear regression models. TRE values were found higher compared to KDE values and these varied depending on the phantom used. The algorithm with the best accuracy achieved average values of TRE = 1.1 mm and KDE ranging from 0.3 to 0.8 mm. DDM was the best predictor of DIR accuracy, with moderate correlations (R ² < 0.61). Poor correlations were obtained at AL for algorithms with better accuracy, which improved when evaluated at RS. Only slight correlation improvement was obtained with a multivariable analysis (R ² < 0.64). DDM can be a useful metric to identify inaccuracies for different DIR algorithms without employing landmarks or digital phantoms.

Tissue-specific deformable image registration using a spatial-contextual filter

Intensity-based deformable registration with spatial-invariant regularization generally fails when distinct motion exists across different types of tissues. The purpose of this work was to develop and validate a new regularization approach for deformable image registration that is tissue-specific and able to handle motion discontinuities. Our approach was built upon a Demons registration framework, and used the image context supplementing the original spatial constraint to regularize displacement vector fields in iterative image registration process. The new regularization was implemented as a spatial-contextual filter, which favors the motion vectors within the same tissue type but penalizes the motion vectors from different tissues. This approach was validated using five public lung cancer patients, each with 300 landmark pairs identified by a thoracic radiation oncologist. The mean and standard deviation of the landmark registration errors were 1.3 ± 0.8 mm, compared with those of 2.3 ± 2.9 mm using the original Demons algorithm. Particularly, for the case with the largest initial landmark displacement of 15 ± 9 mm, the modified Demons algorithm had a registration error of 1.3 ± 1.1 mm, while the original Demons algorithm had a registration error of 3.6 ± 5.9 mm. We also qualitatively evaluated the modified Demons algorithm using two difficult cases in our routine clinic: one lung case with large sliding motion and one head and neck case with large anatomical changes in air cavity. Visual evaluation on the deformed image created by the deformable image registration showed that the modified Demons algorithm achieved reasonable registration accuracy, but the original Demons algorithm produced distinct registration errors.

Consistent and invertible deformation vector fields for a breathing anthropomorphic phantom: a post-processing framework for the XCAT phantom

Breathing motion is challenging for radiotherapy planning and delivery. This requires advanced four-dimensional (4D) imaging and motion mitigation strategies and associated validation tools with known deformations. Numerical phantoms such as the XCAT provide reproducible and realistic data for simulation-based validation. However, the XCAT generates partially inconsistent and non-invertible deformations where tumours remain rigid and structures can move through each other. We address these limitations by post-processing the XCAT deformation vector fields (DVF) to generate a breathing phantom with realistic motion and quantifiable deformation. An open-source post-processing framework was developed that corrects and inverts the XCAT-DVFs while preserving sliding motion between organs. Those post-processed DVFs are used to warp the first XCAT-generated image to consecutive time points providing a 4D phantom with a tumour that moves consistently with the anatomy, the ability to scale lung density as well as consistent and invertible DVFs. For a regularly breathing case, the inverse consistency of the DVFs was verified and the tumour motion was compared to the original XCAT. The generated phantom and DVFs were used to validate a motion-including dose reconstruction (MIDR) method using isocenter shifts to emulate rigid motion. Differences between the reconstructed doses with and without lung density scaling were evaluated. The post-processing framework produced DVFs with a maximum [Formula: see text]-percentile inverse-consistency error of 0.02 mm. The generated phantom preserved the dominant sliding motion between the chest wall and inner organs. The tumour of the original XCAT phantom preserved its trajectory while deforming consistently with the underlying tissue. The MIDR was compared to the ground truth dose reconstruction illustrating its limitations. MIDR with and without lung density scaling resulted in small dose differences up to 1 Gy (prescription 54 Gy). The proposed open-source post-processing framework overcomes important limitations of the original XCAT phantom and makes it applicable to a wider range of validation applications within radiotherapy.

Ventilation heterogeneity in smokers: role of unequal lung expansion and peripheral lung structure

Smoking-induced ventilation heterogeneity measured at the mouth via established washout indices [lung clearance index (LCI) and alveolar mixing efficiency (AME)] potentially results from unequal expansion, which can be quantified by computer tomography (CT), and structural changes down to the lung periphery, characterized by CT parametric response mapping indices [percentage of lung affected by functional small airway disease (PRM^fSAD) and emphysema (PRM^Emph)]. By combining CT imaging and nitrogen (N₂) washout tests in smokers, we specifically examined the roles of unequal lung expansion and peripheral structure. We first extracted three-dimensional maps of local lung expansion from registered inspiratory/expiratory CT images in 50 smokers (GOLD 0-IV) to compute for each smoker the theoretical N₂ washout concentration curve solely attributable to unequal local expansion. By a head-on comparison with washout N₂ concentrations measured at the mouth in the same smokers supine, we observed that 1) LCI increased from 4.8 ± 0.2 (SD) to 6.6 ± 0.8 (SD) due to unequal lung expansion alone and further increased to 9.0 ± 1.5 (SD) independent of local expansion and 2) AME decreased (from 100% by definition) to 95 ± 2 (SD)% due to unequal expansion alone and further decreased to 75 ± 7(SD)% independent of local expansion. In a multiple regression between the washout indices and CT-derived PRM^fSAD and PRM^Emph, LCI was related to PRM^fSAD (r = +0.58; P < 0.001), whereas AME was related to both PRM^fSAD (r_partial = -0.44; P = 0.002) and PRM^Emph (r_partial = -0.31; P = 0.033), in line with AME being dominated by alterations in peripheral structure. We conclude that smokers showing an increased LCI without corresponding AME decrease are predominantly affected by unequal lung expansion, whereas an AME decrease with a commensurate LCI increase indicates a smoking-induced alteration of peripheral structure.NEW & NOTEWORTHY A head-on comparison between imaging and multiple breath washout in supine smokers shows that computer tomography-measured unequal local lung expansion accounts for 50% or less of smoking-induced increase in ventilation heterogeneity. The contributions from unequal lung expansion and peripheral structure to the two main washout indices also explain their respective association with parametric response mapping indices.

Mid-position treatment strategy for locally advanced lung cancer: a dosimetric study

OBJECTIVE

The internal target volume (ITV) strategy generates larger planning target volumes (PTVs) in locally advanced non-small cell lung cancer (LA-NSCLC) than the Mid-position (Mid-p) strategy. We investigated the benefit of the Mid-p strategy regarding PTV reduction and dose to the organs at risk (OARs).

METHODS

44 patients with LA-NSCLC were included in a randomized clinical study to compare ITV and Mid-p strategies. GTV were delineated by a physician on maximum intensity projection images and on Mid-p images from four-dimensional CTs. CTVs were obtained by adding 6 mm uniform margin for microscopic extension. CTV to PTV margins were calculated using the van Herk's recipe for setup and delineation errors. For the Mid-p strategy, the mean target motion amplitude was added as a random error. For both strategies, three-dimensional conformal plans delivering 60-66 Gy to PTV were performed. PTVs, dose-volume parameters for OARs (lung, esophagus, heart, spinal cord) were reported and compared.

RESULTS

With the Mid-p strategy, the median of volume reduction was 23.5 cm3 (p = 0.012) and 8.8 cm3 (p = 0.0083) for PTVT and PTVN respectively; the median mean lung dose reduction was 0.51 Gy (p = 0.0057). For 37.1% of the patients, delineation errors led to smaller PTV with the ITV strategy than with the Mid-p strategy.

CONCLUSION

PTV and mean lung dose were significantly reduced using the Mid-p strategy. Delineation uncertainty can unfavorably impact the advantage.

ADVANCES IN KNOWLEDGE

To the best of our knowledge, this is the first dosimetric comparison study between ITV and Mid-p strategies for LA-NSCLC.

LungRegNet: An unsupervised deformable image registration method for 4D-CT lung

PURPOSE

To develop an accurate and fast deformable image registration (DIR) method for four-dimensional computed tomography (4D-CT) lung images. Deep learning-based methods have the potential to quickly predict the deformation vector field (DVF) in a few forward predictions. We have developed an unsupervised deep learning method for 4D-CT lung DIR with excellent performances in terms of registration accuracies, robustness, and computational speed.

METHODS

A fast and accurate 4D-CT lung DIR method, namely LungRegNet, was proposed using deep learning. LungRegNet consists of two subnetworks which are CoarseNet and FineNet. As the name suggests, CoarseNet predicts large lung motion on a coarse scale image while FineNet predicts local lung motion on a fine scale image. Both the CoarseNet and FineNet include a generator and a discriminator. The generator was trained to directly predict the DVF to deform the moving image. The discriminator was trained to distinguish the deformed images from the original images. CoarseNet was first trained to deform the moving images. The deformed images were then used by the FineNet for FineNet training. To increase the registration accuracy of the LungRegNet, we generated vessel-enhanced images by generating pulmonary vasculature probability maps prior to the network prediction.

RESULTS

We performed fivefold cross validation on ten 4D-CT datasets from our department. To compare with other methods, we also tested our method using separate 10 DIRLAB datasets that provide 300 manual landmark pairs per case for target registration error (TRE) calculation. Our results suggested that LungRegNet has achieved better registration accuracy in terms of TRE than other deep learning-based methods available in the literature on DIRLAB datasets. Compared to conventional DIR methods, LungRegNet could generate comparable registration accuracy with TRE smaller than 2 mm. The integration of both the discriminator and pulmonary vessel enhancements into the network was crucial to obtain high registration accuracy for 4D-CT lung DIR. The mean and standard deviation of TRE were 1.00 ± 0.53 mm and 1.59 ± 1.58 mm on our datasets and DIRLAB datasets respectively.

CONCLUSIONS

An unsupervised deep learning-based method has been developed to rapidly and accurately register 4D-CT lung images. LungRegNet has outperformed its deep-learning-based peers and achieved excellent registration accuracy in terms of TRE.

Pulmonary CT Registration Through Supervised Learning With Convolutional Neural Networks

Deformable image registration can be time consuming and often needs extensive parameterization to perform well on a specific application. We present a deformable registration method based on a 3-D convolutional neural network, together with a framework for training such a network. The network directly learns transformations between pairs of 3-D images. The network is trained on synthetic random transformations which are applied to a small set of representative images for the desired application. Training, therefore, does not require manually annotated ground truth information on the deformation. The framework for the generation of transformations for training uses a sequence of multiple transformations at different scales that are applied to the image. This way, complex transformations with large displacements can be modeled without folding or tearing images. The methodology is demonstrated on public data sets of inhale-exhale lung CT image pairs which come with landmarks for evaluation of the registration quality. We show that a small training set can be used to train the network, while still allowing generalization to a separate pulmonary CT data set containing data from a different patient group, acquired using a different scanner and scan protocol. This approach results in an accurate and very fast deformable registration method, without a requirement for parameterization at test time or manually annotated data for training.

Deep convolutional neural network for segmentation of thoracic organs-at-risk using cropped 3D images

PURPOSE

Automatic segmentation of organs-at-risk (OARs) is a key step in radiation treatment planning to reduce human efforts and bias. Deep convolutional neural networks (DCNN) have shown great success in many medical image segmentation applications but there are still challenges in dealing with large 3D images for optimal results. The purpose of this study is to develop a novel DCNN method for thoracic OARs segmentation using cropped 3D images.

METHODS

To segment the five organs (left and right lungs, heart, esophagus and spinal cord) from the thoracic CT scans, preprocessing to unify the voxel spacing and intensity was first performed, a 3D U-Net was then trained on resampled thoracic images to localize each organ, then the original images were cropped to only contain one organ and served as the input to each individual organ segmentation network. The segmentation maps were then merged to get the final results. The network structures were optimized for each step, as well as the training and testing strategies. A novel testing augmentation with multiple iterations of image cropping was used. The networks were trained on 36 thoracic CT scans with expert annotations provided by the organizers of the 2017 AAPM Thoracic Auto-segmentation Challenge and tested on the challenge testing dataset as well as a private dataset.

RESULTS

The proposed method earned second place in the live phase of the challenge and first place in the subsequent ongoing phase using a newly developed testing augmentation approach. It showed superior-than-human performance on average in terms of Dice scores (spinal cord: 0.893 ± 0.044, right lung: 0.972 ± 0.021, left lung: 0.979 ± 0.008, heart: 0.925 ± 0.015, esophagus: 0.726 ± 0.094), mean surface distance (spinal cord: 0.662 ± 0.248 mm, right lung: 0.933 ± 0.574 mm, left lung: 0.586 ± 0.285 mm, heart: 2.297 ± 0.492 mm, esophagus: 2.341 ± 2.380 mm) and 95% Hausdorff distance (spinal cord: 1.893 ± 0.627 mm, right lung: 3.958 ± 2.845 mm, left lung: 2.103 ± 0.938 mm, heart: 6.570 ± 1.501 mm, esophagus: 8.714 ± 10.588 mm). It also achieved good performance in the private dataset and reduced the editing time to 7.5 min per patient following automatic segmentation.

CONCLUSIONS

The proposed DCNN method demonstrated good performance in automatic OAR segmentation from thoracic CT scans. It has the potential for eventual clinical adoption of deep learning in radiation treatment planning due to improved accuracy and reduced cost for OAR segmentation.

Discontinuity Preserving Liver MR Registration with 3D Active Contour Motion Segmentation

OBJECTIVE

The sliding motion of the liver during respiration violates the homogeneous motion smoothness assumption in conventional non-rigid image registration and commonly results in compromised registration accuracy. This paper presents a novel approach, registration with 3D active contour motion segmentation (RAMS), to improve registration accuracy with discontinuity-aware motion regularization.

METHODS

A Markov random field-based discrete optimization with dense displacement sampling and self-similarity context metric is used for registration, while a graph cuts-based 3D active contour approach is applied to segment the sliding interface. In the first registration pass, a mask-free L1 regularization on an image-derived minimum spanning tree is performed to allow motion discontinuity. Based on the motion field estimates, a coarse segmentation finds the motion boundaries. Next, based on MR signal intensity, a fine segmentation aligns the motion boundaries with anatomical boundaries. In the second registration pass, smoothness constraints across the segmented sliding interface are removed by masked regularization on a minimum spanning forest and masked interpolation of the motion field.

RESULTS

For in vivo breath-hold abdominal MRI data, the motion masks calculated by RAMS are highly consistent with manual segmentations in terms of Dice similarity and bidirectional local distance measure. These automatically obtained masks are shown to substantially improve registration accuracy for both the proposed discrete registration as well as conventional continuous non-rigid algorithms.

CONCLUSION/SIGNIFICANCE

The presented results demonstrated the feasibility of automated segmentation of the respiratory sliding motion interface in liver MR images and the effectiveness of using the derived motion masks to preserve motion discontinuity.

Predicted airway obstruction distribution based on dynamical lung ventilation data: A coupled modeling-machine learning methodology

In asthma and chronic obstructive pulmonary disease, some airways of the tracheobronchial tree can be constricted, from moderate narrowing up to closure. Those pathological patterns of obstructions affect the lung ventilation distribution. While some imaging techniques enable visualization and quantification of constrictions in proximal generations, no noninvasive technique exists to provide the airway morphology and obstruction distribution in distal areas. In this work, we propose a method that exploits lung ventilation measures to access positions of airway obstructions (restrictions and closures) in the tree. This identification approach combines a lung ventilation model, in which a 0D tree is strongly coupled to a 3D parenchyma description, along with a machine learning approach. On the basis of synthetic data generated with typical temporal and spatial resolutions as well as reconstruction errors, we obtain very encouraging results of the obstruction distribution, with a detection rate higher than 85%.

Error estimation of deformable image registration of pulmonary CT scans using convolutional neural networks

Error estimation in nonlinear medical image registration is a nontrivial problem that is important for validation of registration methods. We propose a supervised method for estimation of registration errors in nonlinear registration of three-dimensional (3-D) images. The method is based on a 3-D convolutional neural network that learns to estimate registration errors from a pair of image patches. By applying the network to patches centered around every voxel, we construct registration error maps. The network is trained using a set of representative images that have been synthetically transformed to construct a set of image pairs with known deformations. The method is evaluated on deformable registrations of inhale-exhale pairs of thoracic CT scans. Using ground truth target registration errors on manually annotated landmarks, we evaluate the method's ability to estimate local registration errors. Estimation of full domain error maps is evaluated using a gold standard approach. The two evaluation approaches show that we can train the network to robustly estimate registration errors in a predetermined range, with subvoxel accuracy. We achieved a root-mean-square deviation of 0.51 mm from gold standard registration errors and of 0.66 mm from ground truth landmark registration errors.

GIFTed Demons: deformable image registration with local structure-preserving regularization using supervoxels for liver applications

Deformable image registration, a key component of motion correction in medical imaging, needs to be efficient and provides plausible spatial transformations that reliably approximate biological aspects of complex human organ motion. Standard approaches, such as Demons registration, mostly use Gaussian regularization for organ motion, which, though computationally efficient, rule out their application to intrinsically more complex organ motions, such as sliding interfaces. We propose regularization of motion based on supervoxels, which provides an integrated discontinuity preserving prior for motions, such as sliding. More precisely, we replace Gaussian smoothing by fast, structure-preserving, guided filtering to provide efficient, locally adaptive regularization of the estimated displacement field. We illustrate the approach by applying it to estimate sliding motions at lung and liver interfaces on challenging four-dimensional computed tomography (CT) and dynamic contrast-enhanced magnetic resonance imaging datasets. The results show that guided filter-based regularization improves the accuracy of lung and liver motion correction as compared to Gaussian smoothing. Furthermore, our framework achieves state-of-the-art results on a publicly available CT liver dataset.

Supervoxels for Graph Cuts-Based Deformable Image Registration Using Guided Image Filtering

In this work we propose to combine a supervoxel-based image representation with the concept of graph cuts as an efficient optimization technique for 3D deformable image registration. Due to the pixels/voxels-wise graph construction, the use of graph cuts in this context has been mainly limited to 2D applications. However, our work overcomes some of the previous limitations by posing the problem on a graph created by adjacent supervoxels, where the number of nodes in the graph is reduced from the number of voxels to the number of supervoxels. We demonstrate how a supervoxel image representation, combined with graph cuts-based optimization can be applied to 3D data. We further show that the application of a relaxed graph representation of the image, followed by guided image filtering over the estimated deformation field, allows us to model 'sliding motion'. Applying this method to lung image registration, results in highly accurate image registration and anatomically plausible estimations of the deformations. Evaluation of our method on a publicly available Computed Tomography lung image dataset (www.dir-lab.com) leads to the observation that our new approach compares very favorably with state-of-the-art in continuous and discrete image registration methods achieving Target Registration Error of 1.16mm on average per landmark.

Deformable image registration applied to lung SBRT: Usefulness and limitations

Radiation therapy (RT) of the lung requires deformation analysis. Deformable image registration (DIR) is the fundamental method to quantify deformations for various applications: motion compensation, contour propagation, dose accumulation, etc. DIR is therefore unavoidable in lung RT. DIR algorithms have been studied for decades and are now available both within commercial and academic packages. However, they are complex and have limitations that every user must be aware of before clinical implementation. In this paper, the main applications of DIR for lung RT with their associated uncertainties and their limitations are reviewed.

A hybrid patient-specific biomechanical model based image registration method for the motion estimation of lungs

This paper presents a new hybrid biomechanical model-based non-rigid image registration method for lung motion estimation. In the proposed method, a patient-specific biomechanical modelling process captures major physically realistic deformations with explicit physical modelling of sliding motion, whilst a subsequent non-rigid image registration process compensates for small residuals. The proposed algorithm was evaluated with 10 4D CT datasets of lung cancer patients. The target registration error (TRE), defined as the Euclidean distance of landmark pairs, was significantly lower with the proposed method (TRE = 1.37 mm) than with biomechanical modelling (TRE = 3.81 mm) and intensity-based image registration without specific considerations for sliding motion (TRE = 4.57 mm). The proposed method achieved a comparable accuracy as several recently developed intensity-based registration algorithms with sliding handling on the same datasets. A detailed comparison on the distributions of TREs with three non-rigid intensity-based algorithms showed that the proposed method performed especially well on estimating the displacement field of lung surface regions (mean TRE = 1.33 mm, maximum TRE = 5.3 mm). The effects of biomechanical model parameters (such as Poisson's ratio, friction and tissue heterogeneity) on displacement estimation were investigated. The potential of the algorithm in optimising biomechanical models of lungs through analysing the pattern of displacement compensation from the image registration process has also been demonstrated.

A new CT reconstruction technique using adaptive deformation recovery and intensity correction (ADRIC)

PURPOSE

Sequential same-patient CT images may involve deformation-induced and non-deformation-induced voxel intensity changes. An adaptive deformation recovery and intensity correction (ADRIC) technique was developed to improve the CT reconstruction accuracy, and to separate deformation from non-deformation-induced voxel intensity changes between sequential CT images.

MATERIALS AND METHODS

ADRIC views the new CT volume as a deformation of a prior high-quality CT volume, but with additional non-deformation-induced voxel intensity changes. ADRIC first applies the 2D-3D deformation technique to recover the deformation field between the prior CT volume and the new, to-be-reconstructed CT volume. Using the deformation-recovered new CT volume, ADRIC further corrects the non-deformation-induced voxel intensity changes with an updated algebraic reconstruction technique ("ART-dTV"). The resulting intensity-corrected new CT volume is subsequently fed back into the 2D-3D deformation process to further correct the residual deformation errors, which forms an iterative loop. By ADRIC, the deformation field and the non-deformation voxel intensity corrections are optimized separately and alternately to reconstruct the final CT. CT myocardial perfusion imaging scenarios were employed to evaluate the efficacy of ADRIC, using both simulated data of the extended-cardiac-torso (XCAT) digital phantom and experimentally acquired porcine data. The reconstruction accuracy of the ADRIC technique was compared to the technique using ART-dTV alone, and to the technique using 2D-3D deformation alone. The relative error metric and the universal quality index metric are calculated between the images for quantitative analysis. The relative error is defined as the square root of the sum of squared voxel intensity differences between the reconstructed volume and the "ground-truth" volume, normalized by the square root of the sum of squared "ground-truth" voxel intensities. In addition to the XCAT and porcine studies, a physical lung phantom measurement study was also conducted. Water-filled balloons with various shapes/volumes and concentrations of iodinated contrasts were put inside the phantom to simulate both deformations and non-deformation-induced intensity changes for ADRIC reconstruction. The ADRIC-solved deformations and intensity changes from limited-view projections were compared to those of the "gold-standard" volumes reconstructed from fully sampled projections.

RESULTS

For the XCAT simulation study, the relative errors of the reconstructed CT volume by the 2D-3D deformation technique, the ART-dTV technique, and the ADRIC technique were 14.64%, 19.21%, and 11.90% respectively, by using 20 projections for reconstruction. Using 60 projections for reconstruction reduced the relative errors to 12.33%, 11.04%, and 7.92% for the three techniques, respectively. For the porcine study, the corresponding results were 13.61%, 8.78%, and 6.80% by using 20 projections; and 12.14%, 6.91%, and 5.29% by using 60 projections. The ADRIC technique also demonstrated robustness to varying projection exposure levels. For the physical phantom study, the average DICE coefficient between the initial prior balloon volume and the new "gold-standard" balloon volumes was 0.460. ADRIC reconstruction by 21 projections increased the average DICE coefficient to 0.954.

CONCLUSION

The ADRIC technique outperformed both the 2D-3D deformation technique and the ART-dTV technique in reconstruction accuracy. The alternately solved deformation field and non-deformation voxel intensity corrections can benefit multiple clinical applications, including tumor tracking, radiotherapy dose accumulation, and treatment outcome analysis.

Simultaneous 4D-CBCT reconstruction with sliding motion constraint

PURPOSE

Current approaches using deformable vector field (DVF) for motion-compensated 4D-cone beam CT (CBCT) reconstruction typically utilize an isotropically smoothed DVF between different respiration phases. Such isotropically smoothed DVF does not work well if sliding motion exists between neighboring organs. This study investigated an anisotropic motion modeling scheme by extracting organ boundary local motions (e.g., sliding) and incorporated them into 4D-CBCT reconstruction to optimize the motion modeling and reconstruction methods.

METHODS

Initially, a modified simultaneous algebraic reconstruction technique (mSART) was applied to reconstruct high quality reference phase CBCT using all phase projections. The initial DVFs were precalculated and subsequently updated to achieve the optimized solution. During the DVF update, sliding motion estimation was performed by matching the measured projections to the forward projection of the deformed reference phase CBCT. In this process, each moving organ boundary was first segmented. The normal vectors of the boundary DVF were then extracted and incorporated for further DVF optimization. The regularization term in the objective function adaptively regularizes the DVF by (1) isotopically smoothing the DVF within each organ; (2) smoothing the DVF at boundary along the normal direction; and (3) leaving the tangent direction of boundary DVF unsmoothed (i.e., allowing for sliding motion). A nonlinear conjugate gradient optimizer was used. The algorithm was validated on a digital cubic tube phantom with sliding motion, nonuniform rotational B-spline based cardiac-torso (NCAT) phantom, and two anonymized patient data. The relative reconstruction error (RE), the motion trajectory's root mean square error (RMSE) together with its maximum error (MaxE), and the Dice coefficient of the lung boundary were calculated to evaluate the algorithm performance.

RESULTS

For the cubic tube and NCAT phantom tests, the REs are 10.2% and 7.4% with sliding motion compensation, compared to 13.4% and 8.9% without sliding modeling. The motion trajectory's RMSE and MaxE for NCAT phantom tests are 0.5 and 0.8 mm with sliding motion constraint compared to 3.5 and 7.3 mm without sliding motion modeling. The Dice coefficients for both NCAT phantom and the patients show a consistent trend that sliding motion constraint achieves better similarity for segmented lung boundary compared with the ground truth or patient reference.

CONCLUSIONS

A sliding motion-compensated 4D-CBCT reconstruction and the motion modeling scheme was developed. Both phantom and patient study demonstrated the improved accuracy and motion modeling accuracy in reconstructed 4D-CBCT.

Detection of Alzheimer's Disease by Three-Dimensional Displacement Field Estimation in Structural Magnetic Resonance Imaging

BACKGROUND

Within the past decade, computer scientists have developed many methods using computer vision and machine learning techniques to detect Alzheimer's disease (AD) in its early stages.

OBJECTIVE

However, some of these methods are unable to achieve excellent detection accuracy, and several other methods are unable to locate AD-related regions. Hence, our goal was to develop a novel AD brain detection method.

METHODS

In this study, our method was based on the three-dimensional (3D) displacement-field (DF) estimation between subjects in the healthy elder control group and AD group. The 3D-DF was treated with AD-related features. The three feature selection measures were used in the Bhattacharyya distance, Student's t-test, and Welch's t-test (WTT). Two non-parallel support vector machines, i.e., generalized eigenvalue proximal support vector machine and twin support vector machine (TSVM), were then used for classification. A 50 × 10-fold cross validation was implemented for statistical analysis.

RESULTS

The results showed that "3D-DF+WTT+TSVM" achieved the best performance, with an accuracy of 93.05 ± 2.18, a sensitivity of 92.57 ± 3.80, a specificity of 93.18 ± 3.35, and a precision of 79.51 ± 2.86. This method also exceled in 13 state-of-the-art approaches. Additionally, we were able to detect 17 regions related to AD by using the pure computer-vision technique. These regions include sub-gyral, inferior parietal lobule, precuneus, angular gyrus, lingual gyrus, supramarginal gyrus, postcentral gyrus, third ventricle, superior parietal lobule, thalamus, middle temporal gyrus, precentral gyrus, superior temporal gyrus, superior occipital gyrus, cingulate gyrus, culmen, and insula. These regions were reported in recent publications.

CONCLUSIONS

The 3D-DF is effective in AD subject and related region detection.

Motion-aware temporal regularization for improved 4D cone-beam computed tomography

Four-dimensional cone-beam computed tomography (4D-CBCT) of the free-breathing thorax is a valuable tool in image-guided radiation therapy of the thorax and the upper abdomen. It allows the determination of the position of a tumor throughout the breathing cycle, while only its mean position can be extracted from three-dimensional CBCT. The classical approaches are not fully satisfactory: respiration-correlated methods allow one to accurately locate high-contrast structures in any frame, but contain strong streak artifacts unless the acquisition is significantly slowed down. Motion-compensated methods can yield streak-free, but static, reconstructions. This work proposes a 4D-CBCT method that can be seen as a trade-off between respiration-correlated and motion-compensated reconstruction. It builds upon the existing reconstruction using spatial and temporal regularization (ROOSTER) and is called motion-aware ROOSTER (MA-ROOSTER). It performs temporal regularization along curved trajectories, following the motion estimated on a prior 4D CT scan. MA-ROOSTER does not involve motion-compensated forward and back projections: the input motion is used only during temporal regularization. MA-ROOSTER is compared to ROOSTER, motion-compensated Feldkamp-Davis-Kress (MC-FDK), and two respiration-correlated methods, on CBCT acquisitions of one physical phantom and two patients. It yields streak-free reconstructions, visually similar to MC-FDK, and robust information on tumor location throughout the breathing cycle. MA-ROOSTER also allows a variation of the lung tissue density during the breathing cycle, similar to that of planning CT, which is required for quantitative post-processing.

Biomechanical deformable image registration of longitudinal lung CT images using vessel information

Spatial correlation of lung tissue across longitudinal images, as the patient responds to treatment, is a critical step in adaptive radiotherapy. The goal of this work is to expand a biomechanical model-based deformable registration algorithm (Morfeus) to achieve accurate registration in the presence of significant anatomical changes. Six lung cancer patients previously treated with conventionally fractionated radiotherapy were retrospectively evaluated. Exhale CT scans were obtained at treatment planning and following three weeks of treatment. For each patient, the planning CT was registered to the follow-up CT using Morfeus, a biomechanical model-based deformable registration algorithm. To model the complex response of the lung, an extension to Morfeus has been developed: an initial deformation was estimated with Morfeus consisting of boundary conditions on the chest wall and incorporating a sliding interface with the lungs. It was hypothesized that the addition of boundary conditions based on vessel tree matching would provide a robust reduction of the residual registration error. To achieve this, the vessel trees were segmented on the two images by thresholding a vesselness image based on the Hessian matrix's eigenvalues. For each point on the reference vessel tree centerline, the displacement vector was estimated by applying a variant of the Demons registration algorithm between the planning CT and the deformed follow-up CT. An expert independently identified corresponding landmarks well distributed in the lung to compute target registration errors (TRE). The TRE was: [Formula: see text], [Formula: see text] and [Formula: see text] mm after rigid registration, Morfeus and Morfeus with boundary conditions on the vessel tree, respectively. In conclusion, the addition of boundary conditions on the vessels significantly improved the accuracy in modeling the response of the lung and tumor over the course of radiotherapy. Minimizing and modeling these geometrical uncertainties will enable future plan adaptation strategies.

4D respiratory motion-compensated image reconstruction of free-breathing radial MR data with very high undersampling

PURPOSE

To develop four-dimensional (4D) respiratory time-resolved MRI based on free-breathing acquisition of radial MR data with very high undersampling.

METHODS

We propose the 4D joint motion-compensated high-dimensional total variation (4D joint MoCo-HDTV) algorithm, which alternates between motion-compensated image reconstruction and artifact-robust motion estimation at multiple resolution levels. The algorithm is applied to radial MR data of the thorax and upper abdomen of 12 free-breathing subjects with acquisition times between 37 and 41 s and undersampling factors of 16.8. Resulting images are compared with compressed sensing-based 4D motion-adaptive spatio-temporal regularization (MASTeR) and 4D high-dimensional total variation (HDTV) reconstructions.

RESULTS

For all subjects, 4D joint MoCo-HDTV achieves higher similarity in terms of normalized mutual information and cross-correlation than 4D MASTeR and 4D HDTV when compared with reference 4D gated gridding reconstructions with 8.4 ± 1.1 times longer acquisition times. In a qualitative assessment of artifact level and image sharpness by two radiologists, 4D joint MoCo-HDTV reveals higher scores (P < 0.05) than 4D HDTV and 4D MASTeR at the same undersampling factor and the reference 4D gated gridding reconstructions, respectively.

CONCLUSIONS

4D joint MoCo-HDTV enables time-resolved image reconstruction of free-breathing radial MR data with undersampling factors of 16.8 while achieving low-streak artifact levels and high image sharpness. Magn Reson Med 77:1170-1183, 2017. © 2016 International Society for Magnetic Resonance in Medicine.

4D cone-beam CT reconstruction using multi-organ meshes for sliding motion modeling

A simultaneous motion estimation and image reconstruction (SMEIR) strategy was proposed for 4D cone-beam CT (4D-CBCT) reconstruction and showed excellent results in both phantom and lung cancer patient studies. In the original SMEIR algorithm, the deformation vector field (DVF) was defined on voxel grid and estimated by enforcing a global smoothness regularization term on the motion fields. The objective of this work is to improve the computation efficiency and motion estimation accuracy of SMEIR for 4D-CBCT through developing a multi-organ meshing model. Feature-based adaptive meshes were generated to reduce the number of unknowns in the DVF estimation and accurately capture the organ shapes and motion. Additionally, the discontinuity in the motion fields between different organs during respiration was explicitly considered in the multi-organ mesh model. This will help with the accurate visualization and motion estimation of the tumor on the organ boundaries in 4D-CBCT. To further improve the computational efficiency, a GPU-based parallel implementation was designed. The performance of the proposed algorithm was evaluated on a synthetic sliding motion phantom, a 4D NCAT phantom, and four lung cancer patients. The proposed multi-organ mesh based strategy outperformed the conventional Feldkamp-Davis-Kress, iterative total variation minimization, original SMEIR and single meshing method based on both qualitative and quantitative evaluations.

3D nonrigid medical image registration using a new information theoretic measure

This work presents a novel method for the nonrigid registration of medical images based on the Arimoto entropy, a generalization of the Shannon entropy. The proposed method employed the Jensen-Arimoto divergence measure as a similarity metric to measure the statistical dependence between medical images. Free-form deformations were adopted as the transformation model and the Parzen window estimation was applied to compute the probability distributions. A penalty term is incorporated into the objective function to smooth the nonrigid transformation. The goal of registration is to optimize an objective function consisting of a dissimilarity term and a penalty term, which would be minimal when two deformed images are perfectly aligned using the limited memory BFGS optimization method, and thus to get the optimal geometric transformation. To validate the performance of the proposed method, experiments on both simulated 3D brain MR images and real 3D thoracic CT data sets were designed and performed on the open source elastix package. For the simulated experiments, the registration errors of 3D brain MR images with various magnitudes of known deformations and different levels of noise were measured. For the real data tests, four data sets of 4D thoracic CT from four patients were selected to assess the registration performance of the method, including ten 3D CT images for each 4D CT data covering an entire respiration cycle. These results were compared with the normalized cross correlation and the mutual information methods and show a slight but true improvement in registration accuracy.

Detection of Alzheimer's disease by displacement field and machine learning

Aim. Alzheimer's disease (AD) is a chronic neurodegenerative disease. Recently, computer scientists have developed various methods for early detection based on computer vision and machine learning techniques. Method. In this study, we proposed a novel AD detection method by displacement field (DF) estimation between a normal brain and an AD brain. The DF was treated as the AD-related features, reduced by principal component analysis (PCA), and finally fed into three classifiers: support vector machine (SVM), generalized eigenvalue proximal SVM (GEPSVM), and twin SVM (TSVM). The 10-fold cross validation repeated 50 times. Results. The results showed the "DF + PCA + TSVM" achieved the accuracy of 92.75 ± 1.77, sensitivity of 90.56 ± 1.15, specificity of 93.37 ± 2.05, and precision of 79.61 ± 2.21. This result is better than or comparable with not only the other proposed two methods, but also ten state-of-the-art methods. Besides, our method discovers the AD is related to following brain regions disclosed in recent publications: Angular Gyrus, Anterior Cingulate, Cingulate Gyrus, Culmen, Cuneus, Fusiform Gyrus, Inferior Frontal Gyrus, Inferior Occipital Gyrus, Inferior Parietal Lobule, Inferior Semi-Lunar Lobule, Inferior Temporal Gyrus, Insula, Lateral Ventricle, Lingual Gyrus, Medial Frontal Gyrus, Middle Frontal Gyrus, Middle Occipital Gyrus, Middle Temporal Gyrus, Paracentral Lobule, Parahippocampal Gyrus, Postcentral Gyrus, Posterior Cingulate, Precentral Gyrus, Precuneus, Sub-Gyral, Superior Parietal Lobule, Superior Temporal Gyrus, Supramarginal Gyrus, and Uncus. Conclusion. The displacement filed is effective in detection of AD and related brain-regions.

A hybrid biomechanical intensity based deformable image registration of lung 4DCT

Deformable image registration (DIR) has been extensively studied over the past two decades due to its essential role in many image-guided interventions (IGI). IGI demands a highly accurate registration that maintains its accuracy across the entire region of interest. This work evaluates the improvement in accuracy and consistency by refining the results of Morfeus, a biomechanical model-based DIR algorithm. A hybrid DIR algorithm is proposed based on, a biomechanical model-based DIR algorithm and a refinement step based on a B-spline intensity-based algorithm. Inhale and exhale reconstructions of four-dimensional computed tomography (4DCT) lung images from 31 patients were initially registered using the biomechanical DIR by modeling contact surface between the lungs and the chest cavity. The resulting deformations were then refined using the intensity-based algorithm to reduce any residual uncertainties. Important parameters in the intensity-based algorithm, including grid spacing, number of pyramids, and regularization coefficient, were optimized on 10 randomly-chosen patients (out of 31). Target registration error (TRE) was calculated by measuring the Euclidean distance of common anatomical points on both images after registration. For each patient a minimum of 30 points/lung were used. Grid spacing of 8 mm, 5 levels of grid pyramids, and regularization coefficient of 3.0 were found to provide optimal results on 10 randomly chosen patients. Overall the entire patient population (n = 31), the hybrid method resulted in mean ± SD (90th%) TRE of 1.5 ± 1.4 (2.9) mm compared to 3.1 ± 1.9 (5.6) using biomechanical DIR and 2.6 ± 2.5 (6.1) using intensity-based DIR alone. The proposed hybrid biomechanical modeling intensity based algorithm is a promising DIR technique which could be used in various IGI procedures. The current investigation shows the efficacy of this approach for the registration of 4DCT images of the lungs with average accuracy of 1.5 mm.

Improving thoracic four-dimensional cone-beam CT reconstruction with anatomical-adaptive image regularization (AAIR)

Total-variation (TV) minimization reconstructions can significantly reduce noise and streaks in thoracic four-dimensional cone-beam computed tomography (4D CBCT) images compared to the Feldkamp-Davis-Kress (FDK) algorithm currently used in practice. TV minimization reconstructions are, however, prone to over-smoothing anatomical details and are also computationally inefficient. The aim of this study is to demonstrate a proof of concept that these disadvantages can be overcome by incorporating the general knowledge of the thoracic anatomy via anatomy segmentation into the reconstruction. The proposed method, referred as the anatomical-adaptive image regularization (AAIR) method, utilizes the adaptive-steepest-descent projection-onto-convex-sets (ASD-POCS) framework, but introduces an additional anatomy segmentation step in every iteration. The anatomy segmentation information is implemented in the reconstruction using a heuristic approach to adaptively suppress over-smoothing at anatomical structures of interest. The performance of AAIR depends on parameters describing the weighting of the anatomy segmentation prior and segmentation threshold values. A sensitivity study revealed that the reconstruction outcome is not sensitive to these parameters as long as they are chosen within a suitable range. AAIR was validated using a digital phantom and a patient scan and was compared to FDK, ASD-POCS and the prior image constrained compressed sensing (PICCS) method. For the phantom case, AAIR reconstruction was quantitatively shown to be the most accurate as indicated by the mean absolute difference and the structural similarity index. For the patient case, AAIR resulted in the highest signal-to-noise ratio (i.e. the lowest level of noise and streaking) and the highest contrast-to-noise ratios for the tumor and the bony anatomy (i.e. the best visibility of anatomical details). Overall, AAIR was much less prone to over-smoothing anatomical details compared to ASD-POCS and did not suffer from residual noise/streaking and motion blur migrated from the prior image as in PICCS. AAIR was also found to be more computationally efficient than both ASD-POCS and PICCS, with a reduction in computation time of over 50% compared to ASD-POCS. The use of anatomy segmentation was, for the first time, demonstrated to significantly improve image quality and computational efficiency for thoracic 4D CBCT reconstruction. Further developments are required to facilitate AAIR for practical use.

An implicit sliding-motion preserving regularisation via bilateral filtering for deformable image registration

Several biomedical applications require accurate image registration that can cope effectively with complex organ deformations. This paper addresses this problem by introducing a generic deformable registration algorithm with a new regularization scheme, which is performed through bilateral filtering of the deformation field. The proposed approach is primarily designed to handle smooth deformations both between and within body structures, and also more challenging deformation discontinuities exhibited by sliding organs. The conventional Gaussian smoothing of deformation fields is replaced by a bilateral filtering procedure, which compromises between the spatial smoothness and local intensity similarity kernels, and is further supported by a deformation field similarity kernel. Moreover, the presented framework does not require any explicit prior knowledge about the organ motion properties (e.g. segmentation) and therefore forms a fully automated registration technique. Validation was performed using synthetic phantom data and publicly available clinical 4D CT lung data sets. In both cases, the quantitative analysis shows improved accuracy when compared to conventional Gaussian smoothing. In addition, we provide experimental evidence that masking the lungs in order to avoid the problem of sliding motion during registration performs similarly in terms of the target registration error when compared to the proposed approach, however it requires accurate lung segmentation. Finally, quantification of the level and location of detected sliding motion yields visually plausible results by demonstrating noticeable sliding at the pleural cavity boundaries.

Estimation of lung motion fields in 4D CT data by variational non-linear intensity-based registration: A comparison and evaluation study

Accurate and robust estimation of motion fields in respiration-correlated CT (4D CT) images, usually performed by non-linear registration of the temporal CT frames, is a precondition for the analysis of patient-specific breathing dynamics and subsequent image-supported diagnostics and treatment planning. In this work, we present a comprehensive comparison and evaluation study of non-linear registration variants applied to the task of lung motion estimation in thoracic 4D CT data. In contrast to existing multi-institutional comparison studies (e.g. MIDRAS and EMPIRE10), we focus on the specific but common class of variational intensity-based non-parametric registration and analyze the impact of the different main building blocks of the underlying optimization problem: the distance measure to be minimized, the regularization approach and the transformation space considered during optimization. In total, 90 different combinations of building block instances are compared. Evaluated on proprietary and publicly accessible 4D CT images, landmark-based registration errors (TRE) between 1.14 and 1.20 mm for the most accurate registration variants demonstrate competitive performance of the applied general registration framework compared to other state-of-the-art approaches for lung CT registration. Although some specific trends can be observed, effects of interchanging individual instances of the building blocks on the TRE are in general rather small (no single outstanding registration variant existing); the same level of accuracy is, however, associated with significantly different degrees of motion field smoothness and computational demands. Consequently, the building block combination of choice will depend on application-specific requirements on motion field characteristics.

Edge- and detail-preserving sparse image representations for deformable registration of chest MRI and CT volumes

Deformable medical image registration requires the optimisation of a function with a large number of degrees of freedom. Commonly-used approaches to reduce the computational complexity, such as uniform B-splines and Gaussian image pyramids, introduce translation-invariant homogeneous smoothing, and may lead to less accurate registration in particular for motion fields with discontinuities. This paper introduces the concept of sparse image representation based on supervoxels, which are edge-preserving and therefore enable accurate modelling of sliding organ motions frequently seen in respiratory and cardiac scans. Previous shortcomings of using supervoxels in motion estimation, in particular inconsistent clustering in ambiguous regions, are overcome by employing multiple layers of supervoxels. Furthermore, we propose a new similarity criterion based on a binary shape representation of supervoxels, which improves the accuracy of single-modal registration and enables multimodal registration. We validate our findings based on the registration of two challenging clinical applications of volumetric deformable registration: motion estimation between inhale and exhale phase of CT scans for radiotherapy planning, and deformable multi-modal registration of diagnostic MRI and CT chest scans. The experiments demonstrate state-of-the-art registration accuracy, and require no additional anatomical knowledge with greatly reduced computational complexity.

A locally adaptive regularization based on anisotropic diffusion for deformable image registration of sliding organs

We propose a deformable image registration algorithm that uses anisotropic smoothing for regularization to find correspondences between images of sliding organs. In particular, we apply the method for respiratory motion estimation in longitudinal thoracic and abdominal computed tomography scans. The algorithm uses locally adaptive diffusion tensors to determine the direction and magnitude with which to smooth the components of the displacement field that are normal and tangential to an expected sliding boundary. Validation was performed using synthetic, phantom, and 14 clinical datasets, including the publicly available DIR-Lab dataset. We show that motion discontinuities caused by sliding can be effectively recovered, unlike conventional regularizations that enforce globally smooth motion. In the clinical datasets, target registration error showed improved accuracy for lung landmarks compared to the diffusive regularization. We also present a generalization of our algorithm to other sliding geometries, including sliding tubes (e.g., needles sliding through tissue, or contrast agent flowing through a vessel). Potential clinical applications of this method include longitudinal change detection and radiotherapy for lung or abdominal tumours, especially those near the chest or abdominal wall.

An active contour model for medical image segmentation with application to brain CT image

PURPOSE

Cerebrospinal fluid (CSF) segmentation in computed tomography (CT) is a key step in computer-aided detection (CAD) of acute ischemic stroke. Because of image noise, low contrast and intensity inhomogeneity, CSF segmentation has been a challenging task. A region-based active contour model, which is insensitive to contour initialization and robust to intensity inhomogeneity, was developed for segmenting CSF in brain CT images.

METHODS

The energy function of the region-based active contour model is composed of a range domain kernel function, a space domain kernel function, and an edge indicator function. By minimizing the energy function, the region of edge elements of the target could be automatically identified in images with less dependence on initial contours. The energy function was optimized by means of the deepest descent method with a level set framework. An overlap rate between segmentation results and the reference standard was used to assess the segmentation accuracy. The authors evaluated the performance of the proposed method on both synthetic data and real brain CT images. They also compared the performance level of our method to those of region-scalable fitting (RSF) and global convex segment (GCS) models.

RESULTS

For the experiment of CSF segmentation in 67 brain CT images, their method achieved an average overlap rate of 66% compared to the average overlap rates of 16% and 46% from the RSF model and the GCS model, respectively.

CONCLUSIONS

Their region-based active contour model has the ability to achieve accurate segmentation results in images with high noise level and intensity inhomogeneity. Therefore, their method has great potential in the segmentation of medical images and would be useful for developing CAD schemes for acute ischemic stroke in brain CT images.

MRF-based deformable registration and ventilation estimation of lung CT

Deformable image registration is an important tool in medical image analysis. In the case of lung computed tomography (CT) registration there are three major challenges: large motion of small features, sliding motions between organs, and changing image contrast due to compression. Recently, Markov random field (MRF)-based discrete optimization strategies have been proposed to overcome problems involved with continuous optimization for registration, in particular its susceptibility to local minima. However, to date the simplifications made to obtain tractable computational complexity reduced the registration accuracy. We address these challenges and preserve the potentially higher quality of discrete approaches with three novel contributions. First, we use an image-derived minimum spanning tree as a simplified graph structure, which copes well with the complex sliding motion and allows us to find the global optimum very efficiently. Second, a stochastic sampling approach for the similarity cost between images is introduced within a symmetric, diffeomorphic B-spline transformation model with diffusion regularization. The complexity is reduced by orders of magnitude and enables the minimization of much larger label spaces. In addition to the geometric transform labels, hyper-labels are introduced, which represent local intensity variations in this task, and allow for the direct estimation of lung ventilation. We validate the improvements in accuracy and performance on exhale-inhale CT volume pairs using a large number of expert landmarks.

A reference dataset for deformable image registration spatial accuracy evaluation using the COPDgene study archive

Landmark point-pairs provide a strategy to assess deformable image registration (DIR) accuracy in terms of the spatial registration of the underlying anatomy depicted in medical images. In this study, we propose to augment a publicly available database (www.dir-lab.com) of medical images with large sets of manually identified anatomic feature pairs between breath-hold computed tomography (BH-CT) images for DIR spatial accuracy evaluation. Ten BH-CT image pairs were randomly selected from the COPDgene study cases. Each patient had received CT imaging of the entire thorax in the supine position at one-fourth dose normal expiration and maximum effort full dose inspiration. Using dedicated in-house software, an imaging expert manually identified large sets of anatomic feature pairs between images. Estimates of inter- and intra-observer spatial variation in feature localization were determined by repeat measurements of multiple observers over subsets of randomly selected features. 7298 anatomic landmark features were manually paired between the 10 sets of images. Quantity of feature pairs per case ranged from 447 to 1172. Average 3D Euclidean landmark displacements varied substantially among cases, ranging from 12.29 (SD: 6.39) to 30.90 (SD: 14.05) mm. Repeat registration of uniformly sampled subsets of 150 landmarks for each case yielded estimates of observer localization error, which ranged in average from 0.58 (SD: 0.87) to 1.06 (SD: 2.38) mm for each case. The additions to the online web database (www.dir-lab.com) described in this work will broaden the applicability of the reference data, providing a freely available common dataset for targeted critical evaluation of DIR spatial accuracy performance in multiple clinical settings. Estimates of observer variance in feature localization suggest consistent spatial accuracy for all observers across both four-dimensional CT and COPDgene patient cohorts.