Facilitating tumor functional assessment by spatially relating 3D tumor histology and in vivo MRI: image registration approach

Robust joint registration of multiple stains and MRI for multimodal 3D histology reconstruction: Application to the Allen human brain atlas

Joint registration of a stack of 2D histological sections to recover 3D structure ("3D histology reconstruction") finds application in areas such as atlas building and validation of in vivo imaging. Straightforward pairwise registration of neighbouring sections yields smooth reconstructions but has well-known problems such as "banana effect" (straightening of curved structures) and "z-shift" (drift). While these problems can be alleviated with an external, linearly aligned reference (e.g., Magnetic Resonance (MR) images), registration is often inaccurate due to contrast differences and the strong nonlinear distortion of the tissue, including artefacts such as folds and tears. In this paper, we present a probabilistic model of spatial deformation that yields reconstructions for multiple histological stains that that are jointly smooth, robust to outliers, and follow the reference shape. The model relies on a spanning tree of latent transforms connecting all the sections and slices of the reference volume, and assumes that the registration between any pair of images can be see as a noisy version of the composition of (possibly inverted) latent transforms connecting the two images. Bayesian inference is used to compute the most likely latent transforms given a set of pairwise registrations between image pairs within and across modalities. We consider two likelihood models: Gaussian (ℓ2 norm, which can be minimised in closed form) and Laplacian (ℓ1 norm, minimised with linear programming). Results on synthetic deformations on multiple MR modalities, show that our method can accurately and robustly register multiple contrasts even in the presence of outliers. The framework is used for accurate 3D reconstruction of two stains (Nissl and parvalbumin) from the Allen human brain atlas, showing its benefits on real data with severe distortions. Moreover, we also provide the registration of the reconstructed volume to MNI space, bridging the gaps between two of the most widely used atlases in histology and MRI. The 3D reconstructed volumes and atlas registration can be downloaded from https://openneuro.org/datasets/ds003590. The code is freely available at https://github.com/acasamitjana/3dhirest.

In vivo noninvasive preclinical tumor hypoxia imaging methods: a review

Tumors exhibit areas of decreased oxygenation due to malformed blood vessels. This low oxygen concentration decreases the effectiveness of radiation therapy, and the resulting poor perfusion can prevent drugs from reaching areas of the tumor. Tumor hypoxia is associated with poorer prognosis and disease progression, and is therefore of interest to preclinical researchers. Although there are multiple different ways to measure tumor hypoxia and related factors, there is no standard for quantifying spatial and temporal tumor hypoxia distributions in preclinical research or in the clinic. This review compares imaging methods utilized for the purpose of assessing spatio-temporal patterns of hypoxia in the preclinical setting. Imaging methods provide varying levels of spatial and temporal resolution regarding different aspects of hypoxia, and with varying advantages and disadvantages. The choice of modality requires consideration of the specific experimental model, the nature of the required characterization and the availability of complementary modalities as well as immunohistochemistry.

Histological Validation of MRI: A Review of Challenges in Registration of Imaging and Whole-Mount Histopathology

Rigorous validation with ground truth information such as histology is needed to reliably assess the current and potential value of MRI techniques to characterize tissue and identify disease-related tissue alterations. Commonly used methods that aim to directly correlate histology and MRI data generally fall short of this goal due to spatial errors that preclude direct matching. Errors result from tissue deformation, differences in spatial resolution and slice thickness, non-coplanar and/or nonintersecting plane orientations, and different image contrast mechanisms. Some of these problems arise from limitations in standard protocols for clinical tissue processing and histology-based pathology reporting, and to some extent can be addressed by modifications to standard protocols without compromising the clinical process. Typical modifications include ex vivo specimen MRI, block-face photography, addition of fiducial markers, and 3D printed molds to constrain tissue deformation and guide sectioning. This review summarizes the advantages and limitations of MRI validation techniques based on coregistration of MRI with whole-mount histology of tissue specimens. LEVEL OF EVIDENCE: 4 TECHNICAL EFFICACY STAGE: 1.

Automating three-dimensional osteoarthritis histopathological grading of human osteochondral tissue using machine learning on contrast-enhanced micro-computed tomography

OBJECTIVE

To develop and validate a machine learning (ML) approach for automatic three-dimensional (3D) histopathological grading of osteochondral samples imaged with contrast-enhanced micro-computed tomography (CEμCT).

DESIGN

A total of 79 osteochondral cores from 24 total knee arthroplasty patients and two asymptomatic donors were imaged using CEμCT with phosphotungstic acid -staining. Volumes-of-interest (VOI) in surface (SZ), deep (DZ) and calcified (CZ) zones were extracted depth-wise and subjected to dimensionally reduced Local Binary Pattern -textural feature analysis. Regularized linear and logistic regression (LR) models were trained zone-wise against the manually assessed semi-quantitative histopathological CEμCT grades (diameter = 2 mm samples). Models were validated using nested leave-one-out cross-validation and an independent test set (4 mm samples). The performance was primarily assessed using Mean Squared Error (MSE) and Average Precision (AP, confidence intervals are given in square brackets).

RESULTS

Highest performance on cross-validation was observed for SZ, both on linear regression (MSE = 0.49, 0.69 and 0.71 for SZ, DZ and CZ, respectively) and LR (AP = 0.9 [0.77-0.99], 0.46 [0.28-0.67] and 0.65 [0.41-0.85] for SZ, DZ and CZ, respectively). The test set evaluations yielded increased MSE on all zones. For LR, the performance was also best for the SZ (AP = 0.85 [0.73-0.93], 0.82 [0.70-0.92] and 0.8 [0.67-0.9], for SZ, DZ and CZ, respectively).

CONCLUSION

We present the first ML-based automatic 3D histopathological osteoarthritis (OA) grading method which also adequately perform on grading unseen data, especially in SZ. After further development, the method could potentially be applied by OA researchers since the grading software and all source codes are publicly available.

Multiparametric Analysis of Longitudinal Quantitative MRI data to Identify Distinct Tumor Habitats in Preclinical Models of Breast Cancer

This study identifies physiological tumor habitats from quantitative magnetic resonance imaging (MRI) data and evaluates their alterations in response to therapy. Two models of breast cancer (BT-474 and MDA-MB-231) were imaged longitudinally with diffusion-weighted MRI and dynamic contrast-enhanced MRI to quantify tumor cellularity and vascularity, respectively, during treatment with trastuzumab or albumin-bound paclitaxel. Tumors were stained for anti-CD31, anti-Ki-67, and H&E. Imaging and histology data were clustered to identify tumor habitats and percent tumor volume (MRI) or area (histology) of each habitat was quantified. Histological habitats were correlated with MRI habitats. Clustering of both the MRI and histology data yielded three clusters: high-vascularity high-cellularity (HV-HC), low-vascularity high-cellularity (LV-HC), and low-vascularity low-cellularity (LV-LC). At day 4, BT-474 tumors treated with trastuzumab showed a decrease in LV-HC (p = 0.03) and increase in HV-HC (p = 0.03) percent tumor volume compared to control. MDA-MB-231 tumors treated with low-dose albumin-bound paclitaxel showed a longitudinal decrease in LV-HC percent tumor volume at day 3 (p = 0.01). Positive correlations were found between histological and imaging-derived habitats: HV-HC (BT-474: p = 0.03), LV-HC (MDA-MB-231: p = 0.04), LV-LC (BT-474: p = 0.04; MDA-MB-231: p < 0.01). Physiologically distinct tumor habitats associated with therapeutic response were identified with MRI and histology data in preclinical models of breast cancer.

Muscle percentage index as a marker of disease severity in golden retriever muscular dystrophy

INTRODUCTION

Golden retriever muscular dystrophy (GRMD) is a spontaneous X-linked canine model of Duchenne muscular dystrophy that resembles the human condition. Muscle percentage index (MPI) is proposed as an imaging biomarker of disease severity in GRMD.

METHODS

To assess MPI, we used MRI data acquired from nine GRMD samples using a 4.7 T small-bore scanner. A machine learning approach was used with eight raw quantitative mapping of MRI data images (T1m, T2m, two Dixon maps, and four diffusion tensor imaging maps), three types of texture descriptors (local binary pattern, gray-level co-occurrence matrix, gray-level run-length matrix), and a gradient descriptor (histogram of oriented gradients).

RESULTS

The confusion matrix, averaged over all samples, showed 93.5% of muscle pixels classified correctly. The classification, optimized in a leave-one-out cross-validation, provided an average accuracy of 80% with a discrepancy in overestimation for young (8%) and old (20%) dogs.

DISCUSSION

MPI could be useful for quantifying GRMD severity, but careful interpretation is needed for severe cases.

Multiparametric MRI and Coregistered Histology Identify Tumor Habitats in Breast Cancer Mouse Models

It is well-recognized that solid tumors are genomically, anatomically, and physiologically heterogeneous. In general, more heterogeneous tumors have poorer outcomes, likely due to the increased probability of harboring therapy-resistant cells and regions. It is hypothesized that the genomic and physiologic heterogeneity are related, because physiologically distinct regions will exert variable selection pressures leading to the outgrowth of clones with variable genomic/proteomic profiles. To investigate this, methods must be in place to interrogate and define, at the microscopic scale, the cytotypes that exist within physiologically distinct subregions ("habitats") that are present at mesoscopic scales. MRI provides a noninvasive approach to interrogate physiologically distinct local environments, due to the biophysical principles that govern MRI signal generation. Here, we interrogate different physiologic parameters, such as perfusion, cell density, and edema, using multiparametric MRI (mpMRI). Signals from six different acquisition schema were combined voxel-by-voxel into four clusters identified using a Gaussian mixture model. These were compared with histologic and IHC characterizations of sections that were coregistered using MRI-guided 3D printed tumor molds. Specifically, we identified a specific set of MRI parameters to classify viable-normoxic, viable-hypoxic, nonviable-hypoxic, and nonviable-normoxic tissue types within orthotopic 4T1 and MDA-MB-231 breast tumors. This is the first coregistered study to show that mpMRI can be used to define physiologically distinct tumor habitats within breast tumor models. SIGNIFICANCE: This study demonstrates that noninvasive imaging metrics can be used to distinguish subregions within heterogeneous tumors with histopathologic correlation.

Multiparametric MR for non-invasive evaluation of tumour tissue histological characteristics after radionuclide therapy

Early non-invasive tumour therapy response assessment requires methods sensitive to biological and physiological tumour characteristics. The aim of this study was to find and evaluate magnetic resonance imaging (MRI) derived tumour tissue parameters that correlate with histological parameters and that reflect effects of radionuclide therapy. Mice bearing a subcutaneous human small-intestine neuroendocrine tumour were i.v. injected with ¹⁷⁷ Lu-octreotate. MRI was performed (7 T Bruker Biospec) on different post-therapy intervals (1 and 13 days) using T2-weighted imaging, mapping of T2* and T1 relaxation time constants, as well as diffusion and dynamic contrast enhancement (DCE-MRI) techniques. After MRI, animals were killed and tumours excised. Four differently stained histological sections of the most central imaged tumour plane were digitized, and segmentation techniques were used to produce maps reflecting fibrotic and vascular density, apoptosis, and proliferation. Histological maps were aligned with MRI-derived parametric maps using landmark-based registration. Correlations and predictive power were evaluated using linear mixed-effects models and cross-validation, respectively. Several MR parameters showed statistically significant correlations with histological parameters. In particular, three DCE-MRI-derived parameters reflecting capillary function additionally showed high predictive power regarding apoptosis (2/3) and proliferation (1/3). T1 could be used to predict vascular density, and perfusion fraction derived from diffusion MRI could predict fibrotic density, although with lower predictive power. This work demonstrates the potential to use multiparametric MRI to retrieve important information on the tumour microenvironment after radiotherapy. The non-invasiveness of the method also allows longitudinal tumour tissue characterization. Further investigation is warranted to evaluate the parameters highlighted in this study longitudinally, in larger studies, and with additional histological methods.

A step towards valid detection and quantification of lung cancer volume in experimental mice with contrast agent-based X-ray microtomography

Tumor volume is a parameter used to evaluate the performance of new therapies in lung cancer research. Conventional methods that are used to estimate tumor size in mouse models fail to provide fast and reliable volumetric data for tumors grown non-subcutaneously. Here, we evaluated the use of iodine-staining combined with micro-computed tomography (micro-CT) to estimate the tumor volume of ex vivo tumor-burdened lungs. We obtained fast high spatial resolution three-dimensional information of the lungs, and we demonstrated that iodine-staining highlights tumors and unhealthy tissue. We processed iodine-stained lungs for histopathological analysis with routine hematoxylin and eosin (H&E) staining. We compared the traditional tumor burden estimation performed manually with H&E histological slices with a semi-automated method using micro-CT datasets. In mouse models that develop lung tumors with well precise boundaries, the method that we describe here enables to perform a quick estimation of tumorous tissue volume in micro-CT images. Our method overestimates the tumor burden in tumors surrounded by abnormal tissue, while traditional histopathological analysis underestimates tumor volume. We propose to embed micro-CT imaging to the traditional workflow of tumorous lung analyses in preclinical cancer research as a strategy to obtain a more accurate estimation of the total lung tumor burden.

Automatic deformable registration of histological slides to μCT volume data

Localizing a histological section in the three-dimensional dataset of a different imaging modality is a challenging 2D-3D registration problem. In the literature, several approaches have been proposed to solve this problem; however, they cannot be considered as fully automatic. Recently, we developed an automatic algorithm that could successfully find the position of a histological section in a micro computed tomography (μCT) volume. For the majority of the datasets, the result of localization corresponded to the manual results. However, for some datasets, the matching μCT slice was off the ground-truth position. Furthermore, elastic distortions, due to histological preparation, could not be accounted for in this framework. In the current study, we introduce two optimization frameworks based on normalized mutual information, which enabled us to accurately register histology slides to volume data. The rigid approach allocated 81 % of histological sections with a median position error of 8.4 μm in jaw bone datasets, and the deformable approach improved registration by 33 μm with respect to the median distance error for four histological slides in the cerebellum dataset.

3D volume reconstruction from serial breast specimen radiographs for mapping between histology and 3D whole specimen imaging

PURPOSE

In breast imaging, radiological in vivo images, such as x-ray mammography and magnetic resonance imaging (MRI), are used for tumor detection, diagnosis, and size determination. After excision, the specimen is typically sliced into slabs and a small subset is sampled. Histopathological imaging of the stained samples is used as the gold standard for characterization of the tumor microenvironment. A 3D volume reconstruction of the whole specimen from the 2D slabs could facilitate bridging the gap between histology and in vivo radiological imaging. This task is challenging, however, due to the large deformation that the breast tissue undergoes after surgery and the significant undersampling of the specimen obtained in histology. In this work, we present a method to reconstruct a coherent 3D volume from 2D digital radiographs of the specimen slabs.

METHODS

To reconstruct a 3D breast specimen volume, we propose the use of multiple target neighboring slices, when deforming each 2D slab radiograph in the volume, rather than performing pairwise registrations. The algorithm combines neighborhood slice information with free-form deformations, which enables a flexible, nonlinear deformation to be computed subject to the constraint that a coherent 3D volume is obtained. The neighborhood information provides adequate constraints, without the need for any additional regularization terms.

RESULTS

The volume reconstruction algorithm is validated on clinical mastectomy samples using a quantitative assessment of the volume reconstruction smoothness and a comparison with a whole specimen 3D image acquired for validation before slicing. Additionally, a target registration error of 5 mm (comparable to the specimen slab thickness of 4 mm) was obtained for five cases. The error was computed using manual annotations from four observers as gold standard, with interobserver variability of 3.4 mm. Finally, we illustrate how the reconstructed volumes can be used to map histology images to a 3D specimen image of the whole sample (either MRI or CT).

CONCLUSIONS

Qualitative and quantitative assessment has illustrated the benefit of using our proposed methodology to reconstruct a coherent specimen volume from serial slab radiographs. To our knowledge, this is the first method that has been applied to clinical breast cases, with the goal of reconstructing a whole specimen sample. The algorithm can be used as part of the pipeline of mapping histology images to ex vivo and ultimately in vivo radiological images of the breast.

Framework for 3D histologic reconstruction and fusion with in vivo MRI: Preliminary results of characterizing pulmonary inflammation in a mouse model

PURPOSE

Pulmonary inflammation is associated with a variety of diseases. Assessing pulmonary inflammation on in vivo imaging may facilitate the early detection and treatment of lung diseases. Although routinely used in thoracic imaging, computed tomography has thus far not been compellingly shown to characterize inflammation in vivo. Alternatively, magnetic resonance imaging (MRI) is a nonionizing radiation technique to better visualize and characterize pulmonary tissue. Prior to routine adoption of MRI for early characterization of inflammation in humans, a rigorous and quantitative characterization of the utility of MRI to identify inflammation is required. Such characterization may be achieved by considering ex vivo histology as the ground truth, since it enables the definitive spatial assessment of inflammation. In this study, the authors introduce a novel framework to integrate 2D histology, ex vivo and in vivo imaging to enable the mapping of the extent of disease from ex vivo histology onto in vivo imaging, with the goal of facilitating computerized feature analysis and interrogation of disease appearance on in vivo imaging. The authors' framework was evaluated in a preclinical preliminary study aimed to identify computer extracted features on in vivo MRI associated with chronic pulmonary inflammation.

METHODS

The authors' image analytics framework first involves reconstructing the histologic volume in 3D from individual histology slices. Second, the authors map the disease ground truth onto in vivo MRI via coregistration with 3D histology using the ex vivo lung MRI as a conduit. Finally, computerized feature analysis of the disease extent is performed to identify candidate in vivo imaging signatures of disease presence and extent.

RESULTS

The authors evaluated the framework by assessing the quality of the 3D histology reconstruction and the histology-MRI fusion, in the context of an initial use case involving characterization of chronic inflammation in a mouse model. The authors' evaluation considered three mice, two with an inflammation phenotype and one control. The authors' iterative 3D histology reconstruction yielded a 70.1% ± 2.7% overlap with the ex vivo MRI volume. Across a total of 17 anatomic landmarks manually delineated at the division of airways, the target registration error between the ex vivo MRI and 3D histology reconstruction was 0.85 ± 0.44 mm, suggesting that a good alignment of the ex vivo 3D histology and ex vivo MRI had been achieved. The 3D histology-in vivo MRI coregistered volumes resulted in an overlap of 73.7% ± 0.9%. Preliminary computerized feature analysis was performed on an additional four control mice, for a total of seven mice considered in this study. Gabor texture filters appeared to best capture differences between the inflamed and noninflamed regions on MRI.

CONCLUSIONS

The authors' 3D histology reconstruction and multimodal registration framework were successfully employed to reconstruct the histology volume of the lung and fuse it with in vivo MRI to create a ground truth map for inflammation on in vivo MRI. The analytic platform presented here lays the framework for a rigorous validation of the identified imaging features for chronic lung inflammation on MRI in a large prospective cohort.

The Role of Micro-CT in 3D Histology Imaging

OBJECTIVES

3D histology tissue modeling is a useful analytical technique for understanding anatomy and disease at the cellular level. However, the current accuracy of 3D histology technology is largely unknown, and errors, misalignment and missing information are common in 3D tissue reconstruction. We used micro-CT imaging technology to better understand these issues and the relationship between fresh tissue and its 3D histology counterpart.

METHODS

We imaged formalin-fixed and 2% Lugol-stained mouse brain, human uterus and human lung tissue with micro-CT. We then conducted image analyses on the tissues before and after paraffin embedding using 3D Slicer and ImageJ software to understand how tissue changes between the fixation and embedding steps.

RESULTS

We found that all tissue samples decreased in volume by 19.2-61.5% after embedding, that micro-CT imaging can be used to assess the integrity of tissue blocks, and that micro-CT analysis can help to design an optimized tissue-sectioning protocol.

CONCLUSIONS

Micro-CT reference data help to identify where and to what extent tissue was lost or damaged during slide production, provides valuable anatomical information for reconstructing missing parts of a 3D tissue model, and aids in correcting reconstruction errors when fitting the image information in vivo and ex vivo.

A method for accurate spatial registration of PET images and histopathology slices

BACKGROUND

Accurate alignment between histopathology slices and positron emission tomography (PET) images is important for radiopharmaceutical validation studies. Limited data is available on the registration accuracy that can be achieved between PET and histopathology slices acquired under routine pathology conditions where slices may be non-parallel, non-contiguously cut and of standard block size. The purpose of this study was to demonstrate a method for aligning PET images and histopathology slices acquired from patients with laryngeal cancer and to assess the registration accuracy obtained under these conditions.

METHODS

Six subjects with laryngeal cancer underwent a (64)Cu-copper-II-diacetyl-bis(N4-methylthiosemicarbazone) ((64)Cu-ATSM) PET computed tomography (CT) scan prior to total laryngectomy. Sea urchin spines were inserted into the pathology specimen to act as fiducial markers. The specimen was fixed in formalin, as per standard histopathology operating procedures, and was then CT scanned and cut into millimetre-thick tissue slices. A subset of the tissue slices that included both tumour and fiducial markers was taken and embedded in paraffin blocks. Subsequently, microtome sectioning and haematoxylin and eosin staining were performed to produce 5-μm-thick tissue sections for microscopic digitisation. A series of rigid registration procedures was performed between the different imaging modalities (PET; in vivo CT-i.e. the CT component of the PET-CT; ex vivo CT; histology slices) with the ex vivo CT serving as the reference image. In vivo and ex vivo CTs were registered using landmark-based registration. Histopathology and ex vivo CT images were aligned using the sea urchin spines with additional anatomical landmarks where available. Registration errors were estimated using a leave-one-out strategy for in vivo to ex vivo CT and were estimated from the RMS landmark accuracy for histopathology to ex vivo CT.

RESULTS

The mean ± SD accuracy for registration of the in vivo to ex vivo CT images was 2.66 ± 0.66 mm, and the accuracy for registration of histopathology to ex vivo CT was 0.86 ± 0.41 mm. Estimating the PET to in vivo CT registration accuracy to equal the PET-CT alignment accuracy of 1 mm resulted in an overall average registration error between PET and histopathology slices of 3.0 ± 0.7 mm.

CONCLUSIONS

We have developed a registration method to align PET images and histopathology slices with an accuracy comparable to the spatial resolution of the PET images.

Biomarkers in preclinical cancer imaging

In view of the trend towards personalized treatment strategies for (cancer) patients, there is an increasing need to noninvasively determine individual patient characteristics. Such information enables physicians to administer to patients accurate therapy with appropriate timing. For the noninvasive visualization of disease-related features, imaging biomarkers are expected to play a crucial role. Next to the chemical development of imaging probes, this requires preclinical studies in animal tumour models. These studies provide proof-of-concept of imaging biomarkers and help determine the pharmacokinetics and target specificity of relevant imaging probes, features that provide the fundamentals for translation to the clinic. In this review we describe biological processes derived from the “hallmarks of cancer” that may serve as imaging biomarkers for diagnostic, prognostic and treatment response monitoring that are currently being studied in the preclinical setting. A number of these biomarkers are also being used for the initial preclinical assessment of new intervention strategies. Uniquely, noninvasive imaging approaches allow longitudinal assessment of changes in biological processes, providing information on the safety, pharmacokinetic profiles and target specificity of new drugs, and on the antitumour effectiveness of therapeutic interventions. Preclinical biomarker imaging can help guide translation to optimize clinical biomarker imaging and personalize (combination) therapies.

Correlating 2D histological slice with 3D MRI image volume using smart phone as an interactive tool for muscle study

In muscle dystrophy studies, registration of histological image with MRI image volume enables cross validation of MRI biomarkers using pathological result. However, correlation of 2D histology slice with 3D MRI volume is technically challenging due to the potentially non-orthogonal slice plane and incomplete or distorted histological slice. This paper presents an efficient method to directly perform the 2D-3D registration. The method is unique in that it uses smart phone as a navigation tool for initial alignment followed by an overlap invariant mutual information-based refinement. Experimental results using animal muscle samples images from a 3T MRI and HE stained histological images show that the proposed method is capable of aligning the histological slice with an oblique slice in MR volume.

Automatic segmentation of subcutaneous mouse tumors by multiparametric MR analysis based on endogenous contrast

OBJECT

Contrast-enhanced T1-weighted imaging is usually included in MRI procedures for automatic tumor segmentation. Use of an MR contrast agent may not be appropriate for some applications, however. We assessed the feasability of automatic tumor segmentation by multiparametric cluster analysis that uses intrinsic MRI contrast only.

MATERIALS AND METHODS

Multiparametric MRI consisting of quantitative T1, T2, and apparent diffusion coefficient (ADC) mapping was performed in mice bearing subcutaneous tumors (n = 21). k-means and fuzzy c-means clustering with all possible combinations of MRI parameters, i.e. feature vectors, and 2-7 clusters were performed on the multiparametric data. Clusters associated with tumor tissue were selected on the basis of the relative signal intensity of tumor tissue in T2-weighted images. The optimum segmentation method was determined by quantitative comparison of automatic segmentation with manual segmentation performed by three observers. In addition, the automatically segmented tumor volumes from seven separate tumor data sets were quantitatively compared with histology-derived tumor volumes.

RESULTS

The highest similarity index between manual and automatic segmentation (SI manual,automatic = 0.82 ± 0.06) was observed for k-means clustering with feature vector {T2, ADC} and four clusters. A strong linear correlation between automatically and manually segmented tumor volumes (R (2) = 0.99) was observed for this segmentation method. Automatically segmented tumor volumes also correlated strongly with histology-derived tumor volumes (R (2) = 0.96).

CONCLUSION

Automatic segmentation of mouse subcutaneous tumors can be achieved on the basis of endogenous MR contrast only.

Robust image registration of biological microscopic images

Image registration of biological data is challenging as complex deformation problems are common. Possible deformation effects can be caused in individual data preparation processes, involving morphological deformations, stain variations, stain artifacts, rotation, translation, and missing tissues. The combining deformation effects tend to make existing automatic registration methods perform poor. In our experiments on serial histopathological images, the six state of the art image registration techniques, including TrakEM2, SURF + affine transformation, UnwarpJ, bUnwarpJ, CLAHE + bUnwarpJ and BrainAligner, achieve no greater than 70% averaged accuracies, while the proposed method achieves 91.49% averaged accuracy. The proposed method has also been demonstrated to be significantly better in alignment of laser scanning microscope brain images and serial ssTEM images than the benchmark automatic approaches (p < 0.001). The contribution of this study is to introduce a fully automatic, robust and fast image registration method for 2D image registration.

Multiparametric MRI analysis for the identification of high intensity focused ultrasound-treated tumor tissue

Purpose

In this study endogenous magnetic resonance imaging (MRI) biomarkers for accurate segmentation of High Intensity Focused Ultrasound (HIFU)-treated tumor tissue and residual or recurring non-treated tumor tissue were identified.

Methods

Multiparametric MRI, consisting of quantitative T₁, T₂, Apparent Diffusion Coefficient (ADC) and Magnetization Transfer Ratio (MTR) mapping, was performed in tumor-bearing mice before (n = 14), 1 h after (n = 14) and 72 h (n = 7) after HIFU treatment. A non-treated control group was included (n = 7). Cluster analysis using the Iterative Self Organizing Data Analysis (ISODATA) technique was performed on subsets of MRI parameters (feature vectors). The clusters resulting from the ISODATA segmentation were divided into a viable and non-viable class based on the fraction of pixels assigned to the clusters at the different experimental time points. ISODATA-derived non-viable tumor fractions were quantitatively compared to histology-derived non-viable tumor volume fractions.

Results

The highest agreement between the ISODATA-derived and histology-derived non-viable tumor fractions was observed for feature vector {T₁, T₂, ADC}. R₁ (1/T₁), R₂ (1/T₂), ADC and MTR each were significantly increased in the ISODATA-defined non-viable tumor tissue at 1 h after HIFU treatment compared to viable, non-treated tumor tissue. R₁, ADC and MTR were also significantly increased at 72 h after HIFU.

Conclusions

This study demonstrates that non-viable, HIFU-treated tumor tissue can be distinguished from viable, non-treated tumor tissue using multiparametric MRI analysis. Clinical application of the presented methodology may allow for automated, accurate and objective evaluation of HIFU treatment.

Atherosclerotic plaque component segmentation in combined carotid MRI and CTA data incorporating class label uncertainty

Atherosclerotic plaque composition can indicate plaque vulnerability. We segment atherosclerotic plaque components from the carotid artery on a combination of in vivo MRI and CT-angiography (CTA) data using supervised voxelwise classification. In contrast to previous studies the ground truth for training is directly obtained from 3D registration with histology for fibrous and lipid-rich necrotic tissue, and with μCT for calcification. This registration does, however, not provide accurate voxelwise correspondence. We therefore evaluate three approaches that incorporate uncertainty in the ground truth used for training: I) soft labels are created by Gaussian blurring of the original binary histology segmentations to reduce weights at the boundaries between components, and are weighted by the estimated registration accuracy of the histology and in vivo imaging data (measured by overlap), II) samples are weighted by the local contour distance of the lumen and outer wall between histology and in vivo data, and III) 10% of each class is rejected by Gaussian outlier rejection. Classification was evaluated on the relative volumes (% of tissue type in the vessel wall) for calcified, fibrous and lipid-rich necrotic tissue, using linear discriminant (LDC) and support vector machine (SVM) classification. In addition, the combination of MRI and CTA data was compared to using only one imaging modality. Best results were obtained by LDC and outlier rejection: the volume error per vessel was 0.9±1.0% for calcification, 12.7±7.6% for fibrous and 12.1±8.1% for necrotic tissue, with Spearman rank correlation coefficients of 0.91 (calcification), 0.80 (fibrous) and 0.81 (necrotic). While segmentation using only MRI features yielded low accuracy for calcification, and segmentation using only CTA features yielded low accuracy for necrotic tissue, the combination of features from MRI and CTA gave good results for all studied components.

3D prostate histology reconstruction: an evaluation of image-based and fiducial-based algorithms

PURPOSE

Evaluation of in vivo prostate imaging modalities for determining the spatial distribution and aggressiveness of prostate cancer ideally requires accurate registration of images to an accepted reference standard, such as histopathological examination of radical prostatectomy specimens. Three-dimensional (3D) reconstruction of prostate histology facilitates these registration-based evaluations by reintroducing 3D spatial information lost during histology processing. Because the reconstruction accuracy may constrain the clinical questions that can be answered with these data, it is important to assess the tradeoffs between minimally disruptive methods based on intrinsic image information and potentially more robust methods based on extrinsic fiducial markers.

METHODS

Ex vivo magnetic resonance (MR) images and digitized whole-mount histology images from 12 radical prostatectomy specimens were used to evaluate four 3D histology reconstruction algorithms. 3D reconstructions were computed by registering each histology image to the corresponding ex vivo MR image using one of two similarity metrics (mutual information or fiducial registration error) and one of two search domains (affine transformations or a constrained subset thereof). The algorithms were evaluated for accuracy using the mean target registration error (TRE) computed from homologous intrinsic point landmarks (3-16 per histology section; 232 total) identified on histology and MR images, and for the sensitivity of TRE to rotational, translational, and scaling initialization errors.

RESULTS

The algorithms using fiducial registration error and mutual information had mean ± standard deviation TREs of 0.7 ± 0.4 and 1.2 ± 0.7 mm, respectively, and one algorithm using fiducial registration error and affine transforms had negligible sensitivities to initialization errors. The postoptimization values of the mutual information-based metric showed evidence of errors due to both the optimizer and the similarity metric, and variation of parameters of the mutual information-based metric did not improve its performance.

CONCLUSIONS

The extrinsic fiducial-based algorithm had lower mean TRE and lower sensitivity to initialization than the intrinsic intensity-based algorithm using mutual information. A model relating statistical power to registration error for certain imaging validation study designs estimated that a reconstruction algorithm with a mean TRE of 0.7 mm would require 27% fewer subjects than the method used to initialize the algorithms (mean TRE 1.3 ± 0.7 mm), suggesting the choice of reconstruction technique can have a substantial impact on the design of imaging validation studies, and on their overall cost.

Contrast enhanced-magnetic resonance imaging as a surrogate to map verteporfin delivery in photodynamic therapy

The use of in vivo contrast-enhanced magnetic resonance (MR) imaging as a surrogate for photosensitizer (verteporfin) dosimetry in photodynamic therapy of pancreas cancer is demonstrated by correlating MR contrast uptake to ex vivo fluorescence images on excised tissue. An orthotopic pancreatic xenograft mouse model was used for the study. A strong correlation (r = 0.57) was found for bulk intensity measurements of T1-weighted gadolinium enhancement and verteporfin fluorescence in the tumor region of interest. The use of contrast-enhanced MR imaging shows promise as a method for treatment planning and photosensitizer dosimetry in human photodynamic therapy (PDT) of pancreas cancer.

3D reconstruction of 2D fluorescence histology images and registration with in vivo MR images: application in a rodent stroke model

To validate and add value to non-invasive imaging techniques, the corresponding histology is required to establish biological correlates. We present an efficient, semi-automated image-processing pipeline that uses immunohistochemically stained sections to reconstruct a 3D brain volume from 2D histological images before registering these with the corresponding 3D in vivo magnetic resonance images (MRI). A multistep registration procedure that first aligns the "global" volume by using the centre of mass and then applies a rigid and affine alignment based on signal intensities is described. This technique was applied to a training set of three rat brain volumes before being validated on three normal brains. Application of the approach to register "abnormal" images from a rat model of stroke allowed the neurobiological correlates of the variations in the hyper-intense MRI signal intensity caused by infarction to be investigated. For evaluation, the corresponding anatomical landmarks in MR and histology were defined to measure the registration accuracy. A registration error of 0.249 mm (approximately one in-plane voxel dimension) was evident in healthy rat brains and of 0.323 mm in a rodent model of stroke. The proposed reconstruction and registration pipeline allowed for the precise analysis of non-invasive MRI and corresponding microstructural histological features in 3D. We were thus able to interrogate histology to deduce the cause of MRI signal variations in the lesion cavity and the peri-infarct area.

A Practical Approach Based on Analytic Deformable Algorithm for Scenic Image Registration

Background

Image registration is to produce an entire scene by aligning all the acquired image sequences. A registration algorithm is necessary to tolerance as much as possible for intensity and geometric variation among images. However, captured image views of real scene usually produce unexpected distortions. They are generally derived from the optic characteristics of image sensors or caused by the specific scenes and objects.

Methods and Findings

An analytic registration algorithm considering the deformation is proposed for scenic image applications in this study. After extracting important features by the wavelet-based edge correlation method, an analytic registration approach is then proposed to achieve deformable and accurate matching of point sets. Finally, the registration accuracy is further refined to obtain subpixel precision by a feature-based Levenberg-Marquardt (FLM) method. It converges evidently faster than most other methods because of its feature-based characteristic.

Conclusions

We validate the performance of proposed method by testing with synthetic and real image sequences acquired by a hand-held digital still camera (DSC) and in comparison with an optical flow-based motion technique in terms of the squared sum of intensity differences (SSD) and correlation coefficient (CC). The results indicate that the proposed method is satisfactory in the registration accuracy and quality of DSC images.

Correlated sodium and potassium imbalances within the ischemic core in experimental stroke: a 23Na MRI and histochemical imaging study

This study addresses the spatial relation between local Na(+) and K(+) imbalances in the ischemic core in a rat model of focal ischemic stroke. Quantitative [Na(+)] and [K(+)] brain maps were obtained by (23)Na MRI and histochemical K(+) staining, respectively, and calibrated by emission flame photometry of the micropunch brain samples. Stroke location was verified by diffusion MRI, by changes in tissue surface reflectivity and by immunohistochemistry with microtubule-associated protein 2 antibody. Na(+) and K(+) distribution within the ischemic core was inhomogeneous, with the maximum [Na(+)] increase and [K(+)] decrease typically observed in peripheral regions of the ischemic core. The pattern of the [K(+)] decrease matched the maximum rate of [Na(+)] increase ('slope'). Some residual mismatch between the sites of maximum Na(+) and K(+) imbalances was attributed to the different channels and pathways involved in transport of the two ions. A linear regression of the [Na(+)]br vs. [K(+)]br in the samples of ischemic brain indicates that for each K(+) equivalent leaving ischemic tissue, 0.8±0.1 Eq, on average, of Na(+) enter the tissue. Better understanding of the mechanistic link between the Na(+) influx and K(+) egress would validate the (23)Na MRI slope as a candidate biomarker and a complementary tool for assessing ischemic damage and treatment planning.

Improved image alignment method in application to X-ray images and biological images

MOTIVATION

Alignment of medical images is a vital component of a large number of applications throughout the clinical track of events; not only within clinical diagnostic settings, but prominently so in the area of planning, consummation and evaluation of surgical and radiotherapeutical procedures. However, image registration of medical images is challenging because of variations on data appearance, imaging artifacts and complex data deformation problems. Hence, the aim of this study is to develop a robust image alignment method for medical images.

RESULTS

An improved image registration method is proposed, and the method is evaluated with two types of medical data, including biological microscopic tissue images and dental X-ray images and compared with five state-of-the-art image registration techniques. The experimental results show that the presented method consistently performs well on both types of medical images, achieving 88.44 and 88.93% averaged registration accuracies for biological tissue images and X-ray images, respectively, and outperforms the benchmark methods. Based on the Tukey's honestly significant difference test and Fisher's least square difference test tests, the presented method performs significantly better than all existing methods (P ≤ 0.001) for tissue image alignment, and for the X-ray image registration, the proposed method performs significantly better than the two benchmark b-spline approaches (P < 0.001).

AVAILABILITY

The software implementation of the presented method and the data used in this study are made publicly available for scientific communities to use (http://www-o.ntust.edu.tw/∼cweiwang/ImprovedImageRegistration/).

CONTACT

cweiwang@mail.ntust.edu.tw.

Registration accuracy and quality of real-life images

Background

A common registration problem for the application of consumer device is to align all the acquired image sequences into a complete scene. Image alignment requires a registration algorithm that will compensate as much as possible for geometric variability among images. However, images captured views from a real scene usually produce different distortions. Some are derived from the optic characteristics of image sensors, and others are caused by the specific scenes and objects.

Methodology/Principal Findings

An image registration algorithm considering the perspective projection is proposed for the application of consumer devices in this study. It exploits a multiresolution wavelet-based method to extract significant features. An analytic differential approach is then proposed to achieve fast convergence of point matching. Finally, the registration accuracy is further refined to obtain subpixel precision by a feature-based modified Levenberg-Marquardt method. Due to its feature-based and nonlinear characteristic, it converges considerably faster than most other methods. In addition, vignette compensation and color difference adjustment are also performed to further improve the quality of registration results.

Conclusions/Significance

The performance of the proposed method is evaluated by testing the synthetic and real images acquired by a hand-held digital still camera and in comparison with two registration techniques in terms of the squared sum of intensity differences (SSD) and correlation coefficient (CC). The results indicate that the proposed method is promising in registration accuracy and quality, which are statistically significantly better than other two approaches.