Towards Realistic 3D Models of Tumor Vascular Networks

For reliable in silico or in vitro investigations in, for example, biosensing and drug delivery applications, accurate models of tumor vascular networks down to the capillary size are essential. Compared to images acquired with conventional medical imaging techniques, digitalized histological tumor slices have a higher resolution, enabling the delineation of capillaries. Volume rendering procedures can then be used to generate a 3D model. However, the preparation of such slices leads to misalignments in relative slice orientation between consecutive slices. Thus, image registration algorithms are necessary to re-align the slices. Here, we present an algorithm for the registration and reconstruction of a vascular network from histologic slices applied to 169 tumor slices. The registration includes two steps. First, consecutive images are incrementally pre-aligned using feature- and area-based transformations. Second, using the previous transformations, parallel registration for all images is enabled. Combining intensity- and color-based thresholds along with heuristic analysis, vascular structures are segmented. A 3D interpolation technique is used for volume rendering. This results in a 3D vascular network with approximately 400-450 vessels with diameters down to 25-30 µm. A delineation of vessel structures with close distance was limited in areas of high structural density. Improvement can be achieved by using images with higher resolution and or machine learning techniques.

Comparative analysis of tissue reconstruction algorithms for 3D histology

Motivation

Digital pathology enables new approaches that expand beyond storage, visualization or analysis of histological samples in digital format. One novel opportunity is 3D histology, where a three-dimensional reconstruction of the sample is formed computationally based on serial tissue sections. This allows examining tissue architecture in 3D, for example, for diagnostic purposes. Importantly, 3D histology enables joint mapping of cellular morphology with spatially resolved omics data in the true 3D context of the tissue at microscopic resolution. Several algorithms have been proposed for the reconstruction task, but a quantitative comparison of their accuracy is lacking.

Results

We developed a benchmarking framework to evaluate the accuracy of several free and commercial 3D reconstruction methods using two whole slide image datasets. The results provide a solid basis for further development and application of 3D histology algorithms and indicate that methods capable of compensating for local tissue deformation are superior to simpler approaches.

Availability and implementation

Code: https://github.com/BioimageInformaticsTampere/RegBenchmark. Whole slide image datasets: http://urn.fi/urn: nbn: fi: csc-kata20170705131652639702.

Supplementary information

Supplementary data are available at Bioinformatics online.

Accurate validation of ultrasound imaging of prostate cancer: a review of challenges in registration of imaging and histopathology

As the development of modalities for prostate cancer (PCa) imaging advances, the challenge of accurate registration between images and histopathologic ground truth becomes more pressing. Localization of PCa, rather than detection, requires a pixel-to-pixel validation of imaging based on histopathology after radical prostatectomy. Such a registration procedure is challenging for ultrasound modalities; not only the deformations of the prostate after resection have to be taken into account, but also the deformation due to the employed transrectal probe and the mismatch in orientation between imaging planes and pathology slices. In this work, we review the latest techniques to facilitate accurate validation of PCa localization in ultrasound imaging studies and extrapolate a general strategy for implementation of a registration procedure.

A method for accurate spatial registration of PET images and histopathology slices

BACKGROUND

Accurate alignment between histopathology slices and positron emission tomography (PET) images is important for radiopharmaceutical validation studies. Limited data is available on the registration accuracy that can be achieved between PET and histopathology slices acquired under routine pathology conditions where slices may be non-parallel, non-contiguously cut and of standard block size. The purpose of this study was to demonstrate a method for aligning PET images and histopathology slices acquired from patients with laryngeal cancer and to assess the registration accuracy obtained under these conditions.

METHODS

Six subjects with laryngeal cancer underwent a (64)Cu-copper-II-diacetyl-bis(N4-methylthiosemicarbazone) ((64)Cu-ATSM) PET computed tomography (CT) scan prior to total laryngectomy. Sea urchin spines were inserted into the pathology specimen to act as fiducial markers. The specimen was fixed in formalin, as per standard histopathology operating procedures, and was then CT scanned and cut into millimetre-thick tissue slices. A subset of the tissue slices that included both tumour and fiducial markers was taken and embedded in paraffin blocks. Subsequently, microtome sectioning and haematoxylin and eosin staining were performed to produce 5-μm-thick tissue sections for microscopic digitisation. A series of rigid registration procedures was performed between the different imaging modalities (PET; in vivo CT-i.e. the CT component of the PET-CT; ex vivo CT; histology slices) with the ex vivo CT serving as the reference image. In vivo and ex vivo CTs were registered using landmark-based registration. Histopathology and ex vivo CT images were aligned using the sea urchin spines with additional anatomical landmarks where available. Registration errors were estimated using a leave-one-out strategy for in vivo to ex vivo CT and were estimated from the RMS landmark accuracy for histopathology to ex vivo CT.

RESULTS

The mean ± SD accuracy for registration of the in vivo to ex vivo CT images was 2.66 ± 0.66 mm, and the accuracy for registration of histopathology to ex vivo CT was 0.86 ± 0.41 mm. Estimating the PET to in vivo CT registration accuracy to equal the PET-CT alignment accuracy of 1 mm resulted in an overall average registration error between PET and histopathology slices of 3.0 ± 0.7 mm.

CONCLUSIONS

We have developed a registration method to align PET images and histopathology slices with an accuracy comparable to the spatial resolution of the PET images.

A Method for 3D Histopathology Reconstruction Supporting Mouse Microvasculature Analysis

Structural abnormalities of the microvasculature can impair perfusion and function. Conventional histology provides good spatial resolution with which to evaluate the microvascular structure but affords no 3-dimensional information; this limitation could lead to misinterpretations of the complex microvessel network in health and disease. The objective of this study was to develop and evaluate an accurate, fully automated 3D histology reconstruction method to visualize the arterioles and venules within the mouse hind-limb. Sections of the tibialis anterior muscle from C57BL/J6 mice (both normal and subjected to femoral artery excision) were reconstructed using pairwise rigid and affine registrations of 5 µm-thick, paraffin-embedded serial sections digitized at 0.25 µm/pixel. Low-resolution intensity-based rigid registration was used to initialize the nucleus landmark-based registration, and conventional high-resolution intensity-based registration method. The affine nucleus landmark-based registration was developed in this work and was compared to the conventional affine high-resolution intensity-based registration method. Target registration errors were measured between adjacent tissue sections (pairwise error), as well as with respect to a 3D reference reconstruction (accumulated error, to capture propagation of error through the stack of sections). Accumulated error measures were lower (p < 0.01) for the nucleus landmark technique and superior vasculature continuity was observed. These findings indicate that registration based on automatic extraction and correspondence of small, homologous landmarks may support accurate 3D histology reconstruction. This technique avoids the otherwise problematic "banana-into-cylinder" effect observed using conventional methods that optimize the pairwise alignment of salient structures, forcing them to be section-orthogonal. This approach will provide a valuable tool for high-accuracy 3D histology tissue reconstructions for analysis of diseased microvasculature.

Prostate whole-mount histology reconstruction and registration to MRI for correlating in-vivo observations with biological findings

Multi-parametric magnetic resonance imaging (mMRI) is the standard exam for prostate cancer diagnosis, staging and risk assessment in current clinical routine. Correlating mMRI in-vivo observations with biological findings from radical prostatectomy specimen would improve the optimal therapy selection. Thus, we proposed a method for reconstructing and registering the prostate whole-mount histology (WMH) to the MRI, considering a thin slicing of the prostatectomy specimen. The method was evaluated on 3 patients, included in a prospective study, for which hematein-eosinsafran and immunohistochemistry stainings were performed. The registration error was assessed by measuring the Euclidean distance between landmarks, previously identified by an expert on both mMRI and histological slices. The mean error was 4:90α1:34 mm. Our method demonstrated promising results for registering prostate WMH to in-vivo mMRI, thus allowing for spatial accurate correlation between radiologic observations and biological information.

How accurate is multiparametric MR imaging in evaluation of prostate cancer volume?

PURPOSE

To assess the factors influencing multiparametric (MP) magnetic resonance (MR) imaging accuracy in estimating prostate cancer histologic volume (Vh).

MATERIALS AND METHODS

A prospective database of 202 patients who underwent MP MR imaging before radical prostatectomy was retrospectively used. Institutional review board approval and informed consent were obtained. Two independent radiologists delineated areas suspicious for cancer on images (T2-weighted, diffusion-weighted, dynamic contrast material-enhanced [DCE] pulse sequences) and scored their degree of suspicion of malignancy by using a five-level Likert score. One pathologist delineated cancers on whole-mount prostatectomy sections and calculated their volume by using digitized planimetry. Volumes of MR true-positive lesions were measured on T2-weighted images (VT2), on ADC maps (VADC), and on DCE images [VDCE]). VT2, VADC, VDCE and the greatest volume determined on images from any of the individual MR pulse sequences (Vmax) were compared with Vh (Bland-Altman analysis). Factors influencing MP MR imaging accuracy, or A, calculated as A = Vmax/Vh, were evaluated using generalized linear mixed models.

RESULTS

For both readers, Vh was significantly underestimated with VT2 (P < .0001, both), VADC (P < .0001, both), and VDCE (P = .02 and P = .003, readers 1 and 2, respectively), but not with Vmax (P = .13 and P = .21, readers 1 and 2, respectively). Mean, 25th percentile, and 75th percentile, respectively, for Vmax accuracy were 0.92, 0.54, and 1.85 for reader 1 and 0.95, 0.57, and 1.77 for reader 2. At generalized linear mixed (multivariate) analysis, tumor Likert score (P < .0001), Gleason score (P = .009), and Vh (P < .0001) significantly influenced Vmax accuracy (both readers). This accuracy was good in tumors with a Gleason score of 7 or higher or a Likert score of 5, with a tendency toward underestimation of Vh; accuracy was poor in small (<0.5 cc) or low-grade (Gleason score ≤6) tumors, with a tendency toward overestimation of Vh.

CONCLUSION

Vh can be estimated by using Vmax in aggressive tumors or in tumors with high Likert scores.

A Workflow to Improve the Alignment of Prostate Imaging with Whole-mount Histopathology

RATIONALE AND OBJECTIVES

Evaluation of prostate imaging tests against whole-mount histology specimens requires accurate alignment between radiologic and histologic data sets. Misalignment results in false-positive and -negative zones as assessed by imaging. We describe a workflow for three-dimensional alignment of prostate imaging data against whole-mount prostatectomy reference specimens and assess its performance against a standard workflow.

MATERIALS AND METHODS

Ethical approval was granted. Patients underwent motorized transrectal ultrasound (Prostate Histoscanning) to generate a three-dimensional image of the prostate before radical prostatectomy. The test workflow incorporated steps for axial alignment between imaging and histology, size adjustments following formalin fixation, and use of custom-made parallel cutters and digital caliper instruments. The control workflow comprised freehand cutting and assumed homogeneous block thicknesses at the same relative angles between pathology and imaging sections.

RESULTS

Thirty radical prostatectomy specimens were histologically and radiologically processed, either by an alignment-optimized workflow (n = 20) or a control workflow (n = 10). The optimized workflow generated tissue blocks of heterogeneous thicknesses but with no significant drifting in the cutting plane. The control workflow resulted in significantly nonparallel blocks, accurately matching only one out of four histology blocks to their respective imaging data. The image-to-histology alignment accuracy was 20% greater in the optimized workflow (P < .0001), with higher sensitivity (85% vs. 69%) and specificity (94% vs. 73%) for margin prediction in a 5 × 5-mm grid analysis.

CONCLUSIONS

A significantly better alignment was observed in the optimized workflow. Evaluation of prostate imaging biomarkers using whole-mount histology references should include a test-to-reference spatial alignment workflow.

3D prostate histology reconstruction: an evaluation of image-based and fiducial-based algorithms

PURPOSE

Evaluation of in vivo prostate imaging modalities for determining the spatial distribution and aggressiveness of prostate cancer ideally requires accurate registration of images to an accepted reference standard, such as histopathological examination of radical prostatectomy specimens. Three-dimensional (3D) reconstruction of prostate histology facilitates these registration-based evaluations by reintroducing 3D spatial information lost during histology processing. Because the reconstruction accuracy may constrain the clinical questions that can be answered with these data, it is important to assess the tradeoffs between minimally disruptive methods based on intrinsic image information and potentially more robust methods based on extrinsic fiducial markers.

METHODS

Ex vivo magnetic resonance (MR) images and digitized whole-mount histology images from 12 radical prostatectomy specimens were used to evaluate four 3D histology reconstruction algorithms. 3D reconstructions were computed by registering each histology image to the corresponding ex vivo MR image using one of two similarity metrics (mutual information or fiducial registration error) and one of two search domains (affine transformations or a constrained subset thereof). The algorithms were evaluated for accuracy using the mean target registration error (TRE) computed from homologous intrinsic point landmarks (3-16 per histology section; 232 total) identified on histology and MR images, and for the sensitivity of TRE to rotational, translational, and scaling initialization errors.

RESULTS

The algorithms using fiducial registration error and mutual information had mean ± standard deviation TREs of 0.7 ± 0.4 and 1.2 ± 0.7 mm, respectively, and one algorithm using fiducial registration error and affine transforms had negligible sensitivities to initialization errors. The postoptimization values of the mutual information-based metric showed evidence of errors due to both the optimizer and the similarity metric, and variation of parameters of the mutual information-based metric did not improve its performance.

CONCLUSIONS

The extrinsic fiducial-based algorithm had lower mean TRE and lower sensitivity to initialization than the intrinsic intensity-based algorithm using mutual information. A model relating statistical power to registration error for certain imaging validation study designs estimated that a reconstruction algorithm with a mean TRE of 0.7 mm would require 27% fewer subjects than the method used to initialize the algorithms (mean TRE 1.3 ± 0.7 mm), suggesting the choice of reconstruction technique can have a substantial impact on the design of imaging validation studies, and on their overall cost.

Tumor target volume for focal therapy of prostate cancer-does multiparametric magnetic resonance imaging allow for a reliable estimation?

PURPOSE

We determined whether endorectal multiparametric magnetic resonance imaging at 1.5 Tesla could predict tumor target volume in the perspective of focal therapy of prostate cancer.

MATERIALS AND METHODS

A total of 84 consecutive patients underwent multiparametric magnetic resonance imaging before radical prostatectomy. The volume of each suspicious area detected on magnetic resonance imaging and of all surgical histological foci was determined by planimetry. We first used each magnetic resonance imaging sequence (T2-weighted, diffusion weighted and dynamic contrast enhanced) and then the sequence showing the largest tumor area (multiparametric volume). Finally, the largest area of any sequence was used to calculate a target volume according to the volume of a cylinder. Agreement between magnetic resonance imaging and pathological findings was assessed by linear regression and residual analysis.

RESULTS

Histology revealed 99 significant tumors with a volume of greater than 0.2 cc and/or a Gleason score of greater than 6. Of the tumors 16 (16.2%) were undetected by multiparametric magnetic resonance imaging. Linear regression analysis showed that tumor volume estimated by T2-weighted or diffusion weighted imaging correlated significantly with pathological volume (r(2) = 0.82 and 0.83, respectively). Residuals from diffusion weighted imaging volume estimations did not significantly differ from 0. Nevertheless, diffusion weighted imaging underestimated pathological volume in 43 of 87 cases (49%) by a mean of 0.56 cc (range 0.005 to 2.84). Multiparametric and target volumes significantly overestimated pathological volume by a mean of 16% and 44% with underestimation in 28 (32%) and 15 cases (17%), respectively. Volume underestimation was significantly higher for tumor foci less than 0.5 cc. The percent of Gleason grade 4 did not influence tumor volume estimation.

CONCLUSIONS

Magnetic resonance imaging can detect most significant tumors. However, delineating a target volume may require further adjustment before planning magnetic resonance imaging targeted focal treatment.

3D prostate histology image reconstruction: Quantifying the impact of tissue deformation and histology section location

Background:

Guidelines for localizing prostate cancer on imaging are ideally informed by registered post-prostatectomy histology. 3D histology reconstruction methods can support this by reintroducing 3D spatial information lost during histology processing. The need to register small, high-grade foci drives a need for high accuracy. Accurate 3D reconstruction method design is impacted by the answers to the following central questions of this work. (1) How does prostate tissue deform during histology processing? (2) What spatial misalignment of the tissue sections is induced by microtome cutting? (3) How does the choice of reconstruction model affect histology reconstruction accuracy?

Materials and Methods:

Histology, paraffin block face and magnetic resonance images were acquired for 18 whole mid-gland tissue slices from six prostates. 7-15 homologous landmarks were identified on each image. Tissue deformation due to histology processing was characterized using the target registration error (TRE) after landmark-based registration under four deformation models (rigid, similarity, affine and thin-plate-spline [TPS]). The misalignment of histology sections from the front faces of tissue slices was quantified using manually identified landmarks. The impact of reconstruction models on the TRE after landmark-based reconstruction was measured under eight reconstruction models comprising one of four deformation models with and without constraining histology images to the tissue slice front faces.

Results:

Isotropic scaling improved the mean TRE by 0.8-1.0 mm (all results reported as 95% confidence intervals), while skew or TPS deformation improved the mean TRE by <0.1 mm. The mean misalignment was 1.1-1.9^° (angle) and 0.9-1.3 mm (depth). Using isotropic scaling, the front face constraint raised the mean TRE by 0.6-0.8 mm.

Conclusions:

For sub-millimeter accuracy, 3D reconstruction models should not constrain histology images to the tissue slice front faces and should be flexible enough to model isotropic scaling.