Registration of whole-mount histology and volumetric imaging of the prostate using particle filtering

A review of artificial intelligence in prostate cancer detection on imaging

A multitude of studies have explored the role of artificial intelligence (AI) in providing diagnostic support to radiologists, pathologists, and urologists in prostate cancer detection, risk-stratification, and management. This review provides a comprehensive overview of relevant literature regarding the use of AI models in (1) detecting prostate cancer on radiology images (magnetic resonance and ultrasound imaging), (2) detecting prostate cancer on histopathology images of prostate biopsy tissue, and (3) assisting in supporting tasks for prostate cancer detection (prostate gland segmentation, MRI-histopathology registration, MRI-ultrasound registration). We discuss both the potential of these AI models to assist in the clinical workflow of prostate cancer diagnosis, as well as the current limitations including variability in training data sets, algorithms, and evaluation criteria. We also discuss ongoing challenges and what is needed to bridge the gap between academic research on AI for prostate cancer and commercial solutions that improve routine clinical care.

Correlation of in-vivo imaging with histopathology: A review

Despite tremendous advancements in in vivo imaging modalities, there remains substantial uncertainty with respect to tumor delineation on in these images. Histopathology remains the gold standard for determining the extent of malignancy, with in vivo imaging to histopathologic correlation enabling spatial comparisons. In this review, the steps necessary for successful imaging to histopathologic correlation are described, including in vivo imaging, resection, fixation, specimen sectioning (sectioning technique, securing technique, orientation matching, slice matching), microtome sectioning and staining, correlation (including image registration) and performance evaluation. The techniques used for each of these steps are also discussed. Hundreds of publications from the past 20 years were surveyed, and 62 selected for detailed analysis. For these 62 publications, each stage of the correlative pathology process (and the sub-steps of specimen sectioning) are listed. A statistical analysis was conducted based on 19 studies that reported target registration error as their performance metric. While some methods promise greater accuracy, they may be expensive. Due to the complexity of the processes involved, correlative pathology studies generally include a small number of subjects, which hinders advanced developments in this field.

Registration of Untracked 2D Laparoscopic Ultrasound to CT Images of the Liver Using Multi-Labelled Content-Based Image Retrieval

Laparoscopic Ultrasound (LUS) is recommended as a standard-of-care when performing laparoscopic liver resections as it images sub-surface structures such as tumours and major vessels. Given that LUS probes are difficult to handle and some tumours are iso-echoic, registration of LUS images to a pre-operative CT has been proposed as an image-guidance method. This registration problem is particularly challenging due to the small field of view of LUS, and usually depends on both a manual initialisation and tracking to compose a volume, hindering clinical translation. In this paper, we extend a proposed registration approach using Content-Based Image Retrieval (CBIR), removing the requirement for tracking or manual initialisation. Pre-operatively, a set of possible LUS planes is simulated from CT and a descriptor generated for each image. Then, a Bayesian framework is employed to estimate the most likely sequence of CT simulations that matches a series of LUS images. We extend our CBIR formulation to use multiple labelled objects and constrain the registration by separating liver vessels into portal vein and hepatic vein branches. The value of this new labeled approach is demonstrated in retrospective data from 5 patients. Results show that, by including a series of 5 untracked images in time, a single LUS image can be registered with accuracies ranging from 5.7 to 16.4 mm with a success rate of 78%. Initialisation of the LUS to CT registration with the proposed framework could potentially enable the clinical translation of these image fusion techniques.

Histological Validation of MRI: A Review of Challenges in Registration of Imaging and Whole-Mount Histopathology

Rigorous validation with ground truth information such as histology is needed to reliably assess the current and potential value of MRI techniques to characterize tissue and identify disease-related tissue alterations. Commonly used methods that aim to directly correlate histology and MRI data generally fall short of this goal due to spatial errors that preclude direct matching. Errors result from tissue deformation, differences in spatial resolution and slice thickness, non-coplanar and/or nonintersecting plane orientations, and different image contrast mechanisms. Some of these problems arise from limitations in standard protocols for clinical tissue processing and histology-based pathology reporting, and to some extent can be addressed by modifications to standard protocols without compromising the clinical process. Typical modifications include ex vivo specimen MRI, block-face photography, addition of fiducial markers, and 3D printed molds to constrain tissue deformation and guide sectioning. This review summarizes the advantages and limitations of MRI validation techniques based on coregistration of MRI with whole-mount histology of tissue specimens. LEVEL OF EVIDENCE: 4 TECHNICAL EFFICACY STAGE: 1.

Accurate validation of ultrasound imaging of prostate cancer: a review of challenges in registration of imaging and histopathology

As the development of modalities for prostate cancer (PCa) imaging advances, the challenge of accurate registration between images and histopathologic ground truth becomes more pressing. Localization of PCa, rather than detection, requires a pixel-to-pixel validation of imaging based on histopathology after radical prostatectomy. Such a registration procedure is challenging for ultrasound modalities; not only the deformations of the prostate after resection have to be taken into account, but also the deformation due to the employed transrectal probe and the mismatch in orientation between imaging planes and pathology slices. In this work, we review the latest techniques to facilitate accurate validation of PCa localization in ultrasound imaging studies and extrapolate a general strategy for implementation of a registration procedure.

Intensity-based volumetric registration of magnetic resonance images and whole-mount sections of the prostate

OBJECTIVE

Magnetic Resonance Imaging (MRI) of the prostate provides useful in vivo diagnostic tissue information such as tumor location and aggressiveness, but ex vivo histopathology remains the ground truth. There are several challenges related to the registration of MRI to histopathology. We present a method for registration of standard clinical T2-weighted MRI (T2W-MRI) and transverse histopathology whole-mount (WM) sections of the prostate.

METHODS

An isotropic volume stack was created from the WM sections using 2D rigid and deformable registration combined with linear interpolation. The prostate was segmented manually from the T2W-MRI volume and registered to the WM section volume using a combination of affine and deformable registration. The method was evaluated on a set of 12 patients who had undergone radical prostatectomy. Registration accuracy was assessed using volume overlap (Dice Coefficient, DC) and landmark distances.

RESULTS

The DC was 0.94 for the whole prostate, 0.63 for the peripheral zone and 0.77 for the remaining gland. The landmark distances were on average 5.4 mm.

CONCLUSION

The volume overlap for the whole prostate and remaining gland, as well as the landmark distances indicate good registration accuracy for the proposed method, and shows that it can be highly useful for registering clinical available MRI and WM sections of the prostate.

Optimized SIFTFlow for registration of whole-mount histology to reference optical images

The registration of two-dimensional histology images to reference images from other modalities is an important preprocessing step in the reconstruction of three-dimensional histology volumes. This is a challenging problem because of the differences in the appearances of histology images and other modalities, and the presence of large nonrigid deformations which occur during slide preparation. This paper shows the feasibility of using densely sampled scale-invariant feature transform (SIFT) features and a SIFTFlow deformable registration algorithm for coregistering whole-mount histology images with blockface optical images. We present a method for jointly optimizing the regularization parameters used by the SIFTFlow objective function and use it to determine the most appropriate values for the registration of breast lumpectomy specimens. We demonstrate that tuning the regularization parameters results in significant improvements in accuracy and we also show that SIFTFlow outperforms a previously described edge-based registration method. The accuracy of the histology images to blockface images registration using the optimized SIFTFlow method was assessed using an independent test set of images from five different lumpectomy specimens and the mean registration error was [Formula: see text].

Deformable image registration between pathological images and MR image via an optical macro image

Computed tomography (CT) and magnetic resonance (MR) imaging have been widely used for visualizing the inside of the human body. However, in many cases, pathological diagnosis is conducted through a biopsy or resection of an organ to evaluate the condition of tissues as definitive diagnosis. To provide more advanced information onto CT or MR image, it is necessary to reveal the relationship between tissue information and image signals. We propose a registration scheme for a set of PT images of divided specimens and a 3D-MR image by reference to an optical macro image (OM image) captured by an optical camera. We conducted a fundamental study using a resected human brain after the death of a brain cancer patient. We constructed two kinds of registration processes using the OM image as the base for both registrations to make conversion parameters between the PT and MR images. The aligned PT images had shapes similar to the OM image. On the other hand, the extracted cross-sectional MR image was similar to the OM image. From these resultant conversion parameters, the corresponding region on the PT image could be searched and displayed when an arbitrary pixel on the MR image was selected. The relationship between the PT and MR images of the whole brain can be analyzed using the proposed method. We confirmed that same regions between the PT and MR images could be searched and displayed using resultant information obtained by the proposed method. In terms of the accuracy of proposed method, the TREs were 0.56±0.39mm and 0.87±0.42mm. We can analyze the relationship between tissue information and MR signals using the proposed method.

Framework for 3D histologic reconstruction and fusion with in vivo MRI: Preliminary results of characterizing pulmonary inflammation in a mouse model

PURPOSE

Pulmonary inflammation is associated with a variety of diseases. Assessing pulmonary inflammation on in vivo imaging may facilitate the early detection and treatment of lung diseases. Although routinely used in thoracic imaging, computed tomography has thus far not been compellingly shown to characterize inflammation in vivo. Alternatively, magnetic resonance imaging (MRI) is a nonionizing radiation technique to better visualize and characterize pulmonary tissue. Prior to routine adoption of MRI for early characterization of inflammation in humans, a rigorous and quantitative characterization of the utility of MRI to identify inflammation is required. Such characterization may be achieved by considering ex vivo histology as the ground truth, since it enables the definitive spatial assessment of inflammation. In this study, the authors introduce a novel framework to integrate 2D histology, ex vivo and in vivo imaging to enable the mapping of the extent of disease from ex vivo histology onto in vivo imaging, with the goal of facilitating computerized feature analysis and interrogation of disease appearance on in vivo imaging. The authors' framework was evaluated in a preclinical preliminary study aimed to identify computer extracted features on in vivo MRI associated with chronic pulmonary inflammation.

METHODS

The authors' image analytics framework first involves reconstructing the histologic volume in 3D from individual histology slices. Second, the authors map the disease ground truth onto in vivo MRI via coregistration with 3D histology using the ex vivo lung MRI as a conduit. Finally, computerized feature analysis of the disease extent is performed to identify candidate in vivo imaging signatures of disease presence and extent.

RESULTS

The authors evaluated the framework by assessing the quality of the 3D histology reconstruction and the histology-MRI fusion, in the context of an initial use case involving characterization of chronic inflammation in a mouse model. The authors' evaluation considered three mice, two with an inflammation phenotype and one control. The authors' iterative 3D histology reconstruction yielded a 70.1% ± 2.7% overlap with the ex vivo MRI volume. Across a total of 17 anatomic landmarks manually delineated at the division of airways, the target registration error between the ex vivo MRI and 3D histology reconstruction was 0.85 ± 0.44 mm, suggesting that a good alignment of the ex vivo 3D histology and ex vivo MRI had been achieved. The 3D histology-in vivo MRI coregistered volumes resulted in an overlap of 73.7% ± 0.9%. Preliminary computerized feature analysis was performed on an additional four control mice, for a total of seven mice considered in this study. Gabor texture filters appeared to best capture differences between the inflamed and noninflamed regions on MRI.

CONCLUSIONS

The authors' 3D histology reconstruction and multimodal registration framework were successfully employed to reconstruct the histology volume of the lung and fuse it with in vivo MRI to create a ground truth map for inflammation on in vivo MRI. The analytic platform presented here lays the framework for a rigorous validation of the identified imaging features for chronic lung inflammation on MRI in a large prospective cohort.

3D surface-based registration of ultrasound and histology in prostate cancer imaging

Several transrectal ultrasound (TRUS)-based techniques aiming at accurate localization of prostate cancer are emerging to improve diagnostics or to assist with focal therapy. However, precise validation prior to introduction into clinical practice is required. Histopathology after radical prostatectomy provides an excellent ground truth, but needs accurate registration with imaging. In this work, a 3D, surface-based, elastic registration method was developed to fuse TRUS images with histopathologic results. To maximize the applicability in clinical practice, no auxiliary sensors or dedicated hardware were used for the registration. The mean registration errors, measured in vitro and in vivo, were 1.5±0.2 and 2.1±0.5mm, respectively.

MR elastography of prostate cancer: quantitative comparison with histopathology and repeatability of methods

The purpose of this work was to assess trans-perineal prostate magnetic resonance elastography (MRE) for (1) repeatability in phantoms/volunteers and (2) diagnostic power as correlated with histopathology in prostate cancer patients. The three-dimensional (3D) displacement field was obtained using a fractionally encoded gradient echo sequence using a custom-made transducer. The repeatability of the method was assessed based on three repeat studies and by changing the driving frequency by 3% in studies on a phantom and six healthy volunteers. Subsequently, 11 patients were examined with MRE prior to radical prostatectomy. The areas under the receiver operating characteristic curves were calculated using a windowed voxel-to-voxel approach by comparing the 2D registered slides, masked with the Gleason score. For the repeatability study, the average intraclass correlation coefficient for elasticity images was 99% for repeat phantom studies, 98% for ±6 Hz phantom studies, 95% for volunteer repeat studies with 2 min acquisition time, 82% for ±2 Hz volunteer studies with 2 min acquisition time and 73% for repeat volunteer studies with 8 min acquisition time. For the patient study, the average elasticity was 8.2 ± 1.7 kPa in the prostate capsule, 7.5 ± 1.9 kPa in the peripheral zone (PZ), 9.7 ± 3.0 kPa in the central gland (CG) and 9.0 ± 3.4 kPa in the transition zone. In the patient study, cancerous tissue with Gleason score at least 3 + 3 was significantly (p < 0.05) different from normal tissue in 10 out of 11 cases with tumors in the PZ, and 6 out of 9 cases with tumors in the CG. However, the overall case-averaged area under the curve was 0.72 in the PZ and 0.67 in the CG. Cancerous tissue was not always stiffer than normal tissue. The inversion algorithm was sensitive to (i) vibration amplitude and displacement nodes and (ii) misalignment of the 3D wave field due to subject movement.

MR elastography and diffusion-weighted imaging of ex vivo prostate cancer: quantitative comparison to histopathology

The purpose of this work was (1) to develop a magnetic resonance elastography (MRE) system for imaging of the ex vivo human prostate and (2) to assess the diagnostic power of mono-frequency and multi-frequency MRE and diffusion weighted imaging (DWI) alone and combined as correlated with histopathology in a patient study. An electromagnetic driver was designed specifically for MRE studies in small-bore MR scanners. Ex vivo prostate specimens (post-fixation) of 14 patients who underwent radical prostatectomy were imaged with MRE at 7 T (nine cases had DWI). In six patients, the MRE examination was performed at three frequencies (600, 800, 1000 Hz) to extract the power-law exponent Gamma. The images were registered to wholemount pathology slides marked with the Gleason score. The areas under the receiver-operator-characteristic curves (AUC) were calculated. The methods were validated in a phantom study and it was demonstrated that (i) the driver does not interfere with the acquisition process and (ii) the driver can generate amplitudes greater than 100 µm for frequencies less than 1 kHz. In the quantitative study, cancerous tissue with Gleason score at least 3 + 3 was distinguished from normal tissue in the peripheral zone (PZ) with an average AUC of 0.75 (Gd ), 0.75 (Gl ), 0.70 (Gamma-Gd ), 0.68 (apparent diffusion coefficient, ADC), and 0.82 (Gd  + Gl  + ADC). The differentiation between PZ and central gland was modest for Gd (p < 0.07), Gl (p < 0.06) but not significant for Gamma (p < 0.2). A correlation of 0.4 kPa/h was found between the fixation time of the prostate specimen and the stiffness of the tissue, which could affect the diagnostic power results. DWI and MRE may provide complementary information; in fact MRE performed better than ADC in distinguishing normal from cancerous tissue in some cases. Multi-frequency (Gamma) analysis did not appear to improve the results. However, in light of the effect of tissue fixation, the clinical implication of our results may be inconclusive and more experiments are needed.