Accurate validation of ultrasound imaging of prostate cancer: a review of challenges in registration of imaging and histopathology

A Guideline for Open-Source Tools to Make Medical Imaging Data Ready for Artificial Intelligence Applications: A Society of Imaging Informatics in Medicine (SIIM) Survey

In recent years, the role of Artificial Intelligence (AI) in medical imaging has become increasingly prominent, with the majority of AI applications approved by the FDA being in imaging and radiology in 2023. The surge in AI model development to tackle clinical challenges underscores the necessity for preparing high-quality medical imaging data. Proper data preparation is crucial as it fosters the creation of standardized and reproducible AI models while minimizing biases. Data curation transforms raw data into a valuable, organized, and dependable resource and is a fundamental process to the success of machine learning and analytical projects. Considering the plethora of available tools for data curation in different stages, it is crucial to stay informed about the most relevant tools within specific research areas. In the current work, we propose a descriptive outline for different steps of data curation while we furnish compilations of tools collected from a survey applied among members of the Society of Imaging Informatics (SIIM) for each of these stages. This collection has the potential to enhance the decision-making process for researchers as they select the most appropriate tool for their specific tasks.

[68Ga]Ga-PSMA and [68Ga]Ga-RM2 PET/MRI vs. Histopathological Images in Prostate Cancer: A New Workflow for Spatial Co-Registration

This study proposed a new workflow for co-registering prostate PET images from a dual-tracer PET/MRI study with histopathological images of resected prostate specimens. The method aims to establish an accurate correspondence between PET/MRI findings and histology, facilitating a deeper understanding of PET tracer distribution and enabling advanced analyses like radiomics. To achieve this, images derived by three patients who underwent both [68Ga]Ga-PSMA and [68Ga]Ga-RM2 PET/MRI before radical prostatectomy were selected. After surgery, in the resected fresh specimens, fiducial markers visible on both histology and MR images were inserted. An ex vivo MRI of the prostate served as an intermediate step for co-registration between histological specimens and in vivo MRI examinations. The co-registration workflow involved five steps, ensuring alignment between histopathological images and PET/MRI data. The target registration error (TRE) was calculated to assess the precision of the co-registration. Furthermore, the DICE score was computed between the dominant intraprostatic tumor lesions delineated by the pathologist and the nuclear medicine physician. The TRE for the co-registration of histopathology and in vivo images was 1.59 mm, while the DICE score related to the site of increased intraprostatic uptake on [68Ga]Ga-PSMA and [68Ga]Ga-RM2 PET images was 0.54 and 0.75, respectively. This work shows an accurate co-registration method for histopathological and in vivo PET/MRI prostate examinations that allows the quantitative assessment of dual-tracer PET/MRI diagnostic accuracy at a millimetric scale. This approach may unveil radiotracer uptake mechanisms and identify new PET/MRI biomarkers, thus establishing the basis for precision medicine and future analyses, such as radiomics.

Correlation of in-vivo imaging with histopathology: A review

Despite tremendous advancements in in vivo imaging modalities, there remains substantial uncertainty with respect to tumor delineation on in these images. Histopathology remains the gold standard for determining the extent of malignancy, with in vivo imaging to histopathologic correlation enabling spatial comparisons. In this review, the steps necessary for successful imaging to histopathologic correlation are described, including in vivo imaging, resection, fixation, specimen sectioning (sectioning technique, securing technique, orientation matching, slice matching), microtome sectioning and staining, correlation (including image registration) and performance evaluation. The techniques used for each of these steps are also discussed. Hundreds of publications from the past 20 years were surveyed, and 62 selected for detailed analysis. For these 62 publications, each stage of the correlative pathology process (and the sub-steps of specimen sectioning) are listed. A statistical analysis was conducted based on 19 studies that reported target registration error as their performance metric. While some methods promise greater accuracy, they may be expensive. Due to the complexity of the processes involved, correlative pathology studies generally include a small number of subjects, which hinders advanced developments in this field.

Technical Note: Histological validation of anatomical imaging for breast modeling using a novel cryo-microtome

PURPOSE

Precise correlation between three-dimensional (3D) imaging and histology can aid biomechanical modeling of the breast. We develop a framework to register ex vivo images to histology using a novel cryo-fluorescence tomography (CFT) device.

METHODS

A formalin-fixed cadaveric breast specimen, including chest wall, was subjected to high-resolution magnetic resonance (MR) imaging. The specimen was then frozen and embedded in an optimal cutting temperature (OCT) compound. The OCT block was placed in a CFT device with an overhead camera and 50 μm thick slices were successively shaved off the block. After each shaving, the block-face was photographed. At select locations including connective/adipose tissue, muscle, skin, and fibroglandular tissue, 20 μm sections were transferred onto cryogenic tape for manual hematoxylin and eosin staining, histological assessment, and image capture. A 3D white-light image was automatically reconstructed from the photographs by aligning fiducial markers embedded in the OCT block. The 3D MR image, 3D white-light image, and photomicrographs were rigidly registered. Target registration errors (TREs) were computed based on 10 pairs of points marked at fibroglandular intersections. The overall MR-histology registration was used to compare the MR intensities at tissue extraction sites with a one-way analysis of variance.

RESULTS

The MR image to CFT-captured white-light image registration achieved a mean TRE of 0.73 ± 0.25 mm (less than the 1 mm MR slice resolution). The block-face white-light image and block-face photomicrograph registration showed visually indistinguishable alignment of anatomical structures and tissue boundaries. The MR intensities at the four tissue sites identified from histology differed significantly (p < 0.01). Each tissue pair, except the skin-connective/adipose tissue pair, also had significantly different MR intensities (p < 0.01).

CONCLUSIONS

Fine sectioning in a highly controlled imaging/sectioning environment enables accurate registration between the MR image and histology. Statistically significant differences in MR signal intensities between histological tissues are indicators for the specificity of correlation between MRI and histology.

Non-Invasive Ultrasonic Description of Tumor Evolution

Simple Summary

During tumor evolution, heterogeneous structural and functional changes occur in the tumor microenvironment. These complex changes have pro- or anti-tumorigenesis effects and have an impact on therapy efficiency. Therefore, the tumor microenvironment needs to be non-invasively characterized over time. The aim of this preclinical work is to compare the sensitivity of modifications occurring during tumor evolution of volume, immunohistochemistry and non-invasive quantitative ultrasound parameters (Shear Wave Elastography and dynamic Contrast-Enhanced Ultrasound) and to study the link between them. The complementary evaluation over time of multiple morphological and functional parameters during tumor growth underlines the need to integrate histological, morphological, functional, and, ultimately, genomic information into models that can consider the temporal and spatial variability of features to better understand tumor evolution.

Abstract

Purpose: There is a clinical need to better non-invasively characterize the tumor microenvironment in order to reveal evidence of early tumor response to therapy and to better understand therapeutic response. The goals of this work are first to compare the sensitivity to modifications occurring during tumor growth for measurements of tumor volume, immunohistochemistry parameters, and emerging ultrasound parameters (Shear Wave Elastography (SWE) and dynamic Contrast-Enhanced Ultrasound (CEUS)), and secondly, to study the link between the different parameters. Methods: Five different groups of 9 to 10 BALB/c female mice with subcutaneous CT26 tumors were imaged using B-mode morphological imaging, SWE, and CEUS at different dates. Whole-slice immunohistological data stained for the nuclei, T lymphocytes, apoptosis, and vascular endothelium from these tumors were analyzed. Results: Tumor volume and three CEUS parameters (Time to Peak, Wash-In Rate, and Wash-Out Rate) significantly changed over time. The immunohistological parameters, CEUS parameters, and SWE parameters showed intracorrelation. Four immunohistological parameters (the number of T lymphocytes per mm² and its standard deviation, the percentage area of apoptosis, and the colocalization of apoptosis and vascular endothelium) were correlated with the CEUS parameters (Time to Peak, Wash-In Rate, Wash-Out Rate, and Mean Transit Time). The SWE parameters were not correlated with the CEUS parameters nor with the immunohistological parameters. Conclusions: US imaging can provide additional information on tumoral changes. This could help to better explore the effect of therapies on tumor evolution, by studying the evolution of the parameters over time and by studying their correlations.

Can patient characteristics explain variance in ultrasound strain elastography measures of the quadratus femoris and patellar tendons?

OBJECTIVES

To explore the associations between participant characteristics and magnitudes of difference in paired elastography measures of knee tendon from different ultrasound systems, and to compare strain elastography pattern description.

MATERIALS AND METHODS

Quadriceps and patellar tendons of 20 healthy volunteers (40 tendons) were examined by an experienced operator employing two ultrasound systems (GE S8 and Esaote MyLab 70XVG). Pearson/Spearman correlations explored the influence of participant characteristics (BMI, body fat %, leg circumference, activity level) on the magnitude of differences between measures. Paired-sample t test or Wilcoxon signed rank test were performed to compare repeated measures of individual ultrasound systems.

RESULTS

The quadriceps tendon was characteristically stiffer than the patellar tendon. Participant characteristics were associated with within machine differences of the distal quadriceps tendon (BMI; r = 0.49, p = 0.028-0.03 and body fat %; r = 0.43, p = 0.05-0.056) ER measures.

CONCLUSIONS

Anthropometric and body composition parameters were associated with within machine differences for elasticity measures, where high BMI and body fat % contribute to paired measurement variance at the distal quadriceps tendon. Strain elastography protocols should be standardised, repeated ER measures performed using the same US system and patient characteristics considered for future clinical applications.

Synthetic Elastography Using B-Mode Ultrasound Through a Deep Fully Convolutional Neural Network

Shear-wave elastography (SWE) permits local estimation of tissue elasticity, an important imaging marker in biomedicine. This recently developed, advanced technique assesses the speed of a laterally traveling shear wave after an acoustic radiation force "push" to estimate local Young's moduli in an operator-independent fashion. In this work, we show how synthetic SWE (sSWE) images can be generated based on conventional B-mode imaging through deep learning. Using side-by-side-view B-mode/SWE images collected in 50 patients with prostate cancer, we show that sSWE images with a pixel-wise mean absolute error of 4.5 ± 0.96 kPa with regard to the original SWE can be generated. Visualization of high-level feature levels through t -distributed stochastic neighbor embedding reveals substantial overlap between data from two different scanners. Qualitatively, we examined the use of the sSWE methodology for B-mode images obtained with a scanner without SWE functionality. We also examined the use of this type of network in elasticity imaging in the thyroid. Limitations of the technique reside in the fact that networks have to be retrained for different organs, and that the method requires standardization of the imaging settings and procedure. Future research will be aimed at the development of sSWE as an elasticity-related tissue typing strategy that is solely based on B-mode ultrasound acquisition, and the examination of its clinical utility.

Artificial intelligence in multiparametric prostate cancer imaging with focus on deep-learning methods

Prostate cancer represents today the most typical example of a pathology whose diagnosis requires multiparametric imaging, a strategy where multiple imaging techniques are combined to reach an acceptable diagnostic performance. However, the reviewing, weighing and coupling of multiple images not only places additional burden on the radiologist, it also complicates the reviewing process. Prostate cancer imaging has therefore been an important target for the development of computer-aided diagnostic (CAD) tools. In this survey, we discuss the advances in CAD for prostate cancer over the last decades with special attention to the deep-learning techniques that have been designed in the last few years. Moreover, we elaborate and compare the methods employed to deliver the CAD output to the operator for further medical decision making.

Contrast-Enhanced Ultrasound Quantification: From Kinetic Modeling to Machine Learning

Ultrasound contrast agents (UCAs) have opened up immense diagnostic possibilities by combined use of indicator dilution principles and dynamic contrast-enhanced ultrasound (DCE-US) imaging. UCAs are microbubbles encapsulated in a biocompatible shell. With a rheology comparable to that of red blood cells, UCAs provide an intravascular indicator for functional imaging of the (micro)vasculature by quantitative DCE-US. Several models of the UCA intravascular kinetics have been proposed to provide functional quantitative maps, aiding diagnosis of different pathological conditions. This article is a comprehensive review of the available methods for quantitative DCE-US imaging based on temporal, spatial and spatiotemporal analysis of the UCA kinetics. The recent introduction of novel UCAs that are targeted to specific vascular receptors has advanced DCE-US to a molecular imaging modality. In parallel, new kinetic models of increased complexity have been developed. The extraction of multiple quantitative maps, reflecting complementary variables of the underlying physiological processes, requires an integrative approach to their interpretation. A probabilistic framework based on emerging machine-learning methods represents nowadays the ultimate approach, improving the diagnostic accuracy of DCE-US imaging by optimal combination of the extracted complementary information. The current value and future perspective of all these advances are critically discussed.

An Improved Strategy for Detection and Localization of Nodules in Liver Tissues by a 16 MHz Needle Ultrasonic Probe Mounted on a Robotic Platform

This study presents an improved strategy for the detection and localization of small size nodules (down to few mm) of agar in excised pork liver tissues via pulse-echo ultrasound measurements performed with a 16 MHz needle probe. This work contributes to the development of a new generation of medical instruments to support robotic surgery decision processes that need information about cancerous tissues in a short time (minutes). The developed ultrasonic probe is part of a scanning platform designed for the automation of surgery-associated histological analyses. It was coupled with a force sensor to control the indentation of tissue samples placed on a steel plate. For the detection of nodules, we took advantage of the property of nodules of altering not only the acoustical properties of tissues producing ultrasound attenuation, but also of developing patterns at their boundary that can modify the shape and the amplitude of the received echo signals from the steel plate supporting the tissues. Besides the Correlation Index Amplitude (CIA), which is linked to the overall amplitude changes of the ultrasonic signals, we introduced the Correlation Index Shape (CIS) linked to their shape changes. Furthermore, we applied AND-OR logical operators to these correlation indices. The results were found particularly helpful in the localization of the irregular masses of agar we inserted into some excised liver tissues, and in the individuation of the regions of major interest over which perform the vertical dissections of tissues in an automated analysis finalized to histopathology. We correctly identified up to 89% of inclusions, with an improvement of about 14% with respect to the result obtained (78%) from the analysis performed with the CIA parameter only.

Contrast-enhanced ultrasound with dispersion analysis for the localization of prostate cancer: correlation with radical prostatectomy specimens

PURPOSE

To determine the value of two-dimensional (2D) contrast-enhanced ultrasound (CEUS) imaging and the additional value of contrast ultrasound dispersion imaging (CUDI) for the localization of clinically significant prostate cancer (csPCa).

METHODS

In this multicentre study, subjects scheduled for a radical prostatectomy underwent 2D CEUS imaging preoperatively. CUDI maps were generated from the CEUS recordings. Both CEUS recordings and CUDI maps were scored on the likelihood of presenting csPCa (any Gleason ≥ 4 + 3 and Gleason 3 + 4 larger than 0.5 mL) by five observers and compared to radical prostatectomy histopathology. An automated three-dimensional (3D) fusion protocol was used to match imaging with histopathology. Receiver operator curve (ROC) analysis was performed per observer and imaging modality.

RESULTS

133 of 216 (62%) patients were included in the final analysis. Average area under the ROC for all five readers for CEUS, CUDI and the combination was 0.78, 0.79 and 0.78, respectively. This yields a sensitivity and specificity of 81 and 64% for CEUS, 83 and 56% for CUDI and 83 and 55% for the combination. Interobserver agreement for CEUS, CUDI and the combination showed kappa values of 0.20, 0.18 and 0.18 respectively.

CONCLUSION

The sensitivity and specificity of 2D CEUS and CUDI for csPCa localization are moderate. Despite compressing CEUS in one image, CUDI showed a similar performance to 2D CEUS. With a sensitivity of 83% at cutoff point 3, it could become a useful imaging procedure, especially with 4D acquisition, improved quantification and combination with other US imaging techniques such as elastography.

Automated multiparametric localization of prostate cancer based on B-mode, shear-wave elastography, and contrast-enhanced ultrasound radiomics

OBJECTIVES

The aim of this study was to assess the potential of machine learning based on B-mode, shear-wave elastography (SWE), and dynamic contrast-enhanced ultrasound (DCE-US) radiomics for the localization of prostate cancer (PCa) lesions using transrectal ultrasound.

METHODS

This study was approved by the institutional review board and comprised 50 men with biopsy-confirmed PCa that were referred for radical prostatectomy. Prior to surgery, patients received transrectal ultrasound (TRUS), SWE, and DCE-US for three imaging planes. The images were automatically segmented and registered. First, model-based features related to contrast perfusion and dispersion were extracted from the DCE-US videos. Subsequently, radiomics were retrieved from all modalities. Machine learning was applied through a random forest classification algorithm, using the co-registered histopathology from the radical prostatectomy specimens as a reference to draw benign and malignant regions of interest. To avoid overfitting, the performance of the multiparametric classifier was assessed through leave-one-patient-out cross-validation.

RESULTS

The multiparametric classifier reached a region-wise area under the receiver operating characteristics curve (ROC-AUC) of 0.75 and 0.90 for PCa and Gleason > 3 + 4 significant PCa, respectively, thereby outperforming the best-performing single parameter (i.e., contrast velocity) yielding ROC-AUCs of 0.69 and 0.76, respectively. Machine learning revealed that combinations between perfusion-, dispersion-, and elasticity-related features were favored.

CONCLUSIONS

In this paper, technical feasibility of multiparametric machine learning to improve upon single US modalities for the localization of PCa has been demonstrated. Extended datasets for training and testing may establish the clinical value of automatic multiparametric US classification in the early diagnosis of PCa.

KEY POINTS

• Combination of B-mode ultrasound, shear-wave elastography, and contrast ultrasound radiomics through machine learning is technically feasible. • Multiparametric ultrasound demonstrated a higher prostate cancer localization ability than single ultrasound modalities. • Computer-aided multiparametric ultrasound could help clinicians in biopsy targeting.

Differentiation of Vascular Characteristics Using Contrast-Enhanced Ultrasound Imaging

Ultrasound contrast imaging has been used to assess tumour growth and regression by assessing the flow through the macro- and micro-vasculature. Our aim was to differentiate the blood kinetics of vessels such as veins, arteries and microvasculature within the limits of the spatial resolution of contrast-enhanced ultrasound imaging. The highly vascularised ovine ovary was used as a biological model. Perfusion of the ovary with SonoVue was recorded with a Philips iU22 scanner in contrast imaging mode. One ewe was treated with prostaglandin to induce vascular regression. Time-intensity curves (TIC) for different regions of interest were obtained, a lognormal model was fitted and flow parameters calculated. Parametric maps of the whole imaging plane were generated for 2 × 2 pixel regions of interest. Further analysis of TICs from selected locations helped specify parameters associated with differentiation into four categories of vessels (arteries, veins, medium-sized vessels and micro-vessels). Time-dependent parameters were associated with large veins, whereas intensity-dependent parameters were associated with large arteries. Further development may enable automation of the technique as an efficient way of monitoring vessel distributions in a clinical setting using currently available scanners.

Multiparametric ultrasound: evaluation of greyscale, shear wave elastography and contrast-enhanced ultrasound for prostate cancer detection and localization in correlation to radical prostatectomy specimens

BACKGROUND

The diagnostic pathway for prostate cancer (PCa) is advancing towards an imaging-driven approach. Multiparametric magnetic resonance imaging, although increasingly used, has not shown sufficient accuracy to replace biopsy for now. The introduction of new ultrasound (US) modalities, such as quantitative contrast-enhanced US (CEUS) and shear wave elastography (SWE), shows promise but is not evidenced by sufficient high quality studies, especially for the combination of different US modalities. The primary objective of this study is to determine the individual and complementary diagnostic performance of greyscale US (GS), SWE, CEUS and their combination, multiparametric ultrasound (mpUS), for the detection and localization of PCa by comparison with corresponding histopathology.

METHODS/DESIGN

In this prospective clinical trial, US imaging consisting of GS, SWE and CEUS with quantitative mapping on 3 prostate imaging planes (base, mid and apex) will be performed in 50 patients with biopsy-proven PCa before planned radical prostatectomy using a clinical ultrasound scanner. All US imaging will be evaluated by US readers, scoring the four quadrants of each imaging plane for the likelihood of significant PCa based on a 1 to 5 Likert Scale. Following resection, PCa tumour foci will be identified, graded and attributed to the imaging-derived quadrants in each prostate plane for all prostatectomy specimens. Primary outcome measure will be the sensitivity, specificity, negative predictive value and positive predictive value of each US modality and mpUS to detect and localize significant PCa evaluated for different Likert Scale thresholds using receiver operating characteristics curve analyses.

DISCUSSION

In the evaluation of new PCa imaging modalities, a structured comparison with gold standard radical prostatectomy specimens is essential as first step. This trial is the first to combine the most promising ultrasound modalities into mpUS. It complies with the IDEAL stage 2b recommendations and will be an important step towards the evaluation of mpUS as a possible option for accurate detection and localization of PCa.

TRIAL REGISTRATION

The study protocol for multiparametric ultrasound was prospectively registered on Clinicaltrials.gov on 14 March 2017 with the registry name 'Multiparametric Ultrasound-Study for the Detection of Prostate Cancer' and trial registration number NCT03091231.