A Method for 3D Histopathology Reconstruction Supporting Mouse Microvasculature Analysis

Feasibility and safety of synchrotron-based X-ray phase contrast imaging as a technique complementary to histopathology analysis

X-ray phase contrast imaging (X-PCI) is a powerful technique for high-resolution, three-dimensional imaging of soft tissue samples in a non-destructive manner. In this technical report, we assess the quality of standard histopathological techniques performed on formalin-fixed, paraffin-embedded (FFPE) human tissue samples that have been irradiated with different doses of X-rays in the context of an X-PCI experiment. The data from this study demonstrate that routine histochemical and immunohistochemical staining quality as well as DNA and RNA analyses are not affected by previous X-PCI on human FFPE samples. From these data we conclude it is feasible and acceptable to perform X-PCI on FFPE human biopsies.

Deep feature based cross-slide registration

Registration of multiple sections in a tissue block is an important pre-requisite task before any cross-slide image analysis. Non-rigid registration methods are capable of finding correspondence by locally transforming a moving image. These methods often rely on an initial guess to roughly align an image pair linearly and globally. This is essential to prevent convergence to a non-optimal minimum. We explore a deep feature based registration (DFBR) method which utilises data-driven descriptors to estimate the global transformation. A multi-stage strategy is adopted for improving the quality of registration. A visualisation tool is developed to view registered pairs of WSIs at different magnifications. With the help of this tool, one can apply a transformation on the fly without the need to generate a transformed moving WSI in a pyramidal form. We compare the performance on our dataset of data-driven descriptors with that of hand-crafted descriptors. Our approach can align the images with only small registration errors. The efficacy of our proposed method is evaluated for a subsequent non-rigid registration step. To this end, the first two steps of the ANHIR winner's framework are replaced with DFBR to register image pairs provided by the challenge. The modified framework produce comparable results to those of the challenge winning team.

Three-dimensional imaging and analysis of pathological tissue samples with de novo generation of citrate-based fluorophores

Two-dimensional (2D) histopathology based on the observation of thin tissue slides is the current paradigm in diagnosis and prognosis. However, labeling strategies in conventional histopathology are limited in compatibility with 3D imaging combined with tissue clearing techniques. Here, we present a rapid and efficient volumetric imaging technique of pathological tissues called 3D tissue imaging through de novo formation of fluorophores, or 3DNFC, which is the integration of citrate-based fluorogenic reaction DNFC and tissue clearing techniques. 3DNFC markedly increases the fluorescence intensity of tissues by generating fluorophores on nonfluorescent amino-terminal cysteine and visualizes the 3D structure of the tissues to provide their anatomical morphology and volumetric information. Furthermore, the application of 3DNFC to pathological tissue achieves the 3D reconstruction for the unbiased analysis of diverse features of the disorders in their natural context. We suggest that 3DNFC is a promising volumetric imaging method for the prognosis and diagnosis of pathological tissues.

An integrative web-based software tool for multi-dimensional pathology whole-slide image analytics

Objective.In the era of precision medicine, human tumor atlas-oriented studies have been significantly facilitated by high-resolution, multi-modal tissue based microscopic pathology image analytics. To better support such tissue-based investigations, we have developed Digital Pathology Laboratory (DPLab), a publicly available web-based platform, to assist biomedical research groups, non-technical end users, and clinicians for pathology whole-slide image visualization, annotation, analysis, and sharing via web browsers.Approach.A major advancement of this work is the easy-to-follow methods to reconstruct three-dimension (3D) tissue image volumes by registering two-dimension (2D) whole-slide pathology images of serial tissue sections stained by hematoxylin and eosin (H&E), and immunohistochemistry (IHC). The integration of these serial slides stained by different methods provides cellular phenotype and pathophysiologic states in the context of a 3D tissue micro-environment. DPLab is hosted on a publicly accessible server and connected to a backend computational cluster for intensive image analysis computations, with results visualized, downloaded, and shared via a web interface.Main results.Equipped with an analysis toolbox of numerous image processing algorithms, DPLab supports continued integration of community-contributed algorithms and presents an effective solution to improve the accessibility and dissemination of image analysis algorithms by research communities.Significance.DPLab represents the first step in making next generation tissue investigation tools widely available to the research community, enabling and facilitating discovery of clinically relevant disease mechanisms in a digital 3D tissue space.

Virtual liver needle biopsy from reconstructed three-dimensional histopathological images: Quantification of sampling error

INTRODUCTION

Prevalently considered as the "gold-standard" for diagnosis of hepatic fibrosis and cirrhosis, the clinical liver needle biopsy is known to be subject to inadequate sampling and a high mis-sampling rate. However, quantifying such sampling bias has been difficult as generating a large number of needle biopsies from the same living patient is practically infeasible. We construct a three-dimension (3D) virtual liver tissue volume by spatially registered high resolution Whole Slide Images (WSIs) of serial liver tissue sections with a novel dynamic registration method. We further develop a Virtual Needle Biopsy Sampling (VNBS) method that mimics the needle biopsy sampling process. We apply the VNBS method to the reconstructed digital liver volume at different tissue locations and angles. Additionally, we quantify Collagen Proportionate Area (CPA) in all resulting virtual needle biopsies in 2D and 3D.

RESULTS

The staging score of the center 2D longitudinal image plane from each 3D biopsy is used as the biopsy staging score, and the highest staging score of all sampled needle biopsies is the diagnostic staging score. The Mean Absolute Difference (MAD) in reference to the Scheuer and Ishak diagnostic staging scores are 0.22 and 1.00, respectively. The absolute Scheuer staging score difference in 22.22% of sampled biopsies is 1. By the Ishak staging method, 55.56% and 22.22% of sampled biopsies present score difference 1 and 2, respectively. There are 4 (Scheuer) and 6 (Ishak) out of 18 3D virtual needle biopsies with intra-needle variations. Additionally, we find a positive correlation between CPA and fibrosis stages by Scheuer but not Ishak method. Overall, CPA measures suffer large intra- and inter- needle variations.

CONCLUSIONS

The developed virtual liver needle biopsy sampling pipeline provides a computational avenue for investigating needle biopsy sampling bias with 3D virtual tissue volumes. This method can be applied to other tissue-based disease diagnoses where the needle biopsy sampling bias substantially affects the diagnostic results.

Regenerated Microvascular Networks in Ischemic Skeletal Muscle

Skeletal muscle is the largest organ in humans. The viability and performance of this metabolically demanding organ are exquisitely dependent on the integrity of its microcirculation. The architectural and functional attributes of the skeletal muscle microvasculature are acquired during embryonic and early postnatal development. However, peripheral vascular disease in the adult can damage the distal microvasculature, together with damaging the skeletal myofibers. Importantly, adult skeletal muscle has the capacity to regenerate. Understanding the extent to which the microvascular network also reforms, and acquires structural and functional competence, will thus be critical to regenerative medicine efforts for those with peripheral artery disease (PAD). Herein, we discuss recent advances in studying the regenerating microvasculature in the mouse hindlimb following severe ischemic injury. We highlight new insights arising from real-time imaging of the microcirculation. This includes identifying otherwise hidden flaws in both network microarchitecture and function, deficiencies that could underlie the progressive nature of PAD and its refractoriness to therapy. Recognizing and overcoming these vulnerabilities in regenerative angiogenesis will be important for advancing treatment options for PAD.

Three-dimensional reconstruction of systematic histological sections: application to observations on palatal shelf elevation

Normal mammalian secondary palate development undergoes a series of processes, including palatal shelf (PS) growth, elevation, adhesion and fusion, and palatal bone formation. It has been estimated that more than 90% of isolated cleft palate is caused by defects associated with the elevation process. However, because of the rapidly completed elevation process, the entire process of elevation will never be easy to clarify. In this article, we present a novel method for three-dimensional (3D) reconstruction of thick tissue blocks from two-dimensional (2D) histological sections. We established multiplanar sections of the palate and tongue in coronal and sagittal directions, and further performed 3D reconstruction to observe the morphological interaction and connection between the two components prior to and during elevation. The method completes an imaging system for simultaneous morphological analysis of thick tissue samples using both synthetic and real data. The new method will provide a comprehensive picture of reorientation morphology and gene expression pattern during the palatal elevation process.

Three-Dimensional Vessel Segmentation in Whole-Tissue and Whole-Block Imaging Using a Deep Neural Network: Proof-of-Concept Study

In the field of pathology, micro-computed tomography (micro-CT) has become an attractive imaging modality because it enables full analysis of the three-dimensional characteristics of a tissue sample or organ in a noninvasive manner. However, because of the complexity of the three-dimensional information, understanding would be improved by development of analytical methods and software such as those implemented for clinical CT. As the accurate identification of tissue components is critical for this purpose, we have developed a deep neural network (DNN) to analyze whole-tissue images (WTIs) and whole-block images (WBIs) of neoplastic cancer tissue using micro-CT. The aim of this study was to segment vessels from WTIs and WBIs in a volumetric segmentation method using DNN. To accelerate the segmentation process while retaining accuracy, a convolutional block in DNN was improved by introducing a residual inception block. Three colorectal tissue samples were collected and one WTI and 70 WBIs were acquired by a micro-CT scanner. The implemented segmentation method was then tested on the WTI and WBIs. As a proof-of-concept study, our method successfully segmented the vessels on all WTI and WBIs of the colorectal tissue sample. In addition, despite the large size of the images for analysis, all segmentation processes were completed in 10 minutes.

Recent advances in tissue imaging for cancer research

Image analysis in clinical research has evolved at fast pace in the last decade. This review discusses basic concepts ranging from immunohistochemistry to advanced techniques such as multiplex imaging, digital pathology, flow cytometry and intravital microscopy. Tissue imaging ex vivo is still one of the gold-standards in the field due to feasibility. We describe here different protocols and applications of digital analysis providing basic and clinical researchers with an overview on how to analyse tissue images. In vivo imaging is not easily accessible to researchers; however, it provides invaluable dynamic information. Overall, we discuss a plethora of techniques that - when combined - constitute a powerful platform for basic and translational cancer research.

Society of Toxicologic Pathology Digital Pathology and Image Analysis Special Interest Group Article*: Opinion on the Application of Artificial Intelligence and Machine Learning to Digital Toxicologic Pathology

Toxicologic pathology is transitioning from analog to digital methods. This transition seems inevitable due to a host of ongoing social and medical technological forces. Of these, artificial intelligence (AI) and in particular machine learning (ML) are globally disruptive, rapidly growing sectors of technology whose impact on the long-established field of histopathology is quickly being realized. The development of increasing numbers of algorithms, peering ever deeper into the histopathological space, has demonstrated to the scientific community that AI pathology platforms are now poised to truly impact the future of precision and personalized medicine. However, as with all great technological advances, there are implementation and adoption challenges. This review aims to define common and relevant AI and ML terminology, describe data generation and interpretation, outline current and potential future business cases, discuss validation and regulatory hurdles, and most importantly, propose how overcoming the challenges of this burgeoning technology may shape toxicologic pathology for years to come, enabling pathologists to contribute even more effectively to answering scientific questions and solving global health issues. [Box: see text]

Comparative analysis of tissue reconstruction algorithms for 3D histology

Motivation

Digital pathology enables new approaches that expand beyond storage, visualization or analysis of histological samples in digital format. One novel opportunity is 3D histology, where a three-dimensional reconstruction of the sample is formed computationally based on serial tissue sections. This allows examining tissue architecture in 3D, for example, for diagnostic purposes. Importantly, 3D histology enables joint mapping of cellular morphology with spatially resolved omics data in the true 3D context of the tissue at microscopic resolution. Several algorithms have been proposed for the reconstruction task, but a quantitative comparison of their accuracy is lacking.

Results

We developed a benchmarking framework to evaluate the accuracy of several free and commercial 3D reconstruction methods using two whole slide image datasets. The results provide a solid basis for further development and application of 3D histology algorithms and indicate that methods capable of compensating for local tissue deformation are superior to simpler approaches.

Availability and implementation

Code: https://github.com/BioimageInformaticsTampere/RegBenchmark. Whole slide image datasets: http://urn.fi/urn: nbn: fi: csc-kata20170705131652639702.

Supplementary information

Supplementary data are available at Bioinformatics online.

Interactive Visualization of 3D Histopathology in Native Resolution

We present a visualization application that enables effective interactive visual analysis of large-scale 3D histopathology, that is, high-resolution 3D microscopy data of human tissue. Clinical work flows and research based on pathology have, until now, largely been dominated by 2D imaging. As we will show in the paper, studying volumetric histology data will open up novel and useful opportunities for both research and clinical practice. Our starting point is the current lack of appropriate visualization tools in histopathology, which has been a limiting factor in the uptake of digital pathology. Visualization of 3D histology data does pose difficult challenges in several aspects. The full-color datasets are dense and large in scale, on the order of 100,000 × 100,000× 100 voxels. This entails serious demands on both rendering performance and user experience design. Despite this, our developed application supports interactive study of 3D histology datasets at native resolution. Our application is based on tailoring and tuning of existing methods, system integration work, as well as a careful study of domain specific demands emanating from a close participatory design process with domain experts as team members. Results from a user evaluation employing the tool demonstrate a strong agreement among the 14 participating pathologists that 3D histopathology will be a valuable and enabling tool for their work.

Label-free automated three-dimensional imaging of whole organs by microtomy-assisted photoacoustic microscopy

Three-dimensional (3D) optical imaging of whole biological organs with microscopic resolution has remained a challenge. Most versions of such imaging techniques require special preparation of the tissue specimen. Here we demonstrate microtomy-assisted photoacoustic microscopy (mPAM) of mouse brains and other organs, which automatically acquires serial distortion-free and registration-free images with endogenous absorption contrasts. Without tissue staining or clearing, mPAM generates micrometer-resolution 3D images of paraffin- or agarose-embedded whole organs with high fidelity, achieved by label-free simultaneous sensing of DNA/RNA, hemoglobins, and lipids. mPAM provides histology-like imaging of cell nuclei, blood vessels, axons, and other anatomical structures, enabling the application of histopathological interpretation at the organelle level to analyze a whole organ. Its deep tissue imaging capability leads to less sectioning, resulting in negligible sectioning artifact. mPAM offers a new way to better understand complex biological organs.

DYNAMIC REGISTRATION FOR GIGAPIXEL SERIAL WHOLE SLIDE IMAGES

High-throughput serial histology imaging provides a new avenue for the routine study of micro-anatomical structures in a 3D space. However, the emergence of serial whole slide imaging poses a new registration challenge, as the gigapixel image size precludes the direct application of conventional registration techniques. In this paper, we develop a three-stage registration with multi-resolution mapping and propagation method to dynamically produce registered subvolumes from serial whole slide images. We validate our algorithm with gigapixel images of serial brain tumor sections and synthetic image volumes. The qualitative and quantitative assessment results demonstrate the efficacy of our approach and suggest its promise for 3D histology reconstruction analysis.

Differentiation of arterioles from venules in mouse histology images using machine learning

Analysis and morphological comparison of the arteriolar and venular components of a microvascular network are essential to our understanding of multiple diseases affecting every organ system. We have developed and evaluated the first fully automatic software system for differentiation of arterioles from venules on high-resolution digital histology images of the mouse hind limb immunostained with smooth muscle [Formula: see text]-actin. Classifiers trained on statistical and morphological features by supervised machine learning provided useful classification accuracy for differentiation of arterioles from venules, achieving an area under the receiver operating characteristic curve of 0.89. Feature selection was consistent across cross validation iterations, and a small set of two features was required to achieve the reported performance, suggesting the generalizability of the system. This system eliminates the need for laborious manual classification of the hundreds of microvessels occurring in a typical sample and paves the way for high-throughput analysis of the arteriolar and venular networks in the mouse.

Transformation diffusion reconstruction of three-dimensional histology volumes from two-dimensional image stacks

•

A method for 3D reconstruction of serial 2D histology image stacks is proposed.

•

Pre-alignment to an external pre-cut reference (blockface) prevents shape artifacts.

•

Formulated as diffusion of transformations from each slice to its neighbors.

•

Registrations replaced by much faster transformation operations.

Traditional histology is the gold standard for tissue studies, but it is intrinsically reliant on two-dimensional (2D) images. Study of volumetric tissue samples such as whole hearts produces a stack of misaligned and distorted 2D images that need to be reconstructed to recover a congruent volume with the original sample's shape. In this paper, we develop a mathematical framework called Transformation Diffusion (TD) for stack alignment refinement as a solution to the heat diffusion equation. This general framework does not require contour segmentation, is independent of the registration method used, and is trivially parallelizable. After the first stack sweep, we also replace registration operations by operations in the space of transformations, several orders of magnitude faster and less memory-consuming. Implementing TD with operations in the space of transformations produces our Transformation Diffusion Reconstruction (TDR) algorithm, applicable to general transformations that are closed under inversion and composition. In particular, we provide formulas for translation and affine transformations. We also propose an Approximated TDR (ATDR) algorithm that extends the same principles to tensor-product B-spline transformations. Using TDR and ATDR, we reconstruct a full mouse heart at pixel size 0.92 µm × 0.92 µm, cut 10 µm thick, spaced 20 µm (84G). Our algorithms employ only local information from transformations between neighboring slices, but the TD framework allows theoretical analysis of the refinement as applying a global Gaussian low-pass filter to the unknown stack misalignments. We also show that reconstruction without an external reference produces large shape artifacts in a cardiac specimen while still optimizing slice-to-slice alignment. To overcome this problem, we use a pre-cutting blockface imaging process previously developed by our group that takes advantage of Brewster's angle and a polarizer to capture the outline of only the topmost layer of wax in the block containing embedded tissue for histological sectioning.

Segmentation of digitized histological sections for quantification of the muscularized vasculature in the mouse hind limb

Immunohistochemical tissue staining enhances microvasculature characteristics, including the smooth muscle in the medial layer of the vessel walls that is responsible for regulation of blood flow. The vasculature can be imaged in a comprehensive fashion using whole-slide scanning. However, since each such image potentially contains hundreds of small vessels, manual vessel delineation and quantification is not practically feasible. In this work, we present a fully automatic segmentation and vasculature quantification algorithm for whole-slide images. We evaluated its performance on tissue samples drawn from the hind limbs of wild-type mice, stained for smooth muscle using 3,3'-Diaminobenzidine (DAB) immunostain. The algorithm was designed to be robust to vessel fragmentation due to staining irregularity, and artefactual staining of nonvessel objects. Colour deconvolution was used to isolate the DAB stain for detection of vessel wall fragments. Complete vessels were reconstructed from the fragments by joining endpoints of topological skeletons. Automatic measures of vessel density, perimeter, wall area and local wall thickness were taken. The segmentation algorithm was validated against manual measures, resulting in a Dice similarity coefficient of 89%. The relationships observed between these measures were as expected from a biological standpoint, providing further reinforcement of the accuracy of this system. This system provides a fully automated and accurate means of measuring the arteriolar and venular morphology of vascular smooth muscle.

Feature-based multi-resolution registration of immunostained serial sections

The form and exact function of the blood vessel network in some human organs, like spleen and bone marrow, are still open research questions in medicine. In this paper, we propose a method to register the immunohistological stainings of serial sections of spleen and bone marrow specimens to enable the visualization and visual inspection of blood vessels. As these vary much in caliber, from mesoscopic (millimeter-range) to microscopic (few micrometers, comparable to a single erythrocyte), we need to utilize a multi-resolution approach. Our method is fully automatic; it is based on feature detection and sparse matching. We utilize a rigid alignment and then a non-rigid deformation, iteratively dealing with increasingly smaller features. Our tool pipeline can already deal with series of complete scans at extremely high resolution, up to 620 megapixels. The improvement presented increases the range of represented details up to smallest capillaries. This paper provides details on the multi-resolution non-rigid registration approach we use. Our application is novel in the way the alignment and subsequent deformations are computed (using features, i.e. "sparse"). The deformations are based on all images in the stack ("global"). We also present volume renderings and a 3D reconstruction of the vascular network in human spleen and bone marrow on a level not possible before. Our registration makes easy tracking of even smallest blood vessels possible, thus granting experts a better comprehension. A quantitative evaluation of our method and related state of the art approaches with seven different quality measures shows the efficiency of our method. We also provide z-profiles and enlarged volume renderings from three different registrations for visual inspection.

Patch-Based Nonlinear Image Registration for Gigapixel Whole Slide Images

OBJECTIVE

Image registration of whole slide histology images allows the fusion of fine-grained information-like different immunohistochemical stains-from neighboring tissue slides. Traditionally, pathologists fuse this information by looking subsequently at one slide at a time. If the slides are digitized and accurately aligned at cell level, automatic analysis can be used to ease the pathologist's work. However, the size of those images exceeds the memory capacity of regular computers.

METHODS

We address the challenge to combine a global motion model that takes the physical cutting process of the tissue into account with image data that is not simultaneously globally available. Typical approaches either reduce the amount of data to be processed or partition the data into smaller chunks to be processed separately. Our novel method first registers the complete images on a low resolution with a nonlinear deformation model and later refines this result on patches by using a second nonlinear registration on each patch. Finally, the deformations computed on all patches are combined by interpolation to form one globally smooth nonlinear deformation. The NGF distance measure is used to handle multistain images.

RESULTS

The method is applied to ten whole slide image pairs of human lung cancer data. The alignment of 85 corresponding structures is measured by comparing manual segmentations from neighboring slides. Their offset improves significantly, by at least 15%, compared to the low-resolution nonlinear registration.

CONCLUSION/SIGNIFICANCE

The proposed method significantly improves the accuracy of multistain registration which allows us to compare different antibodies at cell level.