Rigid and non-rigid registration of polarized light imaging data for 3D reconstruction of the temporal lobe of the human brain at micrometer resolution

3-Dimensional Reconstruction From Histopathological Sections: A Systematic Review

Although pathological tissue analysis is typically performed on single 2-dimensional (2D) histologic reference slides, 3-dimensional (3D) reconstruction from a sequence of histologic sections could provide novel opportunities for spatial analysis of the extracted tissue. In this review, we analyze recent works published after 2018 and report information on the extracted tissue types, the section thickness, and the number of sections used for reconstruction. By analyzing the technological requirements for 3D reconstruction, we observe that software tools exist, both free and commercial, which include the functionality to perform 3D reconstruction from a sequence of histologic images. Through the analysis of the most recent works, we provide an overview of the workflows and tools that are currently used for 3D reconstruction from histologic sections and address points for future work, such as a missing common file format or computer-aided analysis of the reconstructed model.

A 3-dimensional histology computer model of malignant melanoma and its implications for digital pathology

BACKGROUND

Historically, cancer diagnoses have been made by pathologists using two-dimensional histological slides. However, with the advent of digital pathology and artificial intelligence, slides are being digitised, providing new opportunities to integrate their information. Since nature is 3-dimensional (3D), it seems intuitive to digitally reassemble the 3D structure for diagnosis.

OBJECTIVE

To develop the first human-3D-melanoma-histology-model with full data and code availability. Further, to evaluate the 3D-simulation together with experienced pathologists in the field and discuss the implications of digital 3D-models for the future of digital pathology.

METHODS

A malignant melanoma of the skin was digitised via 3 µm cuts by a slide scanner; an open-source software was then leveraged to construct the 3D model. A total of nine pathologists from four different countries with at least 10 years of experience in the histologic diagnosis of melanoma tested the model and discussed their experiences as well as implications for future pathology.

RESULTS

We successfully constructed and tested the first 3D-model of human melanoma. Based on testing, 88.9% of pathologists believe that the technology is likely to enter routine pathology within the next 10 years; advantages include a better reflectance of anatomy, 3D assessment of symmetry and the opportunity to simultaneously evaluate different tissue levels at the same time; limitations include the high consumption of tissue and a yet inferior resolution due to computational limitations.

CONCLUSIONS

3D-histology-models are promising for digital pathology of cancer and melanoma specifically, however, there are yet limitations which need to be carefully addressed.

Quantitative imaging of three-dimensional fiber orientation in the human brain via two illumination angles using polarization-sensitive optical coherence tomography

The accurate measurement of three-dimensional (3D) fiber orientation in the brain is crucial for reconstructing fiber pathways and studying their involvement in neurological diseases. Optical imaging methods such as polarization-sensitive optical coherence tomography (PS-OCT) provide important tools to directly quantify fiber orientation at micrometer resolution. However, brain imaging based on the optic axis by PS-OCT so far has been limited to two-dimensional in-plane orientation, preventing the comprehensive study of connectivity in 3D. In this work, we present a novel method to obtain the 3D fiber orientation in full angular space with only two illumination angles. We measure the optic axis orientation and the apparent birefringence by PS-OCT from a normal and a 15 deg tilted illumination, and then apply a computational method yielding the 3D optic axis orientation and true birefringence. We verify that our method accurately recovers a large range of through-plane orientations from -85 deg to 85 deg with a high angular precision. We further present 3D fiber orientation maps of entire coronal sections of human cerebrum and brainstem with 10 μm in-plane resolution, revealing unprecedented details of fiber configurations. We envision that further development of our method will open a promising avenue towards large-scale 3D fiber axis mapping in the human brain and other complex fibrous tissues at microscopic level.

Tensor image registration library: Deformable registration of stand-alone histology images to whole-brain post-mortem MRI data

BACKGROUND

Accurate registration between microscopy and MRI data is necessary for validating imaging biomarkers against neuropathology, and to disentangle complex signal dependencies in microstructural MRI. Existing registration methods often rely on serial histological sampling or significant manual input, providing limited scope to work with a large number of stand-alone histology sections. Here we present a customisable pipeline to assist the registration of stand-alone histology sections to whole-brain MRI data.

METHODS

Our pipeline registers stained histology sections to whole-brain post-mortem MRI in 4 stages, with the help of two photographic intermediaries: a block face image (to undistort histology sections) and coronal brain slab photographs (to insert them into MRI space). Each registration stage is implemented as a configurable stand-alone Python script using our novel platform, Tensor Image Registration Library (TIRL), which provides flexibility for wider adaptation. We report our experience of registering 87 PLP-stained histology sections from 14 subjects and perform various experiments to assess the accuracy and robustness of each stage of the pipeline.

RESULTS

All 87 histology sections were successfully registered to MRI. Histology-to-block registration (Stage 1) achieved 0.2-0.4 mm accuracy, better than commonly used existing methods. Block-to-slice matching (Stage 2) showed great robustness in automatically identifying and inserting small tissue blocks into whole brain slices with 0.2 mm accuracy. Simulations demonstrated sub-voxel level accuracy (0.13 mm) of the slice-to-volume registration (Stage 3) algorithm, which was observed in over 200 actual brain slice registrations, compensating 3D slice deformations up to 6.5 mm. Stage 4 combined the previous stages and generated refined pixelwise aligned multi-modal histology-MRI stacks.

CONCLUSIONS

Our open-source pipeline provides robust automation tools for registering stand-alone histology sections to MRI data with sub-voxel level precision, and the underlying framework makes it readily adaptable to a diverse range of microscopy-MRI studies.

Post mortem mapping of connectional anatomy for the validation of diffusion MRI

Diffusion MRI (dMRI) is a unique tool for the study of brain circuitry, as it allows us to image both the macroscopic trajectories and the microstructural properties of axon bundles in vivo. The Human Connectome Project ushered in an era of impressive advances in dMRI acquisition and analysis. As a result of these efforts, the quality of dMRI data that could be acquired in vivo improved substantially, and large collections of such data became widely available. Despite this progress, the main limitation of dMRI remains: it does not image axons directly, but only provides indirect measurements based on the diffusion of water molecules. Thus, it must be validated by methods that allow direct visualization of axons but that can only be performed in post mortem brain tissue. In this review, we discuss methods for validating the various features of connectional anatomy that are extracted from dMRI, both at the macro-scale (trajectories of axon bundles), and at micro-scale (axonal orientations and other microstructural properties). We present a range of validation tools, including anatomic tracer studies, Klingler's dissection, myelin stains, label-free optical imaging techniques, and others. We provide an overview of the basic principles of each technique, its limitations, and what it has taught us so far about the accuracy of different dMRI acquisition and analysis approaches.

Refractive-index matching enhanced polarization sensitive optical coherence tomography quantification in human brain tissue

The importance of polarization-sensitive optical coherence tomography (PS-OCT) has been increasingly recognized in human brain imaging. Despite the recent progress of PS-OCT in revealing white matter architecture and orientation, quantification of fine-scale fiber tracts in the human brain cortex has been a challenging problem, due to a low birefringence in the gray matter. In this study, we investigated the effect of refractive index matching by 2,2'-thiodiethanol (TDE) immersion on the improvement of PS-OCT measurements in ex vivo human brain tissue. We show that we can obtain fiber orientation maps of U-fibers that underlie sulci, as well as cortical fibers in the gray matter, including radial fibers in gyri and distinct layers of fibers exhibiting laminar organization. Further analysis shows that index matching reduces the noise in axis orientation measurements by 56% and 39%, in white and gray matter, respectively. Index matching also enables precise measurements of apparent birefringence, which was underestimated in the white matter by 82% but overestimated in the gray matter by 16% prior to TDE immersion. Mathematical simulations show that the improvements are primarily attributed to the reduction in the tissue scattering coefficient, leading to an enhanced signal-to-noise ratio in deeper tissue regions, which could not be achieved by conventional noise reduction methods.

High-fidelity approximation of grid- and shell-based sampling schemes from undersampled DSI using compressed sensing: Post mortem validation

While many useful microstructural indices, as well as orientation distribution functions, can be obtained from multi-shell dMRI data, there is growing interest in exploring the richer set of microstructural features that can be extracted from the full ensemble average propagator (EAP). The EAP can be readily computed from diffusion spectrum imaging (DSI) data, at the cost of a very lengthy acquisition. Compressed sensing (CS) has been used to make DSI more practical by reducing its acquisition time. CS applied to DSI (CS-DSI) attempts to reconstruct the EAP from significantly undersampled q-space data. We present a post mortem validation study where we evaluate the ability of CS-DSI to approximate not only fully sampled DSI but also multi-shell acquisitions with high fidelity. Human brain samples are imaged with high-resolution DSI at 9.4T and with polarization-sensitive optical coherence tomography (PSOCT). The latter provides direct measurements of axonal orientations at microscopic resolutions, allowing us to evaluate the mesoscopic orientation estimates obtained from diffusion MRI, in terms of their angular error and the presence of spurious peaks. We test two fast, dictionary-based, L2-regularized algorithms for CS-DSI reconstruction. We find that, for a CS acceleration factor of R=3, i.e., an acquisition with 171 gradient directions, one of these methods is able to achieve both low angular error and low number of spurious peaks. With a scan length similar to that of high angular resolution multi-shell acquisition schemes, this CS-DSI approach is able to approximate both fully sampled DSI and multi-shell data with high accuracy. Thus it is suitable for orientation reconstruction and microstructural modeling techniques that require either grid- or shell-based acquisitions. We find that the signal-to-noise ratio (SNR) of the training data used to construct the dictionary can have an impact on the accuracy of CS-DSI, but that there is substantial robustness to loss of SNR in the test data. Finally, we show that, as the CS acceleration factor increases beyond R=3, the accuracy of these reconstruction methods degrade, either in terms of the angular error, or in terms of the number of spurious peaks. Our results provide useful benchmarks for the future development of even more efficient q-space acceleration techniques.

Map3D: Registration-Based Multi-Object Tracking on 3D Serial Whole Slide Images

There has been a long pursuit for precise and reproducible glomerular quantification on renal pathology to leverage both research and practice. When digitizing the biopsy tissue samples using whole slide imaging (WSI), a set of serial sections from the same tissue can be acquired as a stack of images, similar to frames in a video. In radiology, the stack of images (e.g., computed tomography) are naturally used to provide 3D context for organs, tissues, and tumors. In pathology, it is appealing to do a similar 3D assessment. However, the 3D identification and association of large-scale glomeruli on renal pathology is challenging due to large tissue deformation, missing tissues, and artifacts from WSI. In this paper, we propose a novel Multi-object Association for Pathology in 3D (Map3D) method for automatically identifying and associating large-scale cross-sections of 3D objects from routine serial sectioning and WSI. The innovations of the Multi-Object Association for Pathology in 3D (Map3D) method are three-fold: (1) the large-scale glomerular association is formed as a new multi-object tracking (MOT) perspective; (2) the quality-aware whole series registration is proposed to not only provide affinity estimation but also offer automatic kidney-wise quality assurance (QA) for registration; (3) a dual-path association method is proposed to tackle the large deformation, missing tissues, and artifacts during tracking. To the best of our knowledge, the Map3D method is the first approach that enables automatic and large-scale glomerular association across 3D serial sectioning using WSI. Our proposed method Map3D achieved MOTA = 44.6, which is 12.1% higher than the non-deep learning benchmarks.

Adaptive Weighting Landmark-Based Group-Wise Registration on Lung DCE-MRI Images

Image registration of lung dynamic contrast enhanced magnetic resonance imaging (DCE-MRI) is challenging because the rapid changes in intensity lead to non-realistic deformations of intensity-based registration methods. To address this problem, we propose a novel landmark-based registration framework by incorporating landmark information into a group-wise registration. Robust principal component analysis is used to separate motion from intensity changes caused by a contrast agent. Landmark pairs are detected on the resulting motion components and then incorporated into an intensity-based registration through a constraint term. To reduce the negative effect of inaccurate landmark pairs on registration, an adaptive weighting landmark constraint is proposed. The method for calculating landmark weights is based on an assumption that the displacement of a good matched landmark is consistent with those of its neighbors. The proposed method was tested on 20 clinical lung DCE-MRI image series. Both visual inspection and quantitative assessment are used for the evaluation. Experimental results show that the proposed method effectively reduces the non-realistic deformations in registration and improves the registration performance compared with several state-of-the-art registration methods.

Anatomy of nerve fiber bundles at micrometer-resolution in the vervet monkey visual system

Although the primate visual system has been extensively studied, detailed spatial organization of white matter fiber tracts carrying visual information between areas has not been fully established. This is mainly due to the large gap between tracer studies and diffusion-weighted MRI studies, which focus on specific axonal connections and macroscale organization of fiber tracts, respectively. Here we used 3D polarization light imaging (3D-PLI), which enables direct visualization of fiber tracts at micrometer resolution, to identify and visualize fiber tracts of the visual system, such as stratum sagittale, inferior longitudinal fascicle, vertical occipital fascicle, tapetum and dorsal occipital bundle in vervet monkey brains. Moreover, 3D-PLI data provide detailed information on cortical projections of these tracts, distinction between neighboring tracts, and novel short-range pathways. This work provides essential information for interpretation of functional and diffusion-weighted MRI data, as well as revision of wiring diagrams based upon observations in the vervet visual system.

Insight into the fundamental trade-offs of diffusion MRI from polarization-sensitive optical coherence tomography in ex vivo human brain

In the first study comparing high angular resolution diffusion MRI (dMRI) in the human brain to axonal orientation measurements from polarization-sensitive optical coherence tomography (PSOCT), we compare the accuracy of orientation estimates from various dMRI sampling schemes and reconstruction methods. We find that, if the reconstruction approach is chosen carefully, single-shell dMRI data can yield the same accuracy as multi-shell data, and only moderately lower accuracy than a full Cartesian-grid sampling scheme. Our results suggest that current dMRI reconstruction approaches do not benefit substantially from ultra-high b-values or from very large numbers of diffusion-encoding directions. We also show that accuracy remains stable across dMRI voxel sizes of 1 mm or smaller but degrades at 2 mm, particularly in areas of complex white-matter architecture. We also show that, as the spatial resolution is reduced, axonal configurations in a dMRI voxel can no longer be modeled as a small set of distinct axon populations, violating an assumption that is sometimes made by dMRI reconstruction techniques. Our findings have implications for in vivo studies and illustrate the value of PSOCT as a source of ground-truth measurements of white-matter organization that does not suffer from the distortions typical of histological techniques.