A linear optimal transportation framework for quantifying and visualizing variations in sets of images

Discovering the gene-brain-behavior link in autism via generative machine learning

Autism is traditionally diagnosed behaviorally but has a strong genetic basis. A genetics-first approach could transform understanding and treatment of autism. However, isolating the gene-brain-behavior relationship from confounding sources of variability is a challenge. We demonstrate a novel technique, 3D transport-based morphometry (TBM), to extract the structural brain changes linked to genetic copy number variation (CNV) at the 16p11.2 region. We identified two distinct endophenotypes. In data from the Simons Variation in Individuals Project, detection of these endophenotypes enabled 89 to 95% test accuracy in predicting 16p11.2 CNV from brain images alone. Then, TBM enabled direct visualization of the endophenotypes driving accurate prediction, revealing dose-dependent brain changes among deletion and duplication carriers. These endophenotypes are sensitive to articulation disorders and explain a portion of the intelligence quotient variability. Genetic stratification combined with TBM could reveal new brain endophenotypes in many neurodevelopmental disorders, accelerating precision medicine, and understanding of human neurodiversity.

3D imaging of SARS-CoV-2 infected hamster lungs by X-ray phase contrast tomography enables drug testing

X-ray Phase Contrast Tomography (XPCT) based on wavefield propagation has been established as a high resolution three-dimensional (3D) imaging modality, suitable to reconstruct the intricate structure of soft tissues, and the corresponding pathological alterations. However, for biomedical research, more is needed than 3D visualisation and rendering of the cytoarchitecture in a few selected cases. First, the throughput needs to be increased to cover a statistically relevant number of samples. Second, the cytoarchitecture has to be quantified in terms of morphometric parameters, independent of visual impression. Third, dimensionality reduction and classification are required for identification of effects and interpretation of results. To address these challenges, we here design and implement a novel integrated and high throughput XPCT imaging and analysis workflow for 3D histology, pathohistology and drug testing. Our approach uses semi-automated data acquisition, reconstruction and statistical quantification. We demonstrate its capability for the example of lung pathohistology in Covid-19. Using a small animal model, different Covid-19 drug candidates are administered after infection and tested in view of restoration of the physiological cytoarchitecture, specifically the alveolar morphology. To this end, we then use morphometric parameter determination followed by a dimensionality reduction and classification based on optimal transport. This approach allows efficient discrimination between physiological and pathological lung structure, thereby providing quantitative insights into the pathological progression and partial recovery due to drug treatment. Finally, we stress that the XPCT image chain implemented here only used synchrotron radiation for validation, while the data used for analysis was recorded with laboratory μ CT radiation, more easily accessible for pre-clinical research.

Linear optimal transport subspaces for point set classification

Learning from point sets is an essential component in many computer vision and machine learning applications. Native, unordered, and permutation invariant set structure space is challenging to model, particularly for point set classification under spatial deformations. Here we propose a framework for classifying point sets experiencing certain types of spatial deformations, with a particular emphasis on datasets featuring affine deformations. Our approach employs the Linear Optimal Transport (LOT) transform to obtain a linear embedding of set-structured data. Utilizing the mathematical properties of the LOT transform, we demonstrate its capacity to accommodate variations in point sets by constructing a convex data space, effectively simplifying point set classification problems. Our method, which employs a nearest-subspace algorithm in the LOT space, demonstrates label efficiency, non-iterative behavior, and requires no hyper-parameter tuning. It achieves competitive accuracies compared to state-of-the-art methods across various point set classification tasks. Furthermore, our approach exhibits robustness in out-of-distribution scenarios where training and test distributions vary in terms of deformation magnitudes.

3d virtual histology reveals pathological alterations of cerebellar granule cells in multiple sclerosis

We investigate structural properties of neurons in the granular layer of human cerebellum with respect to their involvement in multiple sclerosis (MS). To this end we analyze data recorded by X-ray phase contrast tomography from tissue samples collected post mortem from a MS and a healthy control group. Using automated segmentation and histogram analysis based on optimal transport theory (OT) we find that the distributions representing nuclear structure in the granular layer move to a more compact nuclear state, i.e. smaller, denser and more heterogeneous nuclei in MS. We have previously made a similar observation for neurons of the dentate gyrus in Alzheimer's disease, suggesting that more compact structure of neuronal nuclei which we attributed to increased levels of heterochromatin, may possibly represent a more general phenomenon of cellular senescence associated with neurodegeneration.

A novel statistical methodology for quantifying the spatial arrangements of axons in peripheral nerves

A thorough understanding of the neuroanatomy of peripheral nerves is required for a better insight into their function and the development of neuromodulation tools and strategies. In biophysical modeling, it is commonly assumed that the complex spatial arrangement of myelinated and unmyelinated axons in peripheral nerves is random, however, in reality the axonal organization is inhomogeneous and anisotropic. Present quantitative neuroanatomy methods analyze peripheral nerves in terms of the number of axons and the morphometric characteristics of the axons, such as area and diameter. In this study, we employed spatial statistics and point process models to describe the spatial arrangement of axons and Sinkhorn distances to compute the similarities between these arrangements (in terms of first- and second-order statistics) in various vagus and pelvic nerve cross-sections. We utilized high-resolution transmission electron microscopy (TEM) images that have been segmented using a custom-built high-throughput deep learning system based on a highly modified U-Net architecture. Our findings show a novel and innovative approach to quantifying similarities between spatial point patterns using metrics derived from the solution to the optimal transport problem. We also present a generalizable pipeline for quantitative analysis of peripheral nerve architecture. Our data demonstrate differences between male- and female-originating samples and similarities between the pelvic and abdominal vagus nerves.

On a Linear Gromov-Wasserstein Distance

Gromov-Wasserstein distances are generalization of Wasserstein distances, which are invariant under distance preserving transformations. Although a simplified version of optimal transport in Wasserstein spaces, called linear optimal transport (LOT), was successfully used in practice, there does not exist a notion of linear Gromov-Wasserstein distances so far. In this paper, we propose a definition of linear Gromov-Wasserstein distances. We motivate our approach by a generalized LOT model, which is based on barycentric projection maps of transport plans. Numerical examples illustrate that the linear Gromov-Wasserstein distances, similarly as LOT, can replace the expensive computation of pairwise Gromov-Wasserstein distances in applications like shape classification.

Three-dimensional virtual histology of the human hippocampus based on phase-contrast computed tomography

We demonstrate multiscale phase-contrast X-ray computed tomography (CT) of postmortem human brain tissue. Large tissue volumes can be covered by parallel-beam CT and combined with subcellular detail for selected regions scanned at high magnification. This has been repeated identically for a larger number of individuals, including both Alzheimer’s-diseased patients and a control group. Optimized phase retrieval, followed by automated segmentation based on machine learning, as well as feature identification and classification based on optimal transport theory, indicates a pathway from healthy to pathological structure without prior hypothesis. This study provides a blueprint for studying the cytoarchitecture of the human brain and its alterations associated with neurodegenerative diseases.

We have studied the three-dimensional (3D) cytoarchitecture of the human hippocampus in neuropathologically healthy and Alzheimer’s disease (AD) individuals, based on phase-contrast X-ray computed tomography of postmortem human tissue punch biopsies. In view of recent findings suggesting a nuclear origin of AD, we target in particular the nuclear structure of the dentate gyrus (DG) granule cells. Tissue samples of 20 individuals were scanned and evaluated using a highly automated approach of measurement and analysis, combining multiscale recordings, optimized phase retrieval, segmentation by machine learning, representation of structural properties in a feature space, and classification based on the theory of optimal transport. Accordingly, we find that the prototypical transformation between a structure representing healthy granule cells and the pathological state involves a decrease in the volume of granule cell nuclei, as well as an increase in the electron density and its spatial heterogeneity. The latter can be explained by a higher ratio of heterochromatin to euchromatin. Similarly, many other structural properties can be derived from the data, reflecting both the natural polydispersity of the hippocampal cytoarchitecture between different individuals in the physiological context and the structural effects associated with AD pathology.

Investigating impact of cardiorespiratory fitness in reducing brain tissue loss caused by ageing

Mitigating the loss of brain tissue due to age is a major problem for an ageing population. Improving cardiorespiratory fitness has been suggested as a possible strategy, but the influenceon brain morphology has not been fully characterized. To investigate the dependent shifts in brain tissue distribution as a function of cardiorespiratory fitness, we used a 3D transport-based morphometry approach. In this study of 172 inactive older adults aged 58-81 (66.5 ± 5.7) years, cardiorespiratory fitness was determined by VO 2 peak (ml/kg/min) during graded exercise and brain morphology was assessed through structural magnetic resonance imaging. After correcting for covariates including age (in the fitness model), gender and level of education, we compared dependent tissue shifts with age to those due to V O 2   peak . We found a significant association between cardiorespiratory fitness and brain tissue distribution (white matter, r = 0.30, P = 0.003; grey matter, r = 0.40, P < 0.001) facilitated by direct visualization of the brain tissue shifts due to cardiorespiratory fitness through inverse transformation-a key capability of 3D transport-based morphometry. A strong statistical correlation was found between brain tissue changes related to ageing and those associated with lower cardiorespiratory fitness (white matter, r = 0.62, P < 0.001; grey matter, r = 0.74, P < 0.001). In both cases, frontotemporal regions shifted the most while basal ganglia shifted the least. Our results highlight the importance of cardiorespiratory fitness in maintaining brain health later in life. Furthermore, this work demonstrates 3D transport-based morphometry as a novel neuroinformatic technology that may aid assessment of therapeutic approaches for brain ageing and neurodegenerative diseases.

Radon Cumulative Distribution Transform Subspace Modeling for Image Classification

We present a new supervised image classification method applicable to a broad class of image deformation models. The method makes use of the previously described Radon Cumulative Distribution Transform (R-CDT) for image data, whose mathematical properties are exploited to express the image data in a form that is more suitable for machine learning. While certain operations such as translation, scaling, and higher-order transformations are challenging to model in native image space, we show the R-CDT can capture some of these variations and thus render the associated image classification problems easier to solve. The method - utilizing a nearest-subspace algorithm in the R-CDT space - is simple to implement, non-iterative, has no hyper-parameters to tune, is computationally efficient, label efficient, and provides competitive accuracies to state-of-the-art neural networks for many types of classification problems. In addition to the test accuracy performances, we show improvements (with respect to neural network-based methods) in terms of computational efficiency (it can be implemented without the use of GPUs), number of training samples needed for training, as well as out-of-distribution generalization. The Python code for reproducing our results is available at [1].

PARTITIONING SIGNAL CLASSES USING TRANSPORT TRANSFORMS FOR DATA ANALYSIS AND MACHINE LEARNING

A relatively new set of transport-based transforms (CDT, R-CDT, LOT) have shown their strength and great potential in various image and data processing tasks such as parametric signal estimation, classification, cancer detection among many others. It is hence worthwhile to elucidate some of the mathematical properties that explain the successes of these transforms when they are used as tools in data analysis, signal processing or data classification. In particular, we give conditions under which classes of signals that are created by algebraic generative models are transformed into convex sets by the transport transforms. Such convexification of the classes simplify the classification and other data analysis and processing problems when viewed in the transform domain. More specifically, we study the extent and limitation of the convexification ability of these transforms under an algebraic generative modeling framework. We hope that this paper will serve as an introduction to these transforms and will encourage mathematicians and other researchers to further explore the theoretical underpinnings and algorithmic tools that will help understand the successes of these transforms and lay the groundwork for further successful applications.

Radial Basis Function Artificial Neural Network for the Investigation of Thyroid Cytological Lesions

Objective

This study investigates the potential of an artificial intelligence (AI) methodology, the radial basis function (RBF) artificial neural network (ANN), in the evaluation of thyroid lesions. Study Design. The study was performed on 447 patients who had both cytological and histological evaluation in agreement. Cytological specimens were prepared using liquid-based cytology, and the histological result was based on subsequent surgical samples. Each specimen was digitized; on these images, nuclear morphology features were measured by the use of an image analysis system. The extracted measurements (41,324 nuclei) were separated into two sets: the training set that was used to create the RBF ANN and the test set that was used to evaluate the RBF performance. The system aimed to predict the histological status as benign or malignant.

Results

The RBF ANN obtained in the training set has sensitivity 82.5%, specificity 94.6%, and overall accuracy 90.3%, while in the test set, these indices were 81.4%, 90.0%, and 86.9%, respectively. Algorithm was used to classify patients on the basis of the RBF ANN, the overall sensitivity was 95.0%, the specificity was 95.5%, and no statistically significant difference was observed.

Conclusion

AI techniques and especially ANNs, only in the recent years, have been studied extensively. The proposed approach is promising to avoid misdiagnoses and assists the everyday practice of the cytopathology. The major drawback in this approach is the automation of a procedure to accurately detect and measure cell nuclei from the digitized images.

Enabling early detection of osteoarthritis from presymptomatic cartilage texture maps via transport-based learning

Many diseases have no visual cues in the early stages, eluding image-based detection. Today, osteoarthritis (OA) is detected after bone damage has occurred, at an irreversible stage of the disease. Currently no reliable method exists for OA detection at a reversible stage. We present an approach that enables sensitive OA detection in presymptomatic individuals. Our approach combines optimal mass transport theory with statistical pattern recognition. Eighty-six healthy individuals were selected from the Osteoarthritis Initiative, with no symptoms or visual signs of disease on imaging. On 3-y follow-up, a subset of these individuals had progressed to symptomatic OA. We trained a classifier to differentiate progressors and nonprogressors on baseline cartilage texture maps, which achieved a robust test accuracy of 78% in detecting future symptomatic OA progression 3 y prior to symptoms. This work demonstrates that OA detection may be possible at a potentially reversible stage. A key contribution of our work is direct visualization of the cartilage phenotype defining predictive ability as our technique is generative. We observe early biochemical patterns of fissuring in cartilage that define future onset of OA. In the future, coupling presymptomatic OA detection with emergent clinical therapies could modify the outcome of a disease that costs the United States healthcare system $16.5 billion annually. Furthermore, our technique is broadly applicable to earlier image-based detection of many diseases currently diagnosed at advanced stages today.

Hyperbolic Wasserstein Distance for Shape Indexing

Shape space is an active research topic in computer vision and medical imaging fields. The distance defined in a shape space may provide a simple and refined index to represent a unique shape. This work studies the Wasserstein space and proposes a novel framework to compute the Wasserstein distance between general topological surfaces by integrating hyperbolic Ricci flow, hyperbolic harmonic map, and hyperbolic power Voronoi diagram algorithms. The resulting hyperbolic Wasserstein distance can intrinsically measure the similarity between general topological surfaces. Our proposed algorithms are theoretically rigorous and practically efficient. It has the potential to be a powerful tool for 3D shape indexing research. We tested our algorithm with human face classification and Alzheimer's disease (AD) progression tracking studies. Experimental results demonstrated that our work may provide a succinct and effective shape index.

Cell Image Classification: A Comparative Overview

Cell image classification methods are currently being used in numerous applications in cell biology and medicine. Applications include understanding the effects of genes and drugs in screening experiments, understanding the role and subcellular localization of different proteins, as well as diagnosis and prognosis of cancer from images acquired using cytological and histological techniques. The article also reviews three main approaches for cell image classification most often used: numerical feature extraction, end-to-end classification with neural networks (NNs), and transport-based morphometry (TBM). In addition, we provide comparisons on four different cell imaging datasets to highlight the relative strength of each method. The results computed using four publicly available datasets show that numerical features tend to carry the best discriminative information for most of the classification tasks. Results also show that NN-based methods produce state-of-the-art results in the dataset that contains a relatively large number of training samples. Data augmentation or the choice of a more recently reported architecture does not necessarily improve the classification performance of NNs in the datasets with limited number of training samples. If understanding and visualization are desired aspects, TBM methods can offer the ability to invert classification functions, and thus can aid in the interpretation of results. These and other comparison outcomes are discussed with the aim of clarifying the advantages and disadvantages of each method. © 2020 International Society for Advancement of Cytometry.

Assessing Postconcussive Reaction Time Using Transport-Based Morphometry of Diffusion Tensor Images

BACKGROUND AND PURPOSE

Cognitive deficits are among the most commonly reported post-concussive symptoms, yet the underlying microstructural injury is poorly understood. Our aim was to discover white matter injury underlying reaction time in mild traumatic brain injury DTI by applying transport-based morphometry.

MATERIALS AND METHODS

In this retrospective study, we performed DTI on 64 postconcussive patients (10-28 years of age; 69% male, 31% female) between January 2006 and March 2013. We measured the reaction time percentile by using Immediate Post-Concussion Assessment and Cognitive Testing. Using the 3D transport-based morphometry technique we developed, we mined fractional anisotropy maps to extract the common microstructural injury associated with reaction time percentile in an automated manner. Permutation testing established statistical significance of the extracted injuries. We visualized the physical substrate responsible for reaction time through inverse transport-based morphometry transformation.

RESULTS

The direction in the transport space most correlated with reaction time was significant after correcting for covariates of age, sex, and time from injury (Pearson r = 0.44, P < .01). Inverting the computed direction using transport-based morphometry illustrates physical shifts in fractional anisotropy in the corpus callosum (increase) and within the optic radiations, corticospinal tracts, and anterior thalamic radiations (decrease) with declining reaction time. The observed shifts are consistent with biologic pathways underlying the visual-spatial interpretation and response-selection aspects of reaction time.

CONCLUSIONS

Transport-based morphometry discovers complex white matter injury underlying postconcussive reaction time in an automated manner. The potential influences of edema and axonal loss are visualized in the visual-spatial interpretation and response-selection pathways. Transport-based morphometry can bridge the gap between brain microstructure and function in diseases in which the structural basis is unknown.

Fast and Robust Symmetric Image Registration Based on Distances Combining Intensity and Spatial Information

Intensity-based image registration approaches rely on similarity measures to guide the search for geometric correspondences with high affinity between images. The properties of the used measure are vital for the robustness and accuracy of the registration. In this study a symmetric, intensity interpolationfree, affine registration framework based on a combination of intensity and spatial information is proposed. The excellent performance of the framework is demonstrated on a combination of synthetic tests, recovering known transformations in the presence of noise, and real applications in biomedical and medical image registration, for both 2D and 3D images. The method exhibits greater robustness and higher accuracy than similarity measures in common use, when inserted into a standard gradientbased registration framework available as part of the open source Insight Segmentation and Registration Toolkit (ITK). The method is also empirically shown to have a low computational cost, making it practical for real applications. Source code is available.

A quantitative approach for analyzing the spatio-temporal distribution of 3D intracellular events in fluorescence microscopy

Analysis of the spatial distribution of endomembrane trafficking is fundamental to understand the mechanisms controlling cellular dynamics, cell homeostasy, and cell interaction with its external environment in normal and pathological situations. We present a semi-parametric framework to quantitatively analyze and visualize the spatio-temporal distribution of intracellular events from different conditions. From the spatial coordinates of intracellular features such as segmented subcellular structures or vesicle trajectories, QuantEv automatically estimates weighted densities that are easy to interpret and performs a comprehensive statistical analysis from distribution distances. We apply this approach to study the spatio-temporal distribution of moving Rab6 fluorescently labeled membranes with respect to their direction of movement in crossbow- and disk-shaped cells. We also investigate the position of the generating hub of Rab11-positive membranes and the effect of actin disruption on Rab11 trafficking in coordination with cell shape.

Proteins are the workhorses of the body, performing a range of roles that are essential for life. Often, this requires these molecules to move from one location to another inside a cell. Scientists are interested in following an individual protein in a living cell ‘in real time’, as this helps understand what this protein does.

Scientists can track the whereabouts of a protein by ‘tagging’ it with a fluorescent molecule that emits light which can be picked up by a powerful microscope. This process is repeated many times on different samples. Finally, researchers have to analyze all the resulting images, and conduct statistical analysis to draw robust conclusions about the overall trajectories of the proteins. This process often relies on experts assessing the images, and it is therefore time-consuming and not easily scalable or applied to other experiments.

To help with this, Pécot et al. have developed QuantEV, a free algorithm that can analyze proteins’ paths within a cell, and then return statistical graphs and 3D visualizations. The program also gives access to the statistical procedure that was used, which means that different experiments can be compared.

Pécot et al. used the method to follow the Rab6 protein in cells of different shapes, and found that the conformation of the cell influences where Rab6 is located. For example, in crossbow-shaped cells, Rab6 is found more often toward the three tips of the crossbow, while its distribution is uniform in cells that look like disks. Another experiment examined where the protein Rab11 is normally placed, and how this changes when the cell’s skeleton is artificially disrupted. Both studies help to gain an insight into the behavior of the cellular structures in which Rab6 and Rab11 are embedded.

Following proteins in the cell is an increasingly popular method, and there is therefore a growing amount of data to process. QuantEV should make it easier for biologists to analyze their results, which could help them to have a better grasp on how cells work in various circumstances.

SetSVM: An Approach to Set Classification in Nuclei-Based Cancer Detection

Due to the importance of nuclear structure in cancer diagnosis, several predictive models have been described for diagnosing a wide variety of cancers based on nuclear morphology. In many computer-aided diagnosis (CAD) systems, cancer detection tasks can be generally formulated as set classification problems, which can not be directly solved by classifying single instances. In this paper, we propose a novel set classification approach SetSVM to build a predictive model by considering any nuclei set as a whole without specific assumptions. SetSVM features highly discriminative power in cancer detection challenges in the sense that it not only optimizes the classifier decision boundary but also transfers discriminative information to set representation learning. During model training, these two processes are unified in the support vector machine (SVM) maximum separation margin problem. Experiment results show that SetSVM provides significant improvements compared with five commonly used approaches in cancer detection tasks utilizing 260 patients in total across three different cancer types, namely, thyroid cancer, liver cancer, and melanoma. In addition, we show that SetSVM enables visual interpretation of discriminative nuclear characteristics representing the nuclei set. These features make SetSVM a potentially practical tool in building accurate and interpretable CAD systems for cancer detection.

Structured illumination microscopy with interleaved reconstruction (SIMILR)

Structured illumination microscopy (SIM) is the commonly used super-resolution (SR) technique for imaging subcellular dynamics. However, due to its need for multiple illumination patterns, the frame rate is just a fraction of that of conventional microscopy and is thus too slow for fast dynamic studies. A new SR image reconstruction method that maximizes the use of each subframe of the acquisition series is proposed for improving the super-resolved frame rate by N times for N illumination directions. The method requires no changes in raw data and is appropriate for many versions of SIM setup, including those implementing fast illumination pattern generation mechanism based on spatial light modulator or digital micromirror device. The performance of the proposed method is demonstrated through imaging the highly dynamic endoplasmic reticulum where continuous rapid growths or shape changes of tiny structures are observed.

Discovery and visualization of structural biomarkers from MRI using transport-based morphometry

Disease in the brain is often associated with subtle, spatially diffuse, or complex tissue changes that may lie beneath the level of gross visual inspection, even on magnetic resonance imaging (MRI). Unfortunately, current computer-assisted approaches that examine pre-specified features, whether anatomically-defined (i.e. thalamic volume, cortical thickness) or based on pixelwise comparison (i.e. deformation-based methods), are prone to missing a vast array of physical changes that are not well-encapsulated by these metrics. In this paper, we have developed a technique for automated pattern analysis that can fully determine the relationship between brain structure and observable phenotype without requiring any a priori features. Our technique, called transport-based morphometry (TBM), is an image transformation that maps brain images losslessly to a domain where they become much more separable. The new approach is validated on structural brain images of healthy older adult subjects where even linear models for discrimination, regression, and blind source separation enable TBM to independently discover the characteristic changes of aging and highlight potential mechanisms by which aerobic fitness may mediate brain health later in life. TBM is a generative approach that can provide visualization of physically meaningful shifts in tissue distribution through inverse transformation. The proposed framework is a powerful technique that can potentially elucidate genotype-structural-behavioral associations in myriad diseases.

A Transportation Lp Distance for Signal Analysis

Transport based distances, such as the Wasserstein distance and earth mover'sdistance, have been shown to be an effective tool in signal and image analysis. The success of transport based distances is in part due to their Lagrangian nature which allows it to capture the important variations in many signal classes. However these distances require the signal to be nonnegative and normalized. Furthermore, the signals are considered as measures and compared by redistributing (transporting) them, which does not directly take into account the signal intensity. Here we study a transport-based distance, called the TLp distance, that combines Lagrangian and intensity modelling and is directly applicable to general, non-positive and multi-channelled signals. The distance can be computed by existing numerical methods. We give an overview of the basic properties of this distance and applications to classification, with multi-channelled non-positive one-dimensional signals and two-dimensional images, and color transfer.

Optimal Mass Transport: Signal processing and machine-learning applications

Transport-based techniques for signal and data analysis have received increased attention recently. Given their ability to provide accurate generative models for signal intensities and other data distributions, they have been used in a variety of applications including content-based retrieval, cancer detection, image super-resolution, and statistical machine learning, to name a few, and shown to produce state of the art results in several applications. Moreover, the geometric characteristics of transport-related metrics have inspired new kinds of algorithms for interpreting the meaning of data distributions. Here we provide a practical overview of the mathematical underpinnings of mass transport-related methods, including numerical implementation, as well as a review, with demonstrations, of several applications. Software accompanying this tutorial is available at [43].

Predictive Nuclear Chromatin Characteristics of Melanoma and Dysplastic Nevi

Background:

The diagnosis of malignant melanoma (MM) is among the diagnostic challenges pathologists encounter on a routine basis. Melanoma may arise in patients with preexisting dysplastic nevi (DN) and it is still the cause of 1.7% of all cancer-related deaths. Melanomas often have overlapping histological features with DN, especially those with severe dysplasia. Nucleotyping for identifying nuclear textural features can analyze nuclear DNA structure and organization. The aim of this study is to differentiate MM and DN using these methodologies.

Methods:

Dermatopathology slides diagnosed as MM and DN were retrieved. The glass slides were scanned using an Aperio ScanScopeXT at ×40 (0.25 μ/pixel). Whole slide images (WSI) were annotated for nuclei selection. Nuclear features to distinguish between MM and DN based on chromatin distributions were extracted from the WSI. The morphological characteristics for each nucleus were quantified with the optimal transport-based linear embedding in the continuous domain. Label predictions for individual cell nucleus are achieved through a modified version of linear discriminant analysis, coupled with the k-nearest neighbor classifier. Label for an unknown patient was set by the voting strategy with its pertaining cell nuclei.

Results:

Nucleotyping of 139 patient cases of melanoma (n = 67) and DN (n = 72) showed that our method had superior classification accuracy of 81.29%. This is a 6.4% gain in differentiating MM and DN, compared with numerical feature-based method. The distribution differences in nuclei morphology between MM and DN can be visualized with biological interpretation.

Conclusions:

Nucleotyping using quantitative and qualitative analyses may provide enough information for differentiating MM from DN using pixel image data. Our method to segment cell nuclei may offer a practical and inexpensive solution in aiding in the accurate diagnosis of melanoma.

Shape Analysis with Hyperbolic Wasserstein Distance

Shape space is an active research field in computer vision study. The shape distance defined in a shape space may provide a simple and refined index to represent a unique shape. Wasserstein distance defines a Riemannian metric for the Wasserstein space. It intrinsically measures the similarities between shapes and is robust to image noise. Thus it has the potential for the 3D shape indexing and classification research. While the algorithms for computing Wasserstein distance have been extensively studied, most of them only work for genus-0 surfaces. This paper proposes a novel framework to compute Wasserstein distance between general topological surfaces with hyperbolic metric. The computational algorithms are based on Ricci flow, hyperbolic harmonic map, and hyperbolic power Voronoi diagram and the method is general and robust. We apply our method to study human facial expression, longitudinal brain cortical morphometry with normal aging, and cortical shape classification in Alzheimer's disease (AD). Experimental results demonstrate that our method may be used as an effective shape index, which outperforms some other standard shape measures in our AD versus healthy control classification study.

A continuous linear optimal transport approach for pattern analysis in image datasets

We present a new approach to facilitate the application of the optimal transport metric to pattern recognition on image databases. The method is based on a linearized version of the optimal transport metric, which provides a linear embedding for the images. Hence, it enables shape and appearance modeling using linear geometric analysis techniques in the embedded space. In contrast to previous work, we use Monge's formulation of the optimal transport problem, which allows for reasonably fast computation of the linearized optimal transport embedding for large images. We demonstrate the application of the method to recover and visualize meaningful variations in a supervised-learning setting on several image datasets, including chromatin distribution in the nuclei of cells, galaxy morphologies, facial expressions, and bird species identification. We show that the new approach allows for high-resolution construction of modes of variations and discrimination and can enhance classification accuracy in a variety of image discrimination problems.

Artificial Neural Networks as Decision Support Tools in Cytopathology: Past, Present, and Future

OBJECTIVE

This study aims to analyze the role of artificial neural networks (ANNs) in cytopathology. More specifically, it aims to highlight the importance of employing ANNs in existing and future applications and in identifying unexplored or poorly explored research topics.

STUDY DESIGN

A systematic search was conducted in scientific databases for articles related to cytopathology and ANNs with respect to anatomical places of the human body where cytopathology is performed. For each anatomic system/organ, the major outcomes described in the scientific literature are presented and the most important aspects are highlighted.

RESULTS

The vast majority of ANN applications are related to cervical cytopathology, specifically for the ANN-based, semiautomated commercial diagnostic system PAPNET. For cervical cytopathology, there is a plethora of studies relevant to the diagnostic accuracy; in addition, there are also efforts evaluating cost-effectiveness and applications on primary, secondary, or hybrid screening. For the rest of the anatomical sites, such as the gastrointestinal system, thyroid gland, urinary tract, and breast, there are significantly less efforts relevant to the application of ANNs. Additionally, there are still anatomical systems for which ANNs have never been applied on their cytological material.

CONCLUSIONS

Cytopathology is an ideal discipline to apply ANNs. In general, diagnosis is performed by experts via the light microscope. However, this approach introduces subjectivity, because this is not a universal and objective measurement process. This has resulted in the existence of a gray zone between normal and pathological cases. From the analysis of related articles, it is obvious that there is a need to perform more thorough analyses, using extensive number of cases and particularly for the nonexplored organs. Efforts to apply such systems within the laboratory test environment are required for their future uptake.

The Radon Cumulative Distribution Transform and Its Application to Image Classification

Invertible image representation methods (transforms) are routinely employed as low-level image processing operations based on which feature extraction and recognition algorithms are developed. Most transforms in current use (e.g., Fourier, wavelet, and so on) are linear transforms and, by themselves, are unable to substantially simplify the representation of image classes for classification. Here, we describe a nonlinear, invertible, low-level image processing transform based on combining the well-known Radon transform for image data, and the 1D cumulative distribution transform proposed earlier. We describe a few of the properties of this new transform, and with both theoretical and experimental results show that it can often render certain problems linearly separable in a transform space.

New methods for quantifying and visualizing information from images of cells: An overview

New microscopy imaging techniques have enabled the acquisition of cellular and sub-cellular information with unprecedented accuracy and specificity. Fluorescence techniques have enabled labeling of numerous, previously inaccessible, molecules and organelles, while Raman spectrographic techniques, for example, have enabled label free acquisition. Together with the development of high throughput techniques, these technologies now allow for the acquisition of a significant amount of information about cellular processes and have enabled high throughput and high content screening. Beyond image formation and acquisition, computational techniques comprise an important part of the process of obtaining biological understanding from such experiments. Here we review the pros and cons of the main approaches that have been used to extract information from digital images of cells. In addition, we also offer an overview of modern computational techniques that beyond allowing for discrimination between two hypothesis, also allow for modeling, visualization, and understanding of biological phenomena.

Detection of malignant mesothelioma using nuclear structure of mesothelial cells in effusion cytology specimens

Mesothelioma is a form of cancer generally caused from previous exposure to asbestos. Although it was considered a rare neoplasm in the past, its incidence is increasing worldwide due to extensive use of asbestos. In the current practice of medicine, the gold standard for diagnosing mesothelioma is through a pleural biopsy with subsequent histologic examination of the tissue. The diagnostic tissue should demonstrate the invasion by the tumor and is obtained through thoracoscopy or open thoracotomy, both being highly invasive surgical operations. On the other hand, thoracocentesis, which is removal of effusion fluid from the pleural space, is a far less invasive procedure that can provide material for cytological examination. In this study, we aim at detecting and classifying malignant mesothelioma based on the nuclear chromatin distribution from digital images of mesothelial cells in effusion cytology specimens. Accordingly, a computerized method is developed to determine whether a set of nuclei belonging to a patient is benign or malignant. The quantification of chromatin distribution is performed by using the optimal transport-based linear embedding for segmented nuclei in combination with the modified Fisher discriminant analysis. Classification is then performed through a k-nearest neighborhood approach and a basic voting strategy. Our experiments on 34 different human cases result in 100% accurate predictions computed with blind cross validation. Experimental comparisons also show that the new method can significantly outperform standard numerical feature-type methods in terms of agreement with the clinical diagnosis gold standard. According to our results, we conclude that nuclear structure of mesothelial cells alone may contain enough information to separate malignant mesothelioma from benign mesothelial proliferations.

Shape Classification Using Wasserstein Distance for Brain Morphometry Analysis

Brain morphometry study plays a fundamental role in medical imaging analysis and diagnosis. This work proposes a novel framework for brain cortical surface classification using Wasserstein distance, based on uniformization theory and Riemannian optimal mass transport theory. By Poincare uniformization theorem, all shapes can be conformally deformed to one of the three canonical spaces: the unit sphere, the Euclidean plane or the hyperbolic plane. The uniformization map will distort the surface area elements. The area-distortion factor gives a probability measure on the canonical uniformization space. All the probability measures on a Riemannian manifold form the Wasserstein space. Given any 2 probability measures, there is a unique optimal mass transport map between them, the transportation cost defines the Wasserstein distance between them. Wasserstein distance gives a Riemannian metric for the Wasserstein space. It intrinsically measures the dissimilarities between shapes and thus has the potential for shape classification. To the best of our knowledge, this is the first. work to introduce the optimal mass transport map to general Riemannian manifolds. The method is based on geodesic power Voronoi diagram. Comparing to the conventional methods, our approach solely depends on Riemannian metrics and is invariant under rigid motions and scalings, thus it intrinsically measures shape distance. Experimental results on classifying brain cortical surfaces with different intelligence quotients demonstrated the efficiency and efficacy of our method.

Carnegie Mellon University bioimaging day 2014: Challenges and opportunities in digital pathology

Recent advances in digital imaging is impacting the practice of pathology. One of the key enabling technologies that is leading the way towards this transformation is the use of whole slide imaging (WSI) which allows glass slides to be converted into large image files that can be shared, stored, and analyzed rapidly. Many applications around this novel technology have evolved in the last decade including education, research and clinical applications. This publication highlights a collection of abstracts, each corresponding to a talk given at Carnegie Mellon University's (CMU) Bioimaging Day 2014 co-sponsored by the Biomedical Engineering and Lane Center for Computational Biology Departments at CMU. Topics related specifically to digital pathology are presented in this collection of abstracts. These include topics related to digital workflow implementation, imaging and artifacts, storage demands, and automated image analysis algorithms.

Cancer diagnosis by nuclear morphometry using spatial information

Methods for extracting quantitative information regarding nuclear morphology from histopathology images have been long used to aid pathologists in determining the degree of differentiation in numerous malignancies. Most methods currently in use, however, employ the naïve Bayes approach to classify a set of nuclear measurements extracted from one patient. Hence, the statistical dependency between the samples (nuclear measurements) is often not directly taken into account. Here we describe a method that makes use of statistical dependency between samples in thyroid tissue to improve patient classification accuracies with respect to standard naïve Bayes approaches. We report results in two sample diagnostic challenges.

Accurate diagnosis of thyroid follicular lesions from nuclear morphology using supervised learning

Follicular lesions of the thyroid remain significant diagnostic challenges in surgical pathology and cytology. The diagnosis often requires considerable resources and ancillary tests including immunohistochemistry, molecular studies, and expert consultation. Visual analyses of nuclear morphological features, generally speaking, have not been helpful in distinguishing this group of lesions. Here we describe a method for distinguishing between follicular lesions of the thyroid based on nuclear morphology. The method utilizes an optimal transport-based linear embedding for segmented nuclei, together with an adaptation of existing classification methods. We show the method outputs assignments (classification results) which are near perfectly correlated with the clinical diagnosis of several lesion types' lesions utilizing a database of 94 patients in total. Experimental comparisons also show the new method can significantly outperform standard numerical feature-type methods in terms of agreement with the clinical diagnosis gold standard. In addition, the new method could potentially be used to derive insights into biologically meaningful nuclear morphology differences in these lesions. Our methods could be incorporated into a tool for pathologists to aid in distinguishing between follicular lesions of the thyroid. In addition, these results could potentially provide nuclear morphological correlates of biological behavior and reduce health care costs by decreasing histotechnician and pathologist time and obviating the need for ancillary testing.

Shape representation for efficient landmark-based segmentation in 3-d

In this paper, we propose a novel approach to landmark-based shape representation that is based on transportation theory, where landmarks are considered as sources and destinations, all possible landmark connections as roads, and established landmark connections as goods transported via these roads. Landmark connections, which are selectively established, are identified through their statistical properties describing the shape of the object of interest, and indicate the least costly roads for transporting goods from sources to destinations. From such a perspective, we introduce three novel shape representations that are combined with an existing landmark detection algorithm based on game theory. To reduce computational complexity, which results from the extension from 2-D to 3-D segmentation, landmark detection is augmented by a concept known in game theory as strategy dominance. The novel shape representations, game-theoretic landmark detection and strategy dominance are combined into a segmentation framework that was evaluated on 3-D computed tomography images of lumbar vertebrae and femoral heads. The best shape representation yielded symmetric surface distance of 0.75 mm and 1.11 mm, and Dice coefficient of 93.6% and 96.2% for lumbar vertebrae and femoral heads, respectively. By applying strategy dominance, the computational costs were further reduced for up to three times.

Interpolation of Longitudinal Shape and Image Data via Optimal Mass Transport

Longitudinal analysis of medical imaging data has become central to the study of many disorders. Unfortunately, various constraints (study design, patient availability, technological limitations) restrict the acquisition of data to only a few time points, limiting the study of continuous disease/treatment progression. Having the ability to produce a sensible time interpolation of the data can lead to improved analysis, such as intuitive visualizations of anatomical changes, or the creation of more samples to improve statistical analysis. In this work, we model interpolation of medical image data, in particular shape data, using the theory of optimal mass transport (OMT), which can construct a continuous transition from two time points while preserving "mass" (e.g., image intensity, shape volume) during the transition. The theory even allows a short extrapolation in time and may help predict short-term treatment impact or disease progression on anatomical structure. We apply the proposed method to the hippocampus-amygdala complex in schizophrenia, the heart in atrial fibrillation, and full head MR images in traumatic brain injury.

Detecting and visualizing cell phenotype differences from microscopy images using transport-based morphometry

Modern microscopic imaging devices are able to extract more information regarding the subcellular organization of different molecules and proteins than can be obtained by visual inspection. Predetermined numerical features (descriptors) often used to quantify cells extracted from these images have long been shown useful for discriminating cell populations (e.g., normal vs. diseased). Direct visual or biological interpretation of results obtained, however, is often not a trivial task. We describe an approach for detecting and visualizing phenotypic differences between classes of cells based on the theory of optimal mass transport. The method is completely automated, does not require the use of predefined numerical features, and at the same time allows for easily interpretable visualizations of the most significant differences. Using this method, we demonstrate that the distribution pattern of peripheral chromatin in the nuclei of cells extracted from liver and thyroid specimens is associated with malignancy. We also show the method can correctly recover biologically interpretable and statistically significant differences in translocation imaging assays in a completely automated fashion.