Detecting and visualizing cell phenotype differences from microscopy images using transport-based morphometry

Discovering the gene-brain-behavior link in autism via generative machine learning

Autism is traditionally diagnosed behaviorally but has a strong genetic basis. A genetics-first approach could transform understanding and treatment of autism. However, isolating the gene-brain-behavior relationship from confounding sources of variability is a challenge. We demonstrate a novel technique, 3D transport-based morphometry (TBM), to extract the structural brain changes linked to genetic copy number variation (CNV) at the 16p11.2 region. We identified two distinct endophenotypes. In data from the Simons Variation in Individuals Project, detection of these endophenotypes enabled 89 to 95% test accuracy in predicting 16p11.2 CNV from brain images alone. Then, TBM enabled direct visualization of the endophenotypes driving accurate prediction, revealing dose-dependent brain changes among deletion and duplication carriers. These endophenotypes are sensitive to articulation disorders and explain a portion of the intelligence quotient variability. Genetic stratification combined with TBM could reveal new brain endophenotypes in many neurodevelopmental disorders, accelerating precision medicine, and understanding of human neurodiversity.

Morphological Signal Processing for Phenotype Recognition of Human Pluripotent Stem Cells Using Machine Learning Methods

Human pluripotent stem cells have the potential for unlimited proliferation and controlled differentiation into various somatic cells, making them a unique tool for regenerative and personalized medicine. Determining the best clone selection is a challenging problem in this field and requires new sensing instruments and methods able to automatically assess the state of a growing colony ('phenotype') and make decisions about its destiny. One possible solution for such label-free, non-invasive assessment is to make phase-contrast images and/or videos of growing stem cell colonies, process the morphological parameters ('morphological portrait', or signal), link this information to the colony phenotype, and initiate an automated protocol for the colony selection. As a step in implementing this strategy, we used machine learning methods to find an effective model for classifying the human pluripotent stem cell colonies of three lines according to their morphological phenotype ('good' or 'bad'), using morphological parameters from the previously published data as predictors. We found that the model using cellular morphological parameters as predictors and artificial neural networks as the classification method produced the best average accuracy of phenotype prediction (67%). When morphological parameters of colonies were used as predictors, logistic regression was the most effective classification method (75% average accuracy). Combining the morphological parameters of cells and colonies resulted in the most effective model, with a 99% average accuracy of phenotype prediction. Random forest was the most efficient classification method for the combined data. We applied feature selection methods and showed that different morphological parameters were important for phenotype recognition via either cellular or colonial parameters. Our results indicate a necessity for retaining both cellular and colonial morphological information for predicting the phenotype and provide an optimal choice for the machine learning method. The classification models reported in this study could be used as a basis for developing and/or improving automated solutions to control the quality of human pluripotent stem cells for medical purposes.

A novel statistical methodology for quantifying the spatial arrangements of axons in peripheral nerves

A thorough understanding of the neuroanatomy of peripheral nerves is required for a better insight into their function and the development of neuromodulation tools and strategies. In biophysical modeling, it is commonly assumed that the complex spatial arrangement of myelinated and unmyelinated axons in peripheral nerves is random, however, in reality the axonal organization is inhomogeneous and anisotropic. Present quantitative neuroanatomy methods analyze peripheral nerves in terms of the number of axons and the morphometric characteristics of the axons, such as area and diameter. In this study, we employed spatial statistics and point process models to describe the spatial arrangement of axons and Sinkhorn distances to compute the similarities between these arrangements (in terms of first- and second-order statistics) in various vagus and pelvic nerve cross-sections. We utilized high-resolution transmission electron microscopy (TEM) images that have been segmented using a custom-built high-throughput deep learning system based on a highly modified U-Net architecture. Our findings show a novel and innovative approach to quantifying similarities between spatial point patterns using metrics derived from the solution to the optimal transport problem. We also present a generalizable pipeline for quantitative analysis of peripheral nerve architecture. Our data demonstrate differences between male- and female-originating samples and similarities between the pelvic and abdominal vagus nerves.

Next-Generation Morphometry for pathomics-data mining in histopathology

Pathology diagnostics relies on the assessment of morphology by trained experts, which remains subjective and qualitative. Here we developed a framework for large-scale histomorphometry (FLASH) performing deep learning-based semantic segmentation and subsequent large-scale extraction of interpretable, quantitative, morphometric features in non-tumour kidney histology. We use two internal and three external, multi-centre cohorts to analyse over 1000 kidney biopsies and nephrectomies. By associating morphometric features with clinical parameters, we confirm previous concepts and reveal unexpected relations. We show that the extracted features are independent predictors of long-term clinical outcomes in IgA-nephropathy. We introduce single-structure morphometric analysis by applying techniques from single-cell transcriptomics, identifying distinct glomerular populations and morphometric phenotypes along a trajectory of disease progression. Our study provides a concept for Next-generation Morphometry (NGM), enabling comprehensive quantitative pathology data mining, i.e., pathomics.

On a Linear Gromov-Wasserstein Distance

Gromov-Wasserstein distances are generalization of Wasserstein distances, which are invariant under distance preserving transformations. Although a simplified version of optimal transport in Wasserstein spaces, called linear optimal transport (LOT), was successfully used in practice, there does not exist a notion of linear Gromov-Wasserstein distances so far. In this paper, we propose a definition of linear Gromov-Wasserstein distances. We motivate our approach by a generalized LOT model, which is based on barycentric projection maps of transport plans. Numerical examples illustrate that the linear Gromov-Wasserstein distances, similarly as LOT, can replace the expensive computation of pairwise Gromov-Wasserstein distances in applications like shape classification.

Investigating impact of cardiorespiratory fitness in reducing brain tissue loss caused by ageing

Mitigating the loss of brain tissue due to age is a major problem for an ageing population. Improving cardiorespiratory fitness has been suggested as a possible strategy, but the influenceon brain morphology has not been fully characterized. To investigate the dependent shifts in brain tissue distribution as a function of cardiorespiratory fitness, we used a 3D transport-based morphometry approach. In this study of 172 inactive older adults aged 58-81 (66.5 ± 5.7) years, cardiorespiratory fitness was determined by VO 2 peak (ml/kg/min) during graded exercise and brain morphology was assessed through structural magnetic resonance imaging. After correcting for covariates including age (in the fitness model), gender and level of education, we compared dependent tissue shifts with age to those due to V O 2   peak . We found a significant association between cardiorespiratory fitness and brain tissue distribution (white matter, r = 0.30, P = 0.003; grey matter, r = 0.40, P < 0.001) facilitated by direct visualization of the brain tissue shifts due to cardiorespiratory fitness through inverse transformation-a key capability of 3D transport-based morphometry. A strong statistical correlation was found between brain tissue changes related to ageing and those associated with lower cardiorespiratory fitness (white matter, r = 0.62, P < 0.001; grey matter, r = 0.74, P < 0.001). In both cases, frontotemporal regions shifted the most while basal ganglia shifted the least. Our results highlight the importance of cardiorespiratory fitness in maintaining brain health later in life. Furthermore, this work demonstrates 3D transport-based morphometry as a novel neuroinformatic technology that may aid assessment of therapeutic approaches for brain ageing and neurodegenerative diseases.

Radon Cumulative Distribution Transform Subspace Modeling for Image Classification

We present a new supervised image classification method applicable to a broad class of image deformation models. The method makes use of the previously described Radon Cumulative Distribution Transform (R-CDT) for image data, whose mathematical properties are exploited to express the image data in a form that is more suitable for machine learning. While certain operations such as translation, scaling, and higher-order transformations are challenging to model in native image space, we show the R-CDT can capture some of these variations and thus render the associated image classification problems easier to solve. The method - utilizing a nearest-subspace algorithm in the R-CDT space - is simple to implement, non-iterative, has no hyper-parameters to tune, is computationally efficient, label efficient, and provides competitive accuracies to state-of-the-art neural networks for many types of classification problems. In addition to the test accuracy performances, we show improvements (with respect to neural network-based methods) in terms of computational efficiency (it can be implemented without the use of GPUs), number of training samples needed for training, as well as out-of-distribution generalization. The Python code for reproducing our results is available at [1].

Raman and quantitative phase imaging allow morpho-molecular recognition of malignancy and stages of B-cell acute lymphoblastic leukemia

Acute lymphoblastic leukemia (ALL) is one of the most common malignancies that account for nearly one-third of all pediatric cancers. The current diagnostic assays are time-consuming, labor-intensive, and require expensive reagents. Here, we report a label-free approach featuring diffraction phase imaging and Raman microscopy that can retrieve both morphological and molecular attributes for label-free optical phenotyping of individual B cells. By investigating leukemia cell lines of early and late stages along with the healthy B cells, we show that phase images can capture subtle morphological differences among the healthy, early, and late stages of leukemic cells. By exploiting its biomolecular specificity, we demonstrate that Raman microscopy is capable of accurately identifying not only different stages of leukemia cells but also individual cell lines at each stage. Overall, our study provides a rationale for employing this hybrid modality to screen leukemia cells using the widefield QPI and using Raman microscopy for accurate differentiation of early and late-stage phenotypes. This contrast-free and rapid diagnostic tool exhibits great promise for clinical diagnosis and staging of leukemia in the near future.

PARTITIONING SIGNAL CLASSES USING TRANSPORT TRANSFORMS FOR DATA ANALYSIS AND MACHINE LEARNING

A relatively new set of transport-based transforms (CDT, R-CDT, LOT) have shown their strength and great potential in various image and data processing tasks such as parametric signal estimation, classification, cancer detection among many others. It is hence worthwhile to elucidate some of the mathematical properties that explain the successes of these transforms when they are used as tools in data analysis, signal processing or data classification. In particular, we give conditions under which classes of signals that are created by algebraic generative models are transformed into convex sets by the transport transforms. Such convexification of the classes simplify the classification and other data analysis and processing problems when viewed in the transform domain. More specifically, we study the extent and limitation of the convexification ability of these transforms under an algebraic generative modeling framework. We hope that this paper will serve as an introduction to these transforms and will encourage mathematicians and other researchers to further explore the theoretical underpinnings and algorithmic tools that will help understand the successes of these transforms and lay the groundwork for further successful applications.

Irregular Respiratory Motion Compensation for Liver Contrast-Enhanced Ultrasound via Transport-Based Motion Estimation

Contrast-enhanced ultrasound (CEUS) imaging has been widely applied for the detection and characterization of focal liver lesions (FLLs), for its ability to visualize the blood flow in real time. However, cyclic liver motion poses a great challenge to the recovery of perfusion curves as well as quantitative kinetic parameters estimation. Recently, a few gating methods have been proposed to eliminate unexpected intensity fluctuations by the breathing phase estimation, with the assumption that each breathing phase corresponds to a specific lesion position strictly. While practical liver motion tends to be irregular due to changes in the patient's underlying physiologic status, that is, the same phase might not correspond to the same position. To tackle the challenge of motion irregularity, we present a novel motion estimation-based respiratory compensation method, named RCME, which first estimates salient motion information through the framework of optimal transport (OT) by jointly modeling pixel intensity as well as their locations and then employs sparse subspace clustering (SSC) to identify the subset of frames acquired at the same position. Our proposed method is evaluated on 15 clinical CEUS sequences in both qualitative and quantitative manners. Experimental results demonstrate good performance on irregular liver motion compensation.

Coarse Raman and optical diffraction tomographic imaging enable label-free phenotyping of isogenic breast cancer cells of varying metastatic potential

Identification of the metastatic potential represents one of the most important tasks for molecular imaging of cancer. While molecular imaging of metastases has witnessed substantial progress as an area of clinical inquiry, determining precisely what differentiates the metastatic phenotype has proven to be more elusive. In this study, we utilize both the morphological and molecular information provided by 3D optical diffraction tomography and Raman spectroscopy, respectively, to propose a label-free route for optical phenotyping of cancer cells at single-cell resolution. By using an isogenic panel of cell lines derived from MDA-MB-231 breast cancer cells that vary in their metastatic potential, we show that 3D refractive index tomograms can capture subtle morphological differences among the parental, circulating tumor cells, and lung metastatic cells. By leveraging its molecular specificity, we demonstrate that coarse Raman microscopy is capable of rapidly mapping a sufficient number of cells for training a random forest classifier that can accurately predict the metastatic potential of cells at a single-cell level. We also perform multivariate curve resolution alternating least squares decomposition of the spectral dataset to demarcate spectra from cytoplasm and nucleus, and test the feasibility of identifying metastatic phenotypes using the spectra only from the cytoplasmic and nuclear regions. Overall, our study provides a rationale for employing coarse Raman mapping to substantially reduce measurement time thereby enabling the acquisition of reasonably large training datasets that hold the key for label-free single-cell analysis and, consequently, for differentiation of indolent from aggressive phenotypes.

In Situ Classification of Cell Types in Human Kidney Tissue Using 3D Nuclear Staining

To understand the physiology and pathology of disease, capturing the heterogeneity of cell types within their tissue environment is fundamental. In such an endeavor, the human kidney presents a formidable challenge because its complex organizational structure is tightly linked to key physiological functions. Advances in imaging-based cell classification may be limited by the need to incorporate specific markers that can link classification to function. Multiplex imaging can mitigate these limitations, but requires cumulative incorporation of markers, which may lead to tissue exhaustion. Furthermore, the application of such strategies in large scale 3-dimensional (3D) imaging is challenging. Here, we propose that 3D nuclear signatures from a DNA stain, DAPI, which could be incorporated in most experimental imaging, can be used for classifying cells in intact human kidney tissue. We developed an unsupervised approach that uses 3D tissue cytometry to generate a large training dataset of nuclei images (NephNuc), where each nucleus is associated with a cell type label. We then devised various supervised machine learning approaches for kidney cell classification and demonstrated that a deep learning approach outperforms classical machine learning or shape-based classifiers. Specifically, a custom 3D convolutional neural network (NephNet3D) trained on nuclei image volumes achieved a balanced accuracy of 80.26%. Importantly, integrating NephNet3D classification with tissue cytometry allowed in situ visualization of cell type classifications in kidney tissue. In conclusion, we present a tissue cytometry and deep learning approach for in situ classification of cell types in human kidney tissue using only a DNA stain. This methodology is generalizable to other tissues and has potential advantages on tissue economy and non-exhaustive classification of different cell types.

Identification and Staging of B-Cell Acute Lymphoblastic Leukemia Using Quantitative Phase Imaging and Machine Learning

Identification and classification of leukemia cells in a rapid and label-free fashion is clinically challenging and thus presents a prime arena for implementing new diagnostic tools. Quantitative phase imaging, which maps optical path length delays introduced by the specimen, has been demonstrated to discern cellular phenotypes based on differential morphological attributes. Rapid acquisition capability and the availability of label-free images with high information content have enabled researchers to use machine learning (ML) to reveal latent features. We developed a set of ML classifiers, including convolutional neural networks, to discern healthy B cells from lymphoblasts and classify stages of B cell acute lymphoblastic leukemia. Here, we show that the average dry mass and volume of normal B cells are lower than those of cancerous cells and that these morphologic parameters increase further alongside disease progression. We find that the relaxed training requirements of a ML approach are conducive to the classification of cell type, with minimal space, training time, and memory requirements. Our findings pave the way for a larger study on clinical samples of acute lymphoblastic leukemia, with the overarching goal of its broader use in hematopathology, where the prospect of objective diagnoses with minimal sample preparation remains highly desirable.

Trajectories from Distribution-valued Functional Curves: A Unified Wasserstein Framework★

Temporal changes in medical images are often evaluated along a parametrized function that represents a structure of interest (e.g. white matter tracts). By attributing samples along these functions with distributions of image properties in the local neighborhood, we create distribution-valued signatures for these functions. We propose a novel and comprehensive framework which models their temporal evolution trajectories. This is achieved under the unifying scheme of Wasserstein distance metric. The regression problem is formulated as a constrained optimization problem and solved using an alternating projection algorithm. The solution simultaneously preserves the functional characteristics of the curve, models the temporal change in distribution profiles and forces the estimated distributions to be valid. Hypothesis testing is applied in two ways using Wasserstein based test statistics. Validation is presented on synthetic data. Detection of delayed growth is shown on DTI tracts, for a pediatric subject with respect to a healthy population of infants.

Enabling early detection of osteoarthritis from presymptomatic cartilage texture maps via transport-based learning

Many diseases have no visual cues in the early stages, eluding image-based detection. Today, osteoarthritis (OA) is detected after bone damage has occurred, at an irreversible stage of the disease. Currently no reliable method exists for OA detection at a reversible stage. We present an approach that enables sensitive OA detection in presymptomatic individuals. Our approach combines optimal mass transport theory with statistical pattern recognition. Eighty-six healthy individuals were selected from the Osteoarthritis Initiative, with no symptoms or visual signs of disease on imaging. On 3-y follow-up, a subset of these individuals had progressed to symptomatic OA. We trained a classifier to differentiate progressors and nonprogressors on baseline cartilage texture maps, which achieved a robust test accuracy of 78% in detecting future symptomatic OA progression 3 y prior to symptoms. This work demonstrates that OA detection may be possible at a potentially reversible stage. A key contribution of our work is direct visualization of the cartilage phenotype defining predictive ability as our technique is generative. We observe early biochemical patterns of fissuring in cartilage that define future onset of OA. In the future, coupling presymptomatic OA detection with emergent clinical therapies could modify the outcome of a disease that costs the United States healthcare system $16.5 billion annually. Furthermore, our technique is broadly applicable to earlier image-based detection of many diseases currently diagnosed at advanced stages today.

Cell Image Classification: A Comparative Overview

Cell image classification methods are currently being used in numerous applications in cell biology and medicine. Applications include understanding the effects of genes and drugs in screening experiments, understanding the role and subcellular localization of different proteins, as well as diagnosis and prognosis of cancer from images acquired using cytological and histological techniques. The article also reviews three main approaches for cell image classification most often used: numerical feature extraction, end-to-end classification with neural networks (NNs), and transport-based morphometry (TBM). In addition, we provide comparisons on four different cell imaging datasets to highlight the relative strength of each method. The results computed using four publicly available datasets show that numerical features tend to carry the best discriminative information for most of the classification tasks. Results also show that NN-based methods produce state-of-the-art results in the dataset that contains a relatively large number of training samples. Data augmentation or the choice of a more recently reported architecture does not necessarily improve the classification performance of NNs in the datasets with limited number of training samples. If understanding and visualization are desired aspects, TBM methods can offer the ability to invert classification functions, and thus can aid in the interpretation of results. These and other comparison outcomes are discussed with the aim of clarifying the advantages and disadvantages of each method. © 2020 International Society for Advancement of Cytometry.

Path-Based Dictionary Augmentation: A Framework for Improving k-Sparse Image Processing

We have previously shown that augmenting orthogonal matching pursuit (OMP) with an additional step in the identification stage of each pursuit iteration yields improved k-sparse reconstruction and denoising performance relative to baseline OMP. At each iteration a "path," or geodesic, is generated between the two dictionary atoms that are most correlated with the residual and from this path a new atom that has a greater correlation to the residual than either of the two bracketing atoms is selected. Here, we provide new computational results illustrating improvements in sparse coding and denoising on canonical datasets using both learned and structured dictionaries. Two methods of constructing a path are investigated for each dictionary type: the Euclidean geodesic formed by a linear combination of the two atoms and the 2-Wasserstein geodesic corresponding to the optimal transport map between the atoms. We prove here the existence of a higher-correlation atom in the Euclidean case under assumptions on the two bracketing atoms and introduce algorithmic modifications to improve the likelihood that the bracketing atoms meet those conditions. Although we demonstrate our augmentation on OMP alone, in general it may be applied to any reconstruction algorithm that relies on the selection and sorting of high-similarity atoms during an analysis or identification phase.

Assessing Postconcussive Reaction Time Using Transport-Based Morphometry of Diffusion Tensor Images

BACKGROUND AND PURPOSE

Cognitive deficits are among the most commonly reported post-concussive symptoms, yet the underlying microstructural injury is poorly understood. Our aim was to discover white matter injury underlying reaction time in mild traumatic brain injury DTI by applying transport-based morphometry.

MATERIALS AND METHODS

In this retrospective study, we performed DTI on 64 postconcussive patients (10-28 years of age; 69% male, 31% female) between January 2006 and March 2013. We measured the reaction time percentile by using Immediate Post-Concussion Assessment and Cognitive Testing. Using the 3D transport-based morphometry technique we developed, we mined fractional anisotropy maps to extract the common microstructural injury associated with reaction time percentile in an automated manner. Permutation testing established statistical significance of the extracted injuries. We visualized the physical substrate responsible for reaction time through inverse transport-based morphometry transformation.

RESULTS

The direction in the transport space most correlated with reaction time was significant after correcting for covariates of age, sex, and time from injury (Pearson r = 0.44, P < .01). Inverting the computed direction using transport-based morphometry illustrates physical shifts in fractional anisotropy in the corpus callosum (increase) and within the optic radiations, corticospinal tracts, and anterior thalamic radiations (decrease) with declining reaction time. The observed shifts are consistent with biologic pathways underlying the visual-spatial interpretation and response-selection aspects of reaction time.

CONCLUSIONS

Transport-based morphometry discovers complex white matter injury underlying postconcussive reaction time in an automated manner. The potential influences of edema and axonal loss are visualized in the visual-spatial interpretation and response-selection pathways. Transport-based morphometry can bridge the gap between brain microstructure and function in diseases in which the structural basis is unknown.

Reagent-Free and Rapid Assessment of T Cell Activation State Using Diffraction Phase Microscopy and Deep Learning

CD8+ T cells constitute an essential compartment of the adaptive immune system. During immune responses, naı̈ve T cells become functional, as they are primed with their cognate determinants by the antigen presenting cells. Current methods of identifying activated CD8+ T cells are laborious, time-consuming and expensive due to the extensive list of required reagents. Here, we demonstrate an optical imaging approach featuring quantitative phase imaging to distinguish activated CD8+ T cells from naı̈ve CD8+ T cells in a rapid and reagent-free manner. We measured the dry mass of live cells and employed transport-based morphometry to better understand their differential morphological attributes. Our results reveal that, upon activation, the dry cell mass of T cells increases significantly in comparison to that of unstimulated cells. By employing deep learning formalism, we are able to accurately predict the population ratios of unknown mixed population based on the acquired quantitative phase images. We envision that, with further refinement, this label-free method of T cell phenotyping will lead to a rapid and cost-effective platform for assaying T cell responses to candidate antigens in the near future.

Learning to Predict Ischemic Stroke Growth on Acute CT Perfusion Data by Interpolating Low-Dimensional Shape Representations

Cerebrovascular diseases, in particular ischemic stroke, are one of the leading global causes of death in developed countries. Perfusion CT and/or MRI are ideal imaging modalities for characterizing affected ischemic tissue in the hyper-acute phase. If infarct growth over time could be predicted accurately from functional acute imaging protocols together with advanced machine-learning based image analysis, the expected benefits of treatment options could be better weighted against potential risks. The quality of the outcome prediction by convolutional neural networks (CNNs) is so far limited, which indicates that even highly complex deep learning algorithms are not fully capable of directly learning physiological principles of tissue salvation through weak supervision due to a lack of data (e.g., follow-up segmentation). In this work, we address these current shortcomings by explicitly taking into account clinical expert knowledge in the form of segmentations of the core and its surrounding penumbra in acute CT perfusion images (CTP), that are trained to be represented in a low-dimensional non-linear shape space. Employing a multi-scale CNN (U-Net) together with a convolutional auto-encoder, we predict lesion tissue probabilities for new patients. The predictions are physiologically constrained to a shape embedding that encodes a continuous progression between the core and penumbra extents. The comparison to a simple interpolation in the original voxel space and an unconstrained CNN shows that the use of such a shape space can be advantageous to predict time-dependent growth of stroke lesions on acute perfusion data, yielding a Dice score overlap of 0.46 for predictions from expert segmentations of core and penumbra. Our interpolation method models monotone infarct growth robustly on a linear time scale to automatically predict clinically plausible tissue outcomes that may serve as a basis for more clinical measures such as the expected lesion volume increase and can support the decision making on treatment options and triage.

A quantitative approach for analyzing the spatio-temporal distribution of 3D intracellular events in fluorescence microscopy

Analysis of the spatial distribution of endomembrane trafficking is fundamental to understand the mechanisms controlling cellular dynamics, cell homeostasy, and cell interaction with its external environment in normal and pathological situations. We present a semi-parametric framework to quantitatively analyze and visualize the spatio-temporal distribution of intracellular events from different conditions. From the spatial coordinates of intracellular features such as segmented subcellular structures or vesicle trajectories, QuantEv automatically estimates weighted densities that are easy to interpret and performs a comprehensive statistical analysis from distribution distances. We apply this approach to study the spatio-temporal distribution of moving Rab6 fluorescently labeled membranes with respect to their direction of movement in crossbow- and disk-shaped cells. We also investigate the position of the generating hub of Rab11-positive membranes and the effect of actin disruption on Rab11 trafficking in coordination with cell shape.

Proteins are the workhorses of the body, performing a range of roles that are essential for life. Often, this requires these molecules to move from one location to another inside a cell. Scientists are interested in following an individual protein in a living cell ‘in real time’, as this helps understand what this protein does.

Scientists can track the whereabouts of a protein by ‘tagging’ it with a fluorescent molecule that emits light which can be picked up by a powerful microscope. This process is repeated many times on different samples. Finally, researchers have to analyze all the resulting images, and conduct statistical analysis to draw robust conclusions about the overall trajectories of the proteins. This process often relies on experts assessing the images, and it is therefore time-consuming and not easily scalable or applied to other experiments.

To help with this, Pécot et al. have developed QuantEV, a free algorithm that can analyze proteins’ paths within a cell, and then return statistical graphs and 3D visualizations. The program also gives access to the statistical procedure that was used, which means that different experiments can be compared.

Pécot et al. used the method to follow the Rab6 protein in cells of different shapes, and found that the conformation of the cell influences where Rab6 is located. For example, in crossbow-shaped cells, Rab6 is found more often toward the three tips of the crossbow, while its distribution is uniform in cells that look like disks. Another experiment examined where the protein Rab11 is normally placed, and how this changes when the cell’s skeleton is artificially disrupted. Both studies help to gain an insight into the behavior of the cellular structures in which Rab6 and Rab11 are embedded.

Following proteins in the cell is an increasingly popular method, and there is therefore a growing amount of data to process. QuantEV should make it easier for biologists to analyze their results, which could help them to have a better grasp on how cells work in various circumstances.

Discovery and visualization of structural biomarkers from MRI using transport-based morphometry

Disease in the brain is often associated with subtle, spatially diffuse, or complex tissue changes that may lie beneath the level of gross visual inspection, even on magnetic resonance imaging (MRI). Unfortunately, current computer-assisted approaches that examine pre-specified features, whether anatomically-defined (i.e. thalamic volume, cortical thickness) or based on pixelwise comparison (i.e. deformation-based methods), are prone to missing a vast array of physical changes that are not well-encapsulated by these metrics. In this paper, we have developed a technique for automated pattern analysis that can fully determine the relationship between brain structure and observable phenotype without requiring any a priori features. Our technique, called transport-based morphometry (TBM), is an image transformation that maps brain images losslessly to a domain where they become much more separable. The new approach is validated on structural brain images of healthy older adult subjects where even linear models for discrimination, regression, and blind source separation enable TBM to independently discover the characteristic changes of aging and highlight potential mechanisms by which aerobic fitness may mediate brain health later in life. TBM is a generative approach that can provide visualization of physically meaningful shifts in tissue distribution through inverse transformation. The proposed framework is a powerful technique that can potentially elucidate genotype-structural-behavioral associations in myriad diseases.

A Transportation Lp Distance for Signal Analysis

Transport based distances, such as the Wasserstein distance and earth mover'sdistance, have been shown to be an effective tool in signal and image analysis. The success of transport based distances is in part due to their Lagrangian nature which allows it to capture the important variations in many signal classes. However these distances require the signal to be nonnegative and normalized. Furthermore, the signals are considered as measures and compared by redistributing (transporting) them, which does not directly take into account the signal intensity. Here we study a transport-based distance, called the TLp distance, that combines Lagrangian and intensity modelling and is directly applicable to general, non-positive and multi-channelled signals. The distance can be computed by existing numerical methods. We give an overview of the basic properties of this distance and applications to classification, with multi-channelled non-positive one-dimensional signals and two-dimensional images, and color transfer.

Optimal Mass Transport: Signal processing and machine-learning applications

Transport-based techniques for signal and data analysis have received increased attention recently. Given their ability to provide accurate generative models for signal intensities and other data distributions, they have been used in a variety of applications including content-based retrieval, cancer detection, image super-resolution, and statistical machine learning, to name a few, and shown to produce state of the art results in several applications. Moreover, the geometric characteristics of transport-related metrics have inspired new kinds of algorithms for interpreting the meaning of data distributions. Here we provide a practical overview of the mathematical underpinnings of mass transport-related methods, including numerical implementation, as well as a review, with demonstrations, of several applications. Software accompanying this tutorial is available at [43].

Predictive Nuclear Chromatin Characteristics of Melanoma and Dysplastic Nevi

Background:

The diagnosis of malignant melanoma (MM) is among the diagnostic challenges pathologists encounter on a routine basis. Melanoma may arise in patients with preexisting dysplastic nevi (DN) and it is still the cause of 1.7% of all cancer-related deaths. Melanomas often have overlapping histological features with DN, especially those with severe dysplasia. Nucleotyping for identifying nuclear textural features can analyze nuclear DNA structure and organization. The aim of this study is to differentiate MM and DN using these methodologies.

Methods:

Dermatopathology slides diagnosed as MM and DN were retrieved. The glass slides were scanned using an Aperio ScanScopeXT at ×40 (0.25 μ/pixel). Whole slide images (WSI) were annotated for nuclei selection. Nuclear features to distinguish between MM and DN based on chromatin distributions were extracted from the WSI. The morphological characteristics for each nucleus were quantified with the optimal transport-based linear embedding in the continuous domain. Label predictions for individual cell nucleus are achieved through a modified version of linear discriminant analysis, coupled with the k-nearest neighbor classifier. Label for an unknown patient was set by the voting strategy with its pertaining cell nuclei.

Results:

Nucleotyping of 139 patient cases of melanoma (n = 67) and DN (n = 72) showed that our method had superior classification accuracy of 81.29%. This is a 6.4% gain in differentiating MM and DN, compared with numerical feature-based method. The distribution differences in nuclei morphology between MM and DN can be visualized with biological interpretation.

Conclusions:

Nucleotyping using quantitative and qualitative analyses may provide enough information for differentiating MM from DN using pixel image data. Our method to segment cell nuclei may offer a practical and inexpensive solution in aiding in the accurate diagnosis of melanoma.

A model of the spatial tumour heterogeneity in colorectal adenocarcinoma tissue

Background

There have been great advancements in the field of digital pathology. The surge in development of analytical methods for such data makes it crucial to develop benchmark synthetic datasets for objectively validating and comparing these methods. In addition, developing a spatial model of the tumour microenvironment can aid our understanding of the underpinning laws of tumour heterogeneity.

Results

We propose a model of the healthy and cancerous colonic crypt microenvironment. Our model is designed to generate synthetic histology image data with parameters that allow control over cancer grade, cellularity, cell overlap ratio, image resolution, and objective level.

Conclusions

To the best of our knowledge, ours is the first model to simulate histology image data at sub-cellular level for healthy and cancerous colon tissue, where the cells have different compartments and are organised to mimic the microenvironment of tissue in situ rather than dispersed cells in a cultured environment. Qualitative and quantitative validation has been performed on the model results demonstrating good similarity to the real data. The simulated data could be used to validate techniques such as image restoration, cell and crypt segmentation, and cancer grading.

Electronic supplementary material

The online version of this article (doi:10.1186/s12859-016-1126-2) contains supplementary material, which is available to authorized users.

Quantitative non-invasive cell characterisation and discrimination based on multispectral autofluorescence features

Automated and unbiased methods of non-invasive cell monitoring able to deal with complex biological heterogeneity are fundamentally important for biology and medicine. Label-free cell imaging provides information about endogenous autofluorescent metabolites, enzymes and cofactors in cells. However extracting high content information from autofluorescence imaging has been hitherto impossible. Here, we quantitatively characterise cell populations in different tissue types, live or fixed, by using novel image processing and a simple multispectral upgrade of a wide-field fluorescence microscope. Our optimal discrimination approach enables statistical hypothesis testing and intuitive visualisations where previously undetectable differences become clearly apparent. Label-free classifications are validated by the analysis of Classification Determinant (CD) antigen expression. The versatility of our method is illustrated by detecting genetic mutations in cancer, non-invasive monitoring of CD90 expression, label-free tracking of stem cell differentiation, identifying stem cell subpopulations with varying functional characteristics, tissue diagnostics in diabetes, and assessing the condition of preimplantation embryos.

A continuous linear optimal transport approach for pattern analysis in image datasets

We present a new approach to facilitate the application of the optimal transport metric to pattern recognition on image databases. The method is based on a linearized version of the optimal transport metric, which provides a linear embedding for the images. Hence, it enables shape and appearance modeling using linear geometric analysis techniques in the embedded space. In contrast to previous work, we use Monge's formulation of the optimal transport problem, which allows for reasonably fast computation of the linearized optimal transport embedding for large images. We demonstrate the application of the method to recover and visualize meaningful variations in a supervised-learning setting on several image datasets, including chromatin distribution in the nuclei of cells, galaxy morphologies, facial expressions, and bird species identification. We show that the new approach allows for high-resolution construction of modes of variations and discrimination and can enhance classification accuracy in a variety of image discrimination problems.

The Radon Cumulative Distribution Transform and Its Application to Image Classification

Invertible image representation methods (transforms) are routinely employed as low-level image processing operations based on which feature extraction and recognition algorithms are developed. Most transforms in current use (e.g., Fourier, wavelet, and so on) are linear transforms and, by themselves, are unable to substantially simplify the representation of image classes for classification. Here, we describe a nonlinear, invertible, low-level image processing transform based on combining the well-known Radon transform for image data, and the 1D cumulative distribution transform proposed earlier. We describe a few of the properties of this new transform, and with both theoretical and experimental results show that it can often render certain problems linearly separable in a transform space.

Lifetime Distributions from Tracking Individual BC3H1 Cells Subjected to Yessotoxin

This work shows examples of lifetime distributions for individual BC3H1 cells after start of exposure to the marine toxin yessotoxin (YTX) in an experimental dish. The present tracking of many single cells from time-lapse microscopy data demonstrates the complexity in individual cell fate and which can be masked in aggregate properties. This contribution also demonstrates the general practicality of cell tracking. It can serve as a conceptually simple and non-intrusive method for high throughput early analysis of cytotoxic effects to assess early and late time points relevant for further analyzes or to assess for variability and sub-populations of interest. The present examples of lifetime distributions seem partly to reflect different cell death modalities. Differences between cell lifetime distributions derived from populations in different experimental dishes can potentially provide measures of inter-cellular influence. Such outcomes may help to understand tumor-cell resistance to drug therapy and to predict the probability of metastasis.

Detection of malignant mesothelioma using nuclear structure of mesothelial cells in effusion cytology specimens

Mesothelioma is a form of cancer generally caused from previous exposure to asbestos. Although it was considered a rare neoplasm in the past, its incidence is increasing worldwide due to extensive use of asbestos. In the current practice of medicine, the gold standard for diagnosing mesothelioma is through a pleural biopsy with subsequent histologic examination of the tissue. The diagnostic tissue should demonstrate the invasion by the tumor and is obtained through thoracoscopy or open thoracotomy, both being highly invasive surgical operations. On the other hand, thoracocentesis, which is removal of effusion fluid from the pleural space, is a far less invasive procedure that can provide material for cytological examination. In this study, we aim at detecting and classifying malignant mesothelioma based on the nuclear chromatin distribution from digital images of mesothelial cells in effusion cytology specimens. Accordingly, a computerized method is developed to determine whether a set of nuclei belonging to a patient is benign or malignant. The quantification of chromatin distribution is performed by using the optimal transport-based linear embedding for segmented nuclei in combination with the modified Fisher discriminant analysis. Classification is then performed through a k-nearest neighborhood approach and a basic voting strategy. Our experiments on 34 different human cases result in 100% accurate predictions computed with blind cross validation. Experimental comparisons also show that the new method can significantly outperform standard numerical feature-type methods in terms of agreement with the clinical diagnosis gold standard. According to our results, we conclude that nuclear structure of mesothelial cells alone may contain enough information to separate malignant mesothelioma from benign mesothelial proliferations.

Carnegie Mellon University bioimaging day 2014: Challenges and opportunities in digital pathology

Recent advances in digital imaging is impacting the practice of pathology. One of the key enabling technologies that is leading the way towards this transformation is the use of whole slide imaging (WSI) which allows glass slides to be converted into large image files that can be shared, stored, and analyzed rapidly. Many applications around this novel technology have evolved in the last decade including education, research and clinical applications. This publication highlights a collection of abstracts, each corresponding to a talk given at Carnegie Mellon University's (CMU) Bioimaging Day 2014 co-sponsored by the Biomedical Engineering and Lane Center for Computational Biology Departments at CMU. Topics related specifically to digital pathology are presented in this collection of abstracts. These include topics related to digital workflow implementation, imaging and artifacts, storage demands, and automated image analysis algorithms.