Applying watershed algorithms to the segmentation of clustered nuclei

CellViT: Vision Transformers for precise cell segmentation and classification

Nuclei detection and segmentation in hematoxylin and eosin-stained (H&E) tissue images are important clinical tasks and crucial for a wide range of applications. However, it is a challenging task due to nuclei variances in staining and size, overlapping boundaries, and nuclei clustering. While convolutional neural networks have been extensively used for this task, we explore the potential of Transformer-based networks in combination with large scale pre-training in this domain. Therefore, we introduce a new method for automated instance segmentation of cell nuclei in digitized tissue samples using a deep learning architecture based on Vision Transformer called CellViT. CellViT is trained and evaluated on the PanNuke dataset, which is one of the most challenging nuclei instance segmentation datasets, consisting of nearly 200,000 annotated nuclei into 5 clinically important classes in 19 tissue types. We demonstrate the superiority of large-scale in-domain and out-of-domain pre-trained Vision Transformers by leveraging the recently published Segment Anything Model and a ViT-encoder pre-trained on 104 million histological image patches - achieving state-of-the-art nuclei detection and instance segmentation performance on the PanNuke dataset with a mean panoptic quality of 0.50 and an F1-detection score of 0.83. The code is publicly available at https://github.com/TIO-IKIM/CellViT.

BetaBuddy: An automated end-to-end computer vision pipeline for analysis of calcium fluorescence dynamics in β-cells

Insulin secretion from pancreatic β-cells is integral in maintaining the delicate equilibrium of blood glucose levels. Calcium is known to be a key regulator and triggers the release of insulin. This sub-cellular process can be monitored and tracked through live-cell imaging and subsequent cell segmentation, registration, tracking, and analysis of the calcium level in each cell. Current methods of analysis typically require the manual outlining of β-cells, involve multiple software packages, and necessitate multiple researchers-all of which tend to introduce biases. Utilizing deep learning algorithms, we have therefore created a pipeline to automatically segment and track thousands of cells, which greatly reduces the time required to gather and analyze a large number of sub-cellular images and improve accuracy. Tracking cells over a time-series image stack also allows researchers to isolate specific calcium spiking patterns and spatially identify those of interest, creating an efficient and user-friendly analysis tool. Using our automated pipeline, a previous dataset used to evaluate changes in calcium spiking activity in β-cells post-electric field stimulation was reanalyzed. Changes in spiking activity were found to be underestimated previously with manual segmentation. Moreover, the machine learning pipeline provides a powerful and rapid computational approach to examine, for example, how calcium signaling is regulated by intracellular interactions.

Green synthesis of silver and iron oxide nanoparticles mediated photothermal effects on Blastocystis hominis

The evolution of parasite resistance to antiparasitic agents has become a serious health issue indicating a critical and pressing need to develop new therapeutics that can conquer drug resistance. Nanoparticles are novel, promising emerging drug carriers that have demonstrated efficiency in treating many parasitic diseases. Lately, attention has been drawn to a broad-spectrum nanoparticle capable of converting absorbed light into heat via the photothermal effect phenomenon. The present study is the first to assess the effect of silver nanoparticles (Ag NPs) and iron oxide nanoparticles (Fe3O4 NPs) as sole agents and with the combined action of the light-emitting diode (LED) on Blastocystis hominins (B. hominis) in vitro. Initially, the aqueous synthesized nanoparticles were characterized by UV-Vis spectroscopy, zeta potential, and transmission electron microscopy (TEM). The anti-blastocyst efficiency of these NPs was tested separately in dark conditions. As these NPs have a wide absorption spectrum in the visible regions, they were also excited by a continuous wave LED of wavelength band (400-700 nm) to test the photothermal effect. The sensitivity of B. hominis cysts was evaluated using scanning laser confocal microscopy whereas the live and dead cells were accurately segmented based on superpixels and the k-mean clustering algorithm. Our findings showed that this excitation led to hyperthermia that induced a significant reduction in the number of cysts treated with photothermally active NPs. The results of this study elucidate the potential role of photothermally active NPs as an effective anti-blastocystis agent. By using this approach, new therapeutic antiparasitic agents can be developed.

A Novel Approach for Identifying Nanoplastics by Assessing Deformation Behavior with Scanning Electron Microscopy

As plastic production continues to increase globally, plastic waste accumulates and degrades into smaller plastic particles. Through chemical and biological processes, nanoscale plastic particles (nanoplastics) are formed and are expected to exist in quantities of several orders of magnitude greater than those found for microplastics. Due to their small size and low mass, nanoplastics remain challenging to detect in the environment using most standard analytical methods. The goal of this research is to adapt existing tools to address the analytical challenges posed by the identification of nanoplastics. Given the unique and well-documented properties of anthropogenic plastics, we hypothesized that nanoplastics could be differentiated by polymer type using spatiotemporal deformation data collected through irradiation with scanning electron microscopy (SEM). We selected polyvinyl chloride (PVC), polyethylene terephthalate (PET), and high-density polyethylene (HDPE) to capture a range of thermodynamic properties and molecular structures encompassed by commercially available plastics. Pristine samples of each polymer type were chosen and individually milled to generate micro and nanoscale particles for SEM analysis. To test the hypothesis that polymers could be differentiated from other constituents in complex samples, the polymers were compared against proxy materials common in environmental media, i.e., algae, kaolinite clay, and nanocellulose. Samples for SEM analysis were prepared uncoated to enable observation of polymer deformation under set electron beam parameters. For each sample type, particles approximately 1 µm in diameter were chosen, and videos of particle deformation were recorded and studied. Blinded samples were also prepared with mixtures of the aforementioned materials to test the viability of this method for identifying near-nanoscale plastic particles in environmental media. Based on the evidence collected, deformation patterns between plastic particles and particles present in common environmental media show significant differences. A computer vision algorithm was also developed and tested against manual measurements to improve the usefulness and efficiency of this method further.

Image analysis-based tumor infiltrating lymphocytes measurement predicts breast cancer pathologic complete response in SWOG S0800 neoadjuvant chemotherapy trial

We assessed the predictive value of an image analysis-based tumor-infiltrating lymphocytes (TILs) score for pathologic complete response (pCR) and event-free survival in breast cancer (BC). About 113 pretreatment samples were analyzed from patients with stage IIB-IIIC HER-2-negative BC randomized to neoadjuvant chemotherapy ± bevacizumab. TILs quantification was performed on full sections using QuPath open-source software with a convolutional neural network cell classifier (CNN11). We used easTILs% as a digital metric of TILs score defined as [sum of lymphocytes area (mm2)/stromal area(mm2)] × 100. Pathologist-read stromal TILs score (sTILs%) was determined following published guidelines. Mean pretreatment easTILs% was significantly higher in cases with pCR compared to residual disease (median 36.1 vs.14.8%, p < 0.001). We observed a strong positive correlation (r = 0.606, p < 0.0001) between easTILs% and sTILs%. The area under the prediction curve (AUC) was higher for easTILs% than sTILs%, 0.709 and 0.627, respectively. Image analysis-based TILs quantification is predictive of pCR in BC and had better response discrimination than pathologist-read sTILs%.

BetaBuddy: An end-to-end computer vision pipeline for the automated analysis of insulin secreting β-cells

Insulin secretion from pancreatic β-cells is integral in maintaining the delicate equilibrium of blood glucose levels. Calcium is known to be a key regulator and triggers the release of insulin. This sub-cellular process can be monitored and tracked through live-cell imaging and subsequent cell segmentation, registration, tracking, and analysis of the calcium level in each cell. Current methods of analysis typically require the manual outlining of β-cells, involve multiple software packages, and necessitate multiple researchers - all of which tend to introduce biases. Utilizing deep learning algorithms, we have therefore created a pipeline to automatically segment and track thousands of cells, which greatly reduces the time required to gather and analyze a large number of sub-cellular images and improve accuracy. Tracking cells over a time-series image stack also allows researchers to isolate specific calcium spiking patterns and spatially identify those of interest, creating an efficient and user-friendly analysis tool. Using our automated pipeline, a previous dataset used to evaluate changes in calcium spiking activity in β-cells post-electric field stimulation was reanalyzed. Changes in spiking activity were found to be underestimated previously with manual segmentation. Moreover, the machine learning pipeline provides a powerful and rapid computational approach to examine, for example, how calcium signaling is regulated by intracellular interactions in a cluster of β-cells.

A pathomic approach for tumor-infiltrating lymphocytes classification on breast cancer digital pathology images

Background and objectives

The detection of tumor-infiltrating lymphocytes (TILs) could aid in the development of objective measures of the infiltration grade and can support decision-making in breast cancer (BC). However, manual quantification of TILs in BC histopathological whole slide images (WSI) is currently based on a visual assessment, thus resulting not standardized, not reproducible, and time-consuming for pathologists. In this work, a novel pathomic approach, aimed to apply high-throughput image feature extraction techniques to analyze the microscopic patterns in WSI, is proposed. In fact, pathomic features provide additional information concerning the underlying biological processes compared to the WSI visual interpretation, thus providing more easily interpretable and explainable results than the most frequently investigated Deep Learning based methods in the literature.

Methods

A dataset containing 1037 regions of interest with tissue compartments and TILs annotated on 195 TNBC and HER2+ BC hematoxylin and eosin (H&E)-stained WSI was used. After segmenting nuclei within tumor-associated stroma using a watershed-based approach, 71 pathomic features were extracted from each nucleus and reduced using a Spearman's correlation filter followed by a nonparametric Wilcoxon rank-sum test and least absolute shrinkage and selection operator. The relevant features were used to classify each candidate nucleus as either TILs or non-TILs using 5 multivariable machine learning classification models trained using 5-fold cross-validation (1) without resampling, (2) with the synthetic minority over-sampling technique and (3) with downsampling. The prediction performance of the models was assessed using ROC curves.

Results

21 features were selected, with most of them related to the well-known TILs properties of having regular shape, clearer margins, high peak intensity, more homogeneous enhancement and different textural pattern than other cells. The best performance was obtained by Random-Forest with ROC AUC of 0.86, regardless of resampling technique.

Conclusions

The presented approach holds promise for the classification of TILs in BC H&E-stained WSI and could provide support to pathologists for a reliable, rapid and interpretable clinical assessment of TILs in BC.

Segmentation of Multiple Nuclei from Non-overlapping Immuno-histochemically Stained Histological Hepatic Images

In this paper, we describe an algorithm for accurately segmenting multiple nudfclei from clumps of non-overlapping immuno-histochemically stained histological hepatic (liver) images. This problem is notoriously difficult because of the degree of presence of stains among the multi-nucleated cells, the poor contrast of cell cytoplasm, and the presence of mucus, blood, and inflammatory cells in the images. Hepatocellular carcinoma, characterized by cellular and nuclear enlargement, nuclear pleomorphism, and multi-nucleation, poses a prominent threat. Our proposed method addresses the aforementioned issues for an automated diagnosis system by judging the presence of multiple nuclei in a two-step process: the Quickhull algorithm defines the convex hull of each cell in the image and candidate nuclei regions are located with morphological operations. A combination of features containing local minima and shape-dependent features is extracted for the detection of single or multiple nuclei in each cell with a significant reduction in the number of false positives and false negatives providing an accuracy of 89.76%.

CPP-Net: Context-aware Polygon Proposal Network for Nucleus Segmentation

Nucleus segmentation is a challenging task due to the crowded distribution and blurry boundaries of nuclei. To differentiate between touching and overlapping nuclei, recent approaches have represented nuclei in the form of polygons, and have accordingly achieved promising performance. Each polygon is represented by a set of centroid-to-boundary distances, which are in turn predicted by features of the centroid pixel for a single nucleus. However, the use of the centroid pixel alone does not provide sufficient contextual information for robust prediction and therefore affects the segmentation accuracy. To address this problem, we propose a Context-aware Polygon Proposal Network (CPP-Net) for nucleus segmentation. First, we sample a point set rather than a single pixel within each cell for distance prediction; this strategy substantially enhances the contextual information and thereby improves the prediction robustness. Second, we propose a Confidence-basedWeighting Module, which adaptively fuses the predictions from the sampled point set. Third, we introduce a novel Shape-Aware Perceptual (SAP) loss that constrains the shape of the predicted polygons. This SAP loss is based on an additional network that is pre-trained by means of mapping the centroid probability map and the pixel-to-boundary distance maps to a different nucleus representation. Extensive experiments demonstrate the effectiveness of each component in the proposed CPP-Net. Finally, CPP-Net is found to achieve state-of-the-art performance on three publicly available databases, namely DSB2018, BBBC06, and PanNuke. The code of this paper will be released.

A Novel Approach for the Manufacturing of Gelatin-Methacryloyl

Gelatin and its derivatives contain cell adhesion moieties as well as sites that enable proteolytic degradation, thus allowing cellular proliferation and migration. The processing of gelatin to its derivatives and/or gelatin-containing products is challenged by its gelation below 30 ∘C. In this study, a novel strategy was developed for the dissolution and subsequent modification of gelatin to its derivative gelatin-methacryloyl (GelMA). This approach was based on the presence of urea in the buffer media, which enabled the processing at room temperature, i.e., lower than the sol-gel transition point of the gelatin solutions. The degree of functionalization was controlled by the ratio of reactant volume to the gelatin concentration. Hydrogels with tailored mechanical properties were produced by variations of the GelMA concentration and its degree of functionalization. Moreover, the biocompatibility of hydrogels was assessed and compared to hydrogels formulated with GelMA produced by the conventional method. NIH 3T3 fibroblasts were seeded onto hydrogels and the viability showed no difference from the control after a three-day incubation period.

Adaptive tracking algorithm for trajectory analysis of cells and layer-by-layer assessment of motility dynamics

Tracking biological objects such as cells or subcellular components imaged with time-lapse microscopy enables us to understand the molecular principles about the dynamics of cell behaviors. However, automatic object detection, segmentation and extracting trajectories remain as a rate-limiting step due to intrinsic challenges of video processing. This paper presents an adaptive tracking algorithm (Adtari) that automatically finds the optimum search radius and cell linkages to determine trajectories in consecutive frames. A critical assumption in most tracking studies is that displacement remains unchanged throughout the movie and cells in a few frames are usually analyzed to determine its magnitude. Tracking errors and inaccurate association of cells may occur if the user does not correctly evaluate the value or prior knowledge is not present on cell movement. The key novelty of our method is that minimum intercellular distance and maximum displacement of cells between frames are dynamically computed and used to determine the threshold distance. Since the space between cells is highly variable in a given frame, our software recursively alters the magnitude to determine all plausible matches in the trajectory analysis. Our method therefore eliminates a major preprocessing step where a constant distance was used to determine the neighbor cells in tracking methods. Cells having multiple overlaps and splitting events were further evaluated by using the shape attributes including perimeter, area, ellipticity and distance. The features were applied to determine the closest matches by minimizing the difference in their magnitudes. Finally, reporting section of our software were used to generate instant maps by overlaying cell features and trajectories. Adtari was validated by using videos with variable signal-to-noise, contrast ratio and cell density. We compared the adaptive tracking with constant distance and other methods to evaluate performance and its efficiency. Our algorithm yields reduced mismatch ratio, increased ratio of whole cell track, higher frame tracking efficiency and allows layer-by-layer assessment of motility to characterize single-cells. Adaptive tracking provides a reliable, accurate, time efficient and user-friendly open source software that is well suited for analysis of 2D fluorescence microscopy video datasets.

The relationship between radiomics and pathomics in Glioblastoma patients: Preliminary results from a cross-scale association study

Glioblastoma multiforme (GBM) typically exhibits substantial intratumoral heterogeneity at both microscopic and radiological resolution scales. Diffusion Weighted Imaging (DWI) and dynamic contrast-enhanced (DCE) magnetic resonance imaging (MRI) are two functional MRI techniques that are commonly employed in clinic for the assessment of GBM tumor characteristics. This work presents initial results aiming at determining if radiomics features extracted from preoperative ADC maps and post-contrast T1 (T1C) images are associated with pathomic features arising from H&E digitized pathology images. 48 patients from the public available CPTAC-GBM database, for which both radiology and pathology images were available, were involved in the study. 91 radiomics features were extracted from ADC maps and post-contrast T1 images using PyRadiomics. 65 pathomic features were extracted from cell detection measurements from H&E images. Moreover, 91 features were extracted from cell density maps of H&E images at four different resolutions. Radiopathomic associations were evaluated by means of Spearman's correlation (ρ) and factor analysis. p values were adjusted for multiple correlations by using a false discovery rate adjustment. Significant cross-scale associations were identified between pathomics and ADC, both considering features (n = 186, 0.45 < ρ < 0.74 in absolute value) and factors (n = 5, 0.48 < ρ < 0.54 in absolute value). Significant but fewer ρ values were found concerning the association between pathomics and radiomics features (n = 53, 0.5 < ρ < 0.65 in absolute value) and factors (n = 2, ρ = 0.63 and ρ = 0.53 in absolute value). The results of this study suggest that cross-scale associations may exist between digital pathology and ADC and T1C imaging. This can be useful not only to improve the knowledge concerning GBM intratumoral heterogeneity, but also to strengthen the role of radiomics approach and its validation in clinical practice as "virtual biopsy", introducing new insights for omics integration toward a personalized medicine approach.

Systematically higher Ki67 scores on core biopsy samples compared to corresponding resection specimen in breast cancer: a multi-operator and multi-institutional study

Ki67 has potential clinical importance in breast cancer but has yet to see broad acceptance due to inter-laboratory variability. Here we tested an open source and calibrated automated digital image analysis (DIA) platform to: (i) investigate the comparability of Ki67 measurement across corresponding core biopsy and resection specimen cases, and (ii) assess section to section differences in Ki67 scoring. Two sets of 60 previously stained slides containing 30 core-cut biopsy and 30 corresponding resection specimens from 30 estrogen receptor-positive breast cancer patients were sent to 17 participating labs for automated assessment of average Ki67 expression. The blocks were centrally cut and immunohistochemically (IHC) stained for Ki67 (MIB-1 antibody). The QuPath platform was used to evaluate tumoral Ki67 expression. Calibration of the DIA method was performed as in published studies. A guideline for building an automated Ki67 scoring algorithm was sent to participating labs. Very high correlation and no systematic error (p = 0.08) was found between consecutive Ki67 IHC sections. Ki67 scores were higher for core biopsy slides compared to paired whole sections from resections (p ≤ 0.001; median difference: 5.31%). The systematic discrepancy between core biopsy and corresponding whole sections was likely due to pre-analytical factors (tissue handling, fixation). Therefore, Ki67 IHC should be tested on core biopsy samples to best reflect the biological status of the tumor.

Guided interactive image segmentation using machine learning and color-based image set clustering

MOTIVATION

Over the last decades, image processing and analysis have become one of the key technologies in systems biology and medicine. The quantification of anatomical structures and dynamic processes in living systems is essential for understanding the complex underlying mechanisms and allows, i.e. the construction of spatio-temporal models that illuminate the interplay between architecture and function. Recently, deep learning significantly improved the performance of traditional image analysis in cases where imaging techniques provide large amounts of data. However, if only a few images are available or qualified annotations are expensive to produce, the applicability of deep learning is still limited.

RESULTS

We present a novel approach that combines machine learning-based interactive image segmentation using supervoxels with a clustering method for the automated identification of similarly colored images in large image sets which enables a guided reuse of interactively trained classifiers. Our approach solves the problem of deteriorated segmentation and quantification accuracy when reusing trained classifiers which is due to significant color variability prevalent and often unavoidable in biological and medical images. This increase in efficiency improves the suitability of interactive segmentation for larger image sets, enabling efficient quantification or the rapid generation of training data for deep learning with minimal effort. The presented methods are applicable for almost any image type and represent a useful tool for image analysis tasks in general.

AVAILABILITY AND IMPLEMENTATION

The presented methods are implemented in our image processing software TiQuant which is freely available at tiquant.hoehme.com.

SUPPLEMENTARY INFORMATION

Supplementary data are available at Bioinformatics online.

CancerCellTracker: a brightfield time-lapse microscopy framework for cancer drug sensitivity estimation

MOTIVATION

Time-lapse microscopy is a powerful technique that relies on images of live cells cultured ex vivo that are captured at regular intervals of time to describe and quantify their behavior under certain experimental conditions. This imaging method has great potential in advancing the field of precision oncology by quantifying the response of cancer cells to various therapies and identifying the most efficacious treatment for a given patient. Digital image processing algorithms developed so far require high-resolution images involving very few cells originating from homogeneous cell line populations. We propose a novel framework that tracks cancer cells to capture their behavior and quantify cell viability to inform clinical decisions in a high-throughput manner.

RESULTS

The brightfield microscopy images a large number of patient-derived cells in an ex vivo reconstruction of the tumor microenvironment treated with 31 drugs for up to 6 days. We developed a robust and user-friendly pipeline CancerCellTracker that detects cells in co-culture, tracks these cells across time and identifies cell death events using changes in cell attributes. We validated our computational pipeline by comparing the timing of cell death estimates by CancerCellTracker from brightfield images and a fluorescent channel featuring ethidium homodimer. We benchmarked our results using a state-of-the-art algorithm implemented in ImageJ and previously published in the literature. We highlighted CancerCellTracker's efficiency in estimating the percentage of live cells in the presence of bone marrow stromal cells.

AVAILABILITY AND IMPLEMENTATION

https://github.com/compbiolabucf/CancerCellTracker.

SUPPLEMENTARY INFORMATION

Supplementary data are available at Bioinformatics online.

Semantic segmentation of human cell nucleus using deep U-Net and other versions of U-Net models

The deep learning models play an essential role in many areas, including medical image analysis. These models extract important features without human intervention. In this paper, we propose a deep convolution neural network, named as deep U-Net model, for the segmentation of the cell nucleus, a critical functional unit that determines the function and structure of the body. The nucleus contains all kinds of DNA, RNA, chromosomes, and genes governing all life activities, and its disorder may lead to different types of diseases such as cancer, heart disease, diabetes, Alzheimer's, etc. If the nucleus structure is known correctly, diseases due to nucleus disorder may be detected early. It may also reduce the drug discovery time if the shape and size of the nucleus are known. We evaluate the performance of the proposed models on the nucleus segmentation data set used by the Data Science Bowl 2018 competition hosted by Kaggle. We compare its performance with that of the U-Net, Attention U-Net, R2U-Net, Attention R2U-Net, and both versions of the U-Net++ with and without supervision, in terms of loss, dice coefficient, dice loss, intersection over union, and accuracy. Our model performs better than the existing models.

Semantic Image Segmentation Using Scant Pixel Annotations

The success of deep networks for the semantic segmentation of images is limited by the availability of annotated training data. The manual annotation of images for segmentation is a tedious and time-consuming task that often requires sophisticated users with significant domain expertise to create high-quality annotations over hundreds of images. In this paper, we propose the segmentation with scant pixel annotations (SSPA) approach to generate high-performing segmentation models using a scant set of expert annotated images. The models are generated by training them on images with automatically generated pseudo-labels along with a scant set of expert annotated images selected using an entropy-based algorithm. For each chosen image, experts are directed to assign labels to a particular group of pixels, while a set of replacement rules that leverage the patterns learned by the model is used to automatically assign labels to the remaining pixels. The SSPA approach integrates active learning and semi-supervised learning with pseudo-labels, where expert annotations are not essential but generated on demand. Extensive experiments on bio-medical and biofilm datasets show that the SSPA approach achieves state-of-the-art performance with less than 5% cumulative annotation of the pixels of the training data by the experts.

Long-term Effects of Riboflavin Ultraviolet-A-Induced CXL With Different Irradiances on the Biomechanics of In Vivo Rabbit Corneas

PURPOSE

To evaluate the long-term effects of ultraviolet-A corneal cross-linking (CXL) with different irrandiances on the biomechanical properties of rabbit corneas and the corresponding changes in stromal microstructure.

METHODS

The study involved the left eyes of 85 healthy white Japanese rabbits, randomly divided into five groups (n = 16 to 18 each). After removing the epithelium, the first four groups were exposed to riboflavin (0.22% concentration by volume) and ultraviolet-A (370 nm) at different CXL irradiations but with the same total dose (5.4 J/cm2). The four groups were defined as standard CXL (SCXL; 3 mW/cm2 for 30 minutes, n = 17), accelerated CXL1 (ACXL1; 9 mW/cm2 for 10 minutes, n = 16), accelerated CXL2 (ACXL2; 18 mW/cm2 for 5 minutes, n = 17), and accelerated CXL3 (ACXL3; 30 mW/cm2 for 3 minutes, n = 17). The control group (n = 18) was treated with riboflavin without ultraviolet-A exposure. Nine months after CXL, 10 corneas from each group were tested ex vivo under inflation, and the tangent modulus (Et) was estimated using an inverse analysis process. The remaining six to eight specimens in each group were examined by electron microscopy to determine the mean fibril diameter and interfibrillar spacing.

RESULTS

The SCXL and ACXL1 groups showed statistically significant differences in Et at all stresses (0.005, 0.010, and 0.015 MPa) analyzed compared to the control group (all P < .01), but the differences were non-significant in the ACXL3 group (P = 1.000, .785, and .679, respectively). For the ACXL2 group, there was no statistical difference in Et under the low stress of 0.005 MPa (P = .155), but the differences became significant at 0.010 and 0.015 MPa when compared with the control group (all P < .05).

CONCLUSIONS

CXL had a significant effect on corneal biomechanics in both standard and accelerated procedures. However, standard CXL was the most effective, and this effectiveness decreased gradually with increasing ultraviolet-A power intensity. [J Refract Surg. 2022;38(6):389-397.].

Automatic detection of head and neck squamous cell carcinoma on histologic slides using hyperspectral microscopic imaging

SIGNIFICANCE

Automatic, fast, and accurate identification of cancer on histologic slides has many applications in oncologic pathology.

AIM

The purpose of this study is to investigate hyperspectral imaging (HSI) for automatic detection of head and neck cancer nuclei in histologic slides, as well as cancer region identification based on nuclei detection.

APPROACH

A customized hyperspectral microscopic imaging system was developed and used to scan histologic slides from 20 patients with squamous cell carcinoma (SCC). Hyperspectral images and red, green, and blue (RGB) images of the histologic slides with the same field of view were obtained and registered. A principal component analysis-based nuclei segmentation method was developed to extract nuclei patches from the hyperspectral images and the coregistered RGB images. Spectra-based support vector machine and patch-based convolutional neural networks (CNNs) were implemented for nuclei classification. The CNNs were trained with RGB patches (RGB-CNN) and hyperspectral patches (HSI-CNN) of the segmented nuclei and the utility of the extra spectral information provided by HSI was evaluated. Furthermore, cancer region identification was implemented by image-wise classification based on the percentage of cancerous nuclei detected in each image.

RESULTS

RGB-CNN, which mainly used the spatial information of nuclei, resulted in a 0.81 validation accuracy and 0.74 testing accuracy. HSI-CNN, which utilized the spatial and spectral features of the nuclei, showed significant improvement in classification performance and achieved 0.89 validation accuracy as well as 0.82 testing accuracy. Furthermore, the image-wise cancer region identification based on nuclei detection could generally improve the cancer detection rate.

CONCLUSIONS

We demonstrated that the morphological and spectral information contribute to SCC nuclei differentiation and that the spectral information within hyperspectral images could improve classification performance.

An Integrative Segmentation Framework for Cell Nucleus of Fluorescence Microscopy

Nucleus segmentation of fluorescence microscopy is a critical step in quantifying measurements in cell biology. Automatic and accurate nucleus segmentation has powerful applications in analyzing intrinsic characterization in nucleus morphology. However, existing methods have limited capacity to perform accurate segmentation in challenging samples, such as noisy images and clumped nuclei. In this paper, inspired by the idea of cascaded U-Net (or W-Net) and its remarkable performance improvement in medical image segmentation, we proposed a novel framework called Attention-enhanced Simplified W-Net (ASW-Net), in which a cascade-like structure with between-net connections was used. Results showed that this lightweight model could reach remarkable segmentation performance in the BBBC039 testing set (aggregated Jaccard index, 0.90). In addition, our proposed framework performed better than the state-of-the-art methods in terms of segmentation performance. Moreover, we further explored the effectiveness of our designed network by visualizing the deep features from the network. Notably, our proposed framework is open source.

Probabilistic spatial analysis in quantitative microscopy with uncertainty-aware cell detection using deep Bayesian regression

The investigation of biological systems with three-dimensional microscopy demands automatic cell identification methods that not only are accurate but also can imply the uncertainty in their predictions. The use of deep learning to regress density maps is a popular successful approach for extracting cell coordinates from local peaks in a postprocessing step, which then, however, hinders any meaningful probabilistic output. We propose a framework that can operate on large microscopy images and output probabilistic predictions (i) by integrating deep Bayesian learning for the regression of uncertainty-aware density maps, where peak detection algorithms generate cell proposals, and (ii) by learning a mapping from prediction proposals to a probabilistic space that accurately represents the chances of a successful prediction. Using these calibrated predictions, we propose a probabilistic spatial analysis with Monte Carlo sampling. We demonstrate this in a bone marrow dataset, where our proposed methods reveal spatial patterns that are otherwise undetectable.

Detection of tumour infiltrating lymphocytes in CD3 and CD8 stained histopathological images using a two-phase deep CNN

BACKGROUND

Immuno-score, a prognostic measure for cancer, employed in determining tumor grade and type, is generated by counting the number of Tumour-Infiltrating Lymphocytes (TILs) in CD3 and CD8 stained histopathological tissue samples. Significant stain variations and heterogeneity in lymphocytes' spatial distribution and density make automated counting of TILs' a challenging task.

METHODS

This work addresses the aforementioned challenges by developing a pipeline "Two-Phase Deep Convolutional Neural Network based Lymphocyte Counter (TDC-LC)" to detect lymphocytes in CD3 and CD8 stained histology images. The proposed pipeline sequentially works by removing hard negative examples (artifacts) in the first phase using a custom CNN "LSATM-Net" that exploits the idea of a split, asymmetric transform, and merge. Whereas, in the second phase, instance segmentation is performed to detect and generate a lymphocyte count against the remaining samples. Furthermore, the effectiveness of the proposed pipeline is measured by comparing it with the state-of-the-art single- and two-stage detectors. The inference code is available at GitHub Repository https://github.com/m-mohsin-zafar/tdc-lc.

RESULTS

The empirical evaluation on samples from LYSTO dataset shows that the proposed LSTAM-Net can learn variations in the images and precisely remove the hard negative stain artifacts with an F-score of 0.74. The detection analysis shows that the proposed TDC-LC outperforms the existing models in identifying and counting lymphocytes with high Recall (0.87) and F-score (0.89). Moreover, the commendable performance of the proposed TDC-LC in different organs suggests a good generalization.

CONCLUSION

The promising performance of the proposed pipeline suggests that it can serve as an automated system for detecting and counting lymphocytes from patches of tissue samples thereby reducing the burden on pathologists.

Foveal blur-boosted segmentation of nuclei in histopathology images with shape prior knowledge and probability map constraints

MOTIVATION

In most tissue-based biomedical research, the lack of sufficient pathology training images with well-annotated ground truth inevitably limits the performance of deep learning systems. In this study, we propose a convolutional neural network with foveal blur enriching datasets with multiple local nuclei regions of interest derived from original pathology images. We further propose a human-knowledge boosted deep learning system by inclusion to the convolutional neural network new loss function terms capturing shape prior knowledge and imposing smoothness constraints on the predicted probability maps.

RESULTS

Our proposed system outperforms all state-of-the-art deep learning and non-deep learning methods by Jaccard coefficient, Dice coefficient, Accuracy and Panoptic Quality in three independent datasets. The high segmentation accuracy and execution speed suggest its promising potential for automating histopathology nuclei segmentation in biomedical research and clinical settings.

AVAILABILITY AND IMPLEMENTATION

The codes, the documentation and example data are available on an open source at: https://github.com/HongyiDuanmu26/FovealBoosted.

SUPPLEMENTARY INFORMATION

Supplementary data are available at Bioinformatics online.

Morphometric quantification of a pseudohyphae forming Saccharomyces cerevisiae strain using in situ microscopy and image analysis

Yeast morphology and counting are highly important in fermentation as they are often associated with productivity and can be influenced by process conditions. At present, time-consuming and offline methods are utilized for routine analysis of yeast morphology and cell counting using a haemocytometer. In this study, we demonstrate the application of an in situ microscope to obtain a fast stream of pseudohyphae images from agitated sample suspensions of a Saccharomyces cerevisiae strain, whose morphology in cell clusters is frequently found in the bioethanol fermentation industry. The large statistics of microscopic images allow for online determination of the principal morphological characteristics of the pseudohyphae, including the number of constituent cells, cell-size, number of branches, and length of branches. The distributions of these feature values are calculated online, constituting morphometric monitoring of the pseudohyphae population. By providing representative data, the proposed system can improve the effectiveness of morphological characterization, which in turn can help to improve the understanding and control of bioprocesses in which pseudohyphal-like morphologies are found.

DMNet: Dual-Stream Marker Guided Deep Network for Dense Cell Segmentation and Lineage Tracking

Accurate segmentation and tracking of cells in microscopy image sequences is extremely beneficial in clinical diagnostic applications and biomedical research. A continuing challenge is the segmentation of dense touching cells and deforming cells with indistinct boundaries, in low signal-to-noise-ratio images. In this paper, we present a dual-stream marker-guided network (DMNet) for segmentation of touching cells in microscopy videos of many cell types. DMNet uses an explicit cell marker-detection stream, with a separate mask-prediction stream using a distance map penalty function, which enables supervised training to focus attention on touching and nearby cells. For multi-object cell tracking we use M2Track tracking-by-detection approach with multi-step data association. Our M2Track with mask overlap includes short term track-to-cell association followed by track-to-track association to re-link tracklets with missing segmentation masks over a short sequence of frames. Our combined detection, segmentation and tracking algorithm has proven its potential on the IEEE ISBI 2021 6th Cell Tracking Challenge (CTC-6) where we achieved multiple top three rankings for diverse cell types. Our team name is MU-Ba-US, and the implementation of DMNet is available at, http://celltrackingchallenge.net/participants/MU-Ba-US/.

Quantitative assessment of inflammatory infiltrates in kidney transplant biopsies using multiplex tyramide signal amplification and deep learning

Delayed graft function (DGF) is a strong risk factor for development of interstitial fibrosis and tubular atrophy (IFTA) in kidney transplants. Quantitative assessment of inflammatory infiltrates in kidney biopsies of DGF patients can reveal predictive markers for IFTA development. In this study, we combined multiplex tyramide signal amplification (mTSA) and convolutional neural networks (CNNs) to assess the inflammatory microenvironment in kidney biopsies of DGF patients (n = 22) taken at 6 weeks post-transplantation. Patients were stratified for IFTA development (<10% versus ≥10%) from 6 weeks to 6 months post-transplantation, based on histopathological assessment by three kidney pathologists. One mTSA panel was developed for visualization of capillaries, T- and B-lymphocytes and macrophages and a second mTSA panel for T-helper cell and macrophage subsets. The slides were multi spectrally imaged and custom-made python scripts enabled conversion to artificial brightfield whole-slide images (WSI). We used an existing CNN for the detection of lymphocytes with cytoplasmatic staining patterns in immunohistochemistry and developed two new CNNs for the detection of macrophages and nuclear-stained lymphocytes. F1-scores were 0.77 (nuclear-stained lymphocytes), 0.81 (cytoplasmatic-stained lymphocytes), and 0.82 (macrophages) on a test set of artificial brightfield WSI. The CNNs were used to detect inflammatory cells, after which we assessed the peritubular capillary extent, cell density, cell ratios, and cell distance in the two patient groups. In this cohort, distance of macrophages to other immune cells and peritubular capillary extent did not vary significantly at 6 weeks post-transplantation between patient groups. CD163+ cell density was higher in patients with ≥10% IFTA development 6 months post-transplantation (p < 0.05). CD3+CD8-/CD3+CD8+ ratios were higher in patients with <10% IFTA development (p < 0.05). We observed a high correlation between CD163+ and CD4+GATA3+ cell density (R = 0.74, p < 0.001). Our study demonstrates that CNNs can be used to leverage reliable, quantitative results from mTSA-stained, multi spectrally imaged slides of kidney transplant biopsies.

An Open Source, Automated Tumor Infiltrating Lymphocyte Algorithm for Prognosis in Triple-Negative Breast Cancer

PURPOSE

Although tumor infiltrating lymphocytes (TIL) assessment has been acknowledged to have both prognostic and predictive importance in triple negative breast cancer (TNBC), it is subject to inter and intra-observer variability that has prevented widespread adoption. Here we constructed a machine-learning based breast cancer TIL scoring approach and validated its prognostic potential in multiple TNBC cohorts.

EXPERIMENTAL DESIGN

Using the QuPath open source software, we built a neural-network classifier for tumor cells, lymphocytes, fibroblasts and "other" cells on hematoxylin-eosin (H&E) stained sections. We analyzed the classifier-derived TIL measurements with five unique constructed TIL variables. A retrospective collection of 171 TNBC cases was used as the discovery set to identify the optimal association of machine-read TIL variables with patient outcome. For validation we evaluated a retrospective collection of 749 TNBC patients comprised of four independent validation subsets.

RESULTS

We found that all five machine TIL variables had significant prognostic association with outcomes (p{less than or equal to}0.01 for all comparisons) but showed cell specific variation in validation sets. Cox regression analysis demonstrated that all five TIL variables were independently associated with improved overall survival after adjusting for clinicopathological factors including stage, age and histological grade (p{less than or equal to}0.003 for all analyses).

CONCLUSIONS

Neural net driven cell classifier defined TIL variables were robust and independent prognostic factors in several independent validation cohorts of TNBC patients. These objective, open source TIL variables are freely available to download and can now be considered for testing in a prospective setting to assess clinical utility.

Prognostic role of serum thymidine kinase 1 kinetics during neoadjuvant chemotherapy for early breast cancer

Background

Emerging data support the use of thymidine kinase 1 (TK1) activity as a prognostic marker and for monitoring of response in breast cancer (BC). The long-term prognostic value of TK1 kinetics during neoadjuvant chemotherapy is unclear, which this study aimed to elucidate.

Methods

Material from patients enrolled to the single-arm prospective PROMIX trial of neoadjuvant epirubicin, docetaxel and bevacizumab for early BC was used. Ki67 in baseline biopsies was assessed both centrally and by automated digital imaging analysis. TK1 activity was measured from blood samples obtained at baseline and following two cycles of chemotherapy. The associations of TK1 and its kinetics as well as Ki67 with event-free survival and overall survival (OS) were evaluated using multivariable Cox regression models.

Results

Central Ki67 counting had excellent correlation with the results of digital image analysis (r = 0.814), but not with the diagnostic samples (r = 0.234), while it was independently prognostic for worse OS [adjusted hazard ratio (HR_adj) = 2.72, 95% confidence interval (CI) 1.19-6.21, P = 0.02]. Greater increase in TK1 activity after two cycles of chemotherapy resulted in improved event-free survival (HR_adj = 0.50, 95% CI 0.26-0.97, P = 0.04) and OS (HR_adj = 0.46, 95% CI 0.95, P = 0.04). There was significant interaction between the prognostic value of TK1 kinetics and Ki67 (p_interaction 0.04).

Conclusion

Serial measurement of serum TK1 activity during neoadjuvant chemotherapy provides long-term prognostic information in BC patients. The ease of obtaining serial samples for TK1 assessment motivates further evaluation in larger studies.

•

This is a correlative analysis of a prospective phase II study on neoadjuvant chemotherapy for breast cancer.

•

Serial measurement of serum TK1 activity during treatment provides independent long-term prognostic information.

•

We demonstrate the validity and clinical utility of both central and automated image analysis-based Ki67 assessment.

•

Finally, we explore the biologic correlations between TK1 and Ki67.

Automated mesenchymal stem cell segmentation and machine learning-based phenotype classification using morphometric and textural analysis

Purpose: Mesenchymal stem cells (MSCs) have demonstrated clinically relevant therapeutic effects for treatment of trauma and chronic diseases. The proliferative potential, immunomodulatory characteristics, and multipotentiality of MSCs in monolayer culture is reflected by their morphological phenotype. Standard techniques to evaluate culture viability are subjective, destructive, or time-consuming. We present an image analysis approach to objectively determine morphological phenotype of MSCs for prediction of culture efficacy. Approach: The algorithm was trained using phase-contrast micrographs acquired during the early and mid-logarithmic stages of MSC expansion. Cell regions are localized using edge detection, thresholding, and morphological operations, followed by cell marker identification using H-minima transform within each region to differentiate individual cells from cell clusters. Clusters are segmented using marker-controlled watershed to obtain single cells. Morphometric and textural features are extracted to classify cells based on phenotype using machine learning. Results: Algorithm performance was validated using an independent test dataset of 186 MSCs in 36 culture images. Results show 88% sensitivity and 86% precision for overall cell detection and a mean Sorensen-Dice coefficient of 0.849 ± 0.106 for segmentation per image. The algorithm exhibited an area under the curve of 0.816 (CI 95 = 0.769 to 0.886) and 0.787 (CI 95 = 0.716 to 0.851) for classifying MSCs according to their phenotype at early and mid-logarithmic expansion, respectively. Conclusions: The proposed method shows potential to segment and classify low and moderately dense MSCs based on phenotype with high accuracy and robustness. It enables quantifiable and consistent morphology-based quality assessment for various culture protocols to facilitate cytotherapy development.

PAExM: label-free hyper-resolution photoacoustic expansion microscopy

Reflection-mode ultraviolet photoacoustic microscopy (UV-PAM) is capable of imaging cell nuclei in thick tissue without complex preparation procedures, but it is challenging to distinguish adjacent nuclei due to the limited spatial resolution. Tissue expansion technology has recently been developed to exceed the diffraction-limited fluorescence microscopies, but it is accompanied by limitations including additional staining. Herein, photoacoustic expansion microscopy (PAExM) is presented, which is an advanced histologic imaging strategy combining advantages of fast label-free reflection-mode UV-PAM and the tissue expansion technology. Clustered cell nuclei in an enlarged volume of a mouse brain section can be visually resolved without staining, demonstrating a great potential of the system to be widely used for histologic applications throughout biomedical fields.

Automatic Human Dendritic Cells Segmentation Using K-Means Clustering and Chan-Vese Active Contour Model

BACKGROUND AND OBJECTIVE

Nowadays, the number of pathologies related to food are multiplied. Mycotoxins are one of the most severe food contaminants that cause serious effects on the human health. Therefore, it is necessary to develop an assessment tool for evaluating their impact on the immune response. Recently, a new investigational method using human dendritic cells was endorsed by biologists. Nevertheless, analysis of the morphological features and the behavior of these cells remains merely visual. In addition, this manual analysis is difficult and time-consuming. Here, we focus mainly on automating the evaluation process by using advanced image processing technology.

METHODS

An automatic segmentation approach of microscopic dendritic cell images is developed to provide a fast and objective evaluation. First, a combination of K-means clustering and mathematical morphology is used to detect dendritic cells. Second, a region-based Chan-Vese active contour model is used to segment the detected cells more precisely. Finally, dendritic cells are extracted by a filtering based on eccentricity measure.

RESULTS

The proposed scheme is tested on an actual dataset containing 421 microscopic dendritic cell images. The experimental results show high conformity between the results of the proposed scheme and ground-truth elaborated by biological expert. Moreover, a comparative study with other state-of-art segmentation schemes demonstrates the efficiency of the proposed method. It gives the highest average accuracy rate (99.42 %) compared to recent studied approaches.

CONCLUSIONS

The proposed image segmentation method for morphological analysis of dendrite inhibition can consistently be used as an assessment tool for biologists to facilitate the evaluation of serious health impacts of mycotoxins.

NuSeT: A deep learning tool for reliably separating and analyzing crowded cells

Segmenting cell nuclei within microscopy images is a ubiquitous task in biological research and clinical applications. Unfortunately, segmenting low-contrast overlapping objects that may be tightly packed is a major bottleneck in standard deep learning-based models. We report a Nuclear Segmentation Tool (NuSeT) based on deep learning that accurately segments nuclei across multiple types of fluorescence imaging data. Using a hybrid network consisting of U-Net and Region Proposal Networks (RPN), followed by a watershed step, we have achieved superior performance in detecting and delineating nuclear boundaries in 2D and 3D images of varying complexities. By using foreground normalization and additional training on synthetic images containing non-cellular artifacts, NuSeT improves nuclear detection and reduces false positives. NuSeT addresses common challenges in nuclear segmentation such as variability in nuclear signal and shape, limited training sample size, and sample preparation artifacts. Compared to other segmentation models, NuSeT consistently fares better in generating accurate segmentation masks and assigning boundaries for touching nuclei.

AI in Medical Imaging Informatics: Current Challenges and Future Directions

This paper reviews state-of-the-art research solutions across the spectrum of medical imaging informatics, discusses clinical translation, and provides future directions for advancing clinical practice. More specifically, it summarizes advances in medical imaging acquisition technologies for different modalities, highlighting the necessity for efficient medical data management strategies in the context of AI in big healthcare data analytics. It then provides a synopsis of contemporary and emerging algorithmic methods for disease classification and organ/ tissue segmentation, focusing on AI and deep learning architectures that have already become the de facto approach. The clinical benefits of in-silico modelling advances linked with evolving 3D reconstruction and visualization applications are further documented. Concluding, integrative analytics approaches driven by associate research branches highlighted in this study promise to revolutionize imaging informatics as known today across the healthcare continuum for both radiology and digital pathology applications. The latter, is projected to enable informed, more accurate diagnosis, timely prognosis, and effective treatment planning, underpinning precision medicine.

Prognostic potential of automated Ki67 evaluation in breast cancer: different hot spot definitions versus true global score

PURPOSE

The proliferation-associated biomarker Ki67 has potential utility in breast cancer, including aiding decisions based on prognosis, but has unacceptable inter- and intralaboratory variability. The aim of this study was to compare the prognostic potential for Ki67 hot spot scoring and global scoring using different digital image analysis (DIA) platforms.

METHODS

An ER+/HER2- breast cancer cohort (n = 139) with whole slide images of sequential sections stained for hematoxylin-eosin, pancytokeratin and Ki67, was analyzed using two DIA platforms. For hot spot analysis virtual dual staining was applied, aligning pancytokeratin and Ki67 images and 22 hot spot algorithms with different features were designed. For global Ki67 scoring an automated QuPath algorithm was applied on Ki67-stained whole slide images. Clinicopathological data included overall survival (OS) and recurrence-free survival (RFS) along with PAM50 molecular subtypes.

RESULTS

We show significant variations in Ki67 hot spot scoring depending on number of included tumor cells, hot spot size, shape and location. The higher the number of scored tumor cells, the higher the reproducibility of Ki67 proliferation values. Hot spot scoring had greater prognostic potential for RFS in high versus low Ki67 subgroups (hazard ratio (HR) 6.88, CI 2.07-22.87, p = 0.002), compared to global scoring (HR 3.13, CI 1.41-6.96, p = 0.005). Regarding OS, global scoring (HR 7.46, CI 2.46-22.58, p < 0.001) was slightly better than hot spot scoring (HR 6.93, CI 1.61-29.91, p = 0.009). In adjusted multivariate analysis, only global scoring was an independent prognostic marker for both RFS and OS. In addition, global Ki67-based surrogate subtypes reached higher concordance with PAM50 molecular subtype for luminal A and B tumors (66.3% concordance rate, κ = 0.345), than using hot spot scoring (55.8% concordance rate, κ = 0.250).

CONCLUSIONS

We conclude that the automated global Ki67 scoring is feasible and shows clinical validity, which, however, needs to be confirmed in a larger cohort before clinical implementation.

Hyperspectral Microscopic Imaging for Automatic Detection of Head and Neck Squamous Cell Carcinoma Using Histologic Image and Machine Learning

The purpose of this study is to develop hyperspectral imaging (HSI) for automatic detection of head and neck cancer cells on histologic slides. A compact hyperspectral microscopic system is developed in this study. Histologic slides from 15 patients with squamous cell carcinoma (SCC) of the larynx and hypopharynx are imaged with the system. The proposed nuclei segmentation method based on principle component analysis (PCA) can extract most nuclei in the hyperspectral image without extracting other sub-cellular components. Both spectra-based support vector machine (SVM) and patch-based convolutional neural network (CNN) are used for nuclei classification. CNNs were trained with both hyperspectral images and pseudo RGB images of extracted nuclei, in order to evaluate the usefulness of extra information provided by hyperspectral imaging. The average accuracy of spectra-based SVM classification is 68%. The average AUC and average accuracy of the HSI patch-based CNN classification is 0.94 and 82.4%, respectively. The hyperspectral microscopic imaging and classification methods provide an automatic tool to aid pathologists in detecting SCC on histologic slides.

The Prediction of Fine Sediment Distribution in Gravel-Bed Rivers Using a Combination of DEM and FNN

Large amounts of fine sediment infiltration into void spaces of coarse bed material have the ability to alter the morphodynamics of rivers and their aquatic ecosystems. Modelling the mechanisms of fine sediment infiltration in gravel-bed is therefore of high significance. We proposed a framework for calculating the sediment exchange in two layers. On the basis of the conventional approaches, we derived a two-layer fine sediment sorting, which considers the transportation of fine sediment in the form of infiltration into the void spaces of the gravel-bed. The relationship between the fine sediment exchange and the affected factors was obtained by using the discrete element method (DEM) in combination with feedforward neural networks (FNN). The DEM model was validated and applied for gravel-bed flumes with different sizes of fine sediments. Further, we developed algorithms for extracting information in terms of gravel-bed packing, grain size distribution, and porosity variation. On the basis of the DEM results with this extracted information, we developed an FNN model for fine sediment sorting. Analyzing the calculated results and comparing them with the available measurements showed that our framework can successfully simulate the exchange of fine sediment in gravel-bed rivers.

Automated image analysis of stained cytospins to quantify Schwann cell purity and proliferation

In response to injury, adult Schwann cells (SCs) re-enter the cell cycle, change their expression profile, and exert repair functions important for wound healing and the re-growth of axons. While this phenotypical instability of SCs is essential for nerve regeneration, it has also been implicated in cancer progression and de-myelinating neuropathies. Thus, SCs became an important research tool to study the molecular mechanisms involved in repair and disease and to identify targets for therapeutic intervention. A high purity of isolated SC cultures used for experimentation must be demonstrated to exclude that novel findings are derived from a contaminating fibroblasts population. In addition, information about the SC proliferation status is an important parameter to be determined in response to different treatments. The evaluation of SC purity and proliferation, however, usually depends on the time consuming, manual assessment of immunofluorescence stainings or comes with the sacrifice of a large amount of SCs for flow cytometry analysis. We here show that rat SC culture derived cytospins stained for SC marker SOX10, proliferation marker EdU, intermediate filament vimentin and DAPI allowed the determination of SC identity and proliferation by requiring only a small number of cells. Furthermore, the CellProfiler software was used to develop an automated image analysis pipeline that quantified SCs and proliferating SCs from the obtained immunofluorescence images. By comparing the results of total cell count, SC purity and SC proliferation rate between manual counting and the CellProfiler output, we demonstrated applicability and reliability of the established pipeline. In conclusion, we here combined the cytospin technique, a multi-colour immunofluorescence staining panel, and an automated image analysis pipeline to enable the quantification of SC purity and SC proliferation from small cell aliquots. This procedure represents a solid read-out to simplify and standardize the quantification of primary SC culture purity and proliferation.

Rapid antibiotic susceptibility testing based on bacterial motion patterns with long short-term memory neural networks

Antibiotic resistance is an increasing public health threat. To combat it, a fast method to determine the antibiotic susceptibility of infecting pathogens is required. Here we present an optical imaging-based method to track the motion of single bacterial cells and generate a model to classify active and inactive cells based on the motion patterns of the individual cells. The model includes an image-processing algorithm to segment individual bacterial cells and track the motion of the cells over time, and a deep learning algorithm (Long Short-Term Memory network) to learn and determine if a bacterial cell is active or inactive. By applying the model to human urine specimens spiked with an Escherichia coli lab strain, we show that the method can accurately perform antibiotic susceptibility testing as fast as 30 minutes for five commonly used antibiotics.

High-Throughput Segmentation of Tiled Biological Structures using Random-Walk Distance Transforms

Various 3D imaging techniques are routinely used to examine biological materials, the results of which are usually a stack of grayscale images. In order to quantify structural aspects of the biological materials, however, they must first be extracted from the dataset in a process called segmentation. If the individual structures to be extracted are in contact or very close to each other, distance-based segmentation methods utilizing the Euclidean distance transform are commonly employed. Major disadvantages of the Euclidean distance transform, however, are its susceptibility to noise (very common in biological data), which often leads to incorrect segmentations (i.e., poor separation of objects of interest), and its limitation of being only effective for roundish objects. In the present work, we propose an alternative distance transform method, the random-walk distance transform, and demonstrate its effectiveness in high-throughput segmentation of three microCT datasets of biological tilings (i.e., structures composed of a large number of similar repeating units). In contrast to the Euclidean distance transform, the random-walk approach represents the global, rather than the local, geometric character of the objects to be segmented and, thus, is less susceptible to noise. In addition, it is directly applicable to structures with anisotropic shape characteristics. Using three case studies—tessellated cartilage from a stingray, the dermal endoskeleton of a starfish, and the prismatic layer of a bivalve mollusc shell—we provide a typical workflow for the segmentation of tiled structures, describe core image processing concepts that are underused in biological research, and show that for each study system, large amounts of biologically-relevant data can be rapidly segmented, visualized, and analyzed.

Improved small blob detection in 3D images using jointly constrained deep learning and Hessian analysis

Imaging biomarkers are being rapidly developed for early diagnosis and staging of disease. The development of these biomarkers requires advances in both image acquisition and analysis. Detecting and segmenting objects from images are often the first steps in quantitative measurement of these biomarkers. The challenges of detecting objects in images, particularly small objects known as blobs, include low image resolution, image noise and overlap between the blobs. The Difference of Gaussian (DoG) detector has been used to overcome these challenges in blob detection. However, the DoG detector is susceptible to over-detection and must be refined for robust, reproducible detection in a wide range of medical images. In this research, we propose a joint constraint blob detector from U-Net, a deep learning model, and Hessian analysis, to overcome these problems and identify true blobs from noisy medical images. We evaluate this approach, UH-DoG, using a public 2D fluorescent dataset for cell nucleus detection and a 3D kidney magnetic resonance imaging dataset for glomerulus detection. We then compare this approach to methods in the literature. While comparable to the other four comparing methods on recall, the UH-DoG outperforms them on both precision and F-score.

Nucleus segmentation across imaging experiments: the 2018 Data Science Bowl

Segmenting the nuclei of cells in microscopy images is often the first step in the quantitative analysis of imaging data for biological and biomedical applications. Many bioimage analysis tools can segment nuclei in images but need to be selected and configured for every experiment. The 2018 Data Science Bowl attracted 3,891 teams worldwide to make the first attempt to build a segmentation method that could be applied to any two-dimensional light microscopy image of stained nuclei across experiments, with no human interaction. Top participants in the challenge succeeded in this task, developing deep-learning-based models that identified cell nuclei across many image types and experimental conditions without the need to manually adjust segmentation parameters. This represents an important step toward configuration-free bioimage analysis software tools.

An open source automated tumor infiltrating lymphocyte algorithm for prognosis in melanoma

Assessment of tumor infiltrating lymphocytes (TILs) as a prognostic variable in melanoma has not seen broad adoption due to lack of standardization. Automation could represent a solution. Here, using open source software, we build an algorithm for image-based automated assessment of TILs on hematoxylin-eosin stained sections in melanoma. Using a retrospective collection of 641 melanoma patients comprising four independent cohorts; one training set (N = 227) and three validation cohorts (N = 137, N = 201, N = 76) from 2 institutions, we show that the automated TIL scoring algorithm separates patients into favorable and poor prognosis cohorts, where higher TILs scores were associated with favorable prognosis. In multivariable analyses, automated TIL scores show an independent association with disease-specific overall survival. Therefore, the open source, automated TIL scoring is an independent prognostic marker in melanoma. With further study, we believe that this algorithm could be useful to define a subset of patients that could potentially be spared immunotherapy.

Fluorescence anisotropy imaging in drug discovery

Non-invasive measurement of drug-target engagement can provide critical insights in the molecular pharmacology of small molecule drugs. Fluorescence polarization/fluorescence anisotropy measurements are commonly employed in protein/cell screening assays. However, the expansion of such measurements to the in vivo setting has proven difficult until recently. With the advent of high-resolution fluorescence anisotropy microscopy it is now possible to perform kinetic measurements of intracellular drug distribution and target engagement in commonly used mouse models. In this review we discuss the background, current advances and future perspectives in intravital fluorescence anisotropy measurements to derive pharmacokinetic and pharmacodynamic measurements in single cells and whole organs.

Imaging, Visualization, and Computation in Developmental Biology

Embryonic development is highly complex and dynamic, requiring the coordination of numerous molecular and cellular events at precise times and places. Advances in imaging technology have made it possible to follow developmental processes at cellular, tissue, and organ levels over time as they take place in the intact embryo. Parallel innovations of in vivo probes permit imaging to report on molecular, physiological, and anatomical events of embryogenesis, but the resulting multidimensional data sets pose significant challenges for extracting knowledge. In this review, we discuss recent and emerging advances in imaging technologies, in vivo labeling, and data processing that offer the greatest potential for jointly deciphering the intricate cellular dynamics and the underlying molecular mechanisms. Our discussion of the emerging area of “image-omics” highlights both the challenges of data analysis and the promise of more fully embracing computation and data science for rapidly advancing our understanding of biology.

IFN-γ-response mediator GBP-1 represses human cell proliferation by inhibiting the Hippo signaling transcription factor TEAD

Interferon-gamma (IFN-γ) is a pleiotropic cytokine that exerts important functions in inflammation, infectious diseases, and cancer. The large GTPase human guanylate-binding protein 1 (GBP-1) is among the most strongly IFN-γ-induced cellular proteins. Previously, it has been shown that GBP-1 mediates manifold cellular responses to IFN-γ including the inhibition of proliferation, spreading, migration, and invasion and through this exerts anti-tumorigenic activity. However, the mechanisms of GBP-1 anti-tumorigenic activities remain poorly understood. Here, we elucidated the molecular mechanism of the human GBP-1-mediated suppression of proliferation by demonstrating for the first time a cross-talk between the anti-tumorigenic IFN-γ and Hippo pathways. The α9-helix of GBP-1 was found to be sufficient to inhibit proliferation. Protein-binding and molecular modeling studies revealed that the α9-helix binds to the DNA-binding domain of the Hippo signaling transcription factor TEA domain protein (TEAD) mediated by the ³⁷⁶VDHLFQK³⁸² sequence at the N-terminus of the GBP-1-α9-helix. Mutation of this sequence resulted in abrogation of both TEAD interaction and suppression of proliferation. Further on, the interaction caused inhibition of TEAD transcriptional activity associated with the down-regulation of TEAD-target genes. In agreement with these results, IFN-γ treatment of the cells also impaired TEAD activity, and this effect was abrogated by siRNA-mediated inhibition of GBP-1 expression. Altogether, this demonstrated that the α9-helix is the proliferation inhibitory domain of GBP-1, which acts independent of the GTPase activity through the inhibition of the Hippo transcription factor TEAD in mediating the anti-proliferative cell response to IFN-γ.