Automated Classification of Apoptosis in Phase Contrast Microscopy Using Capsule Network

Transformer-based spatial-temporal detection of apoptotic cell death in live-cell imaging

Intravital microscopy has revolutionized live-cell imaging by allowing the study of spatial-temporal cell dynamics in living animals. However, the complexity of the data generated by this technology has limited the development of effective computational tools to identify and quantify cell processes. Amongst them, apoptosis is a crucial form of regulated cell death involved in tissue homeostasis and host defense. Live-cell imaging enabled the study of apoptosis at the cellular level, enhancing our understanding of its spatial-temporal regulation. However, at present, no computational method can deliver robust detection of apoptosis in microscopy timelapses. To overcome this limitation, we developed ADeS, a deep learning-based apoptosis detection system that employs the principle of activity recognition. We trained ADeS on extensive datasets containing more than 10,000 apoptotic instances collected both in vitro and in vivo, achieving a classification accuracy above 98% and outperforming state-of-the-art solutions. ADeS is the first method capable of detecting the location and duration of multiple apoptotic events in full microscopy timelapses, surpassing human performance in the same task. We demonstrated the effectiveness and robustness of ADeS across various imaging modalities, cell types, and staining techniques. Finally, we employed ADeS to quantify cell survival in vitro and tissue damage in mice, demonstrating its potential application in toxicity assays, treatment evaluation, and inflammatory dynamics. Our findings suggest that ADeS is a valuable tool for the accurate detection and quantification of apoptosis in live-cell imaging and, in particular, intravital microscopy data, providing insights into the complex spatial-temporal regulation of this process.

Scanning ion conductance microscopy revealed cisplatin-induced morphological changes related to apoptosis in single adenocarcinoma cells

The studies of drug-induced apoptosis play a vital role in the identification of potential drugs that could treat diseases such as cancer. Alterations in the native morphology of cancer cells following treatment with anticancer drugs serve as one of the indicators that reveal drug efficacy. Various techniques such as optical microscopy, electron microscopy (EM), and atomic force microscopy (AFM) have been used to map the three dimensional (3D) morphological changes in cells induced with drugs. However, caution should be exercised when interpreting morphological data from techniques that might alter the native morphology of cells, caused by phototoxicity, electron beam invasiveness, intrusive sample preparation, and cell membrane deformation. Herein, we have used scanning ion conductance microscopy (SICM) to study the 3D morphology and roughness of A549 adenocarcinoma cells under physiological conditions before and after cisplatin induced apoptosis, where we observed an increase in height, overall shrinkage of the cells, and irregular features form on the cell membrane. Tracking the morphology of the same single A549 cells exposed to cisplatin unveiled heterogeneity in response to the drug, formation of membrane blebs, and an increase in membrane roughness. We have also demonstrated the use of SICM for studying the effect of cisplatin on the dynamic changes in the volume of A549 cells over days. SICM is demonstrated as a technique for studying the effect of drug induced apoptosis in the same cells over time, and for multiple different single cells.

Automated detection of apoptotic bodies and cells in label-free time-lapse high-throughput video microscopy using deep convolutional neural networks

MOTIVATION

Reliable label-free methods are needed for detecting and profiling apoptotic events in time-lapse cell-cell interaction assays. Prior studies relied on fluorescent markers of apoptosis, e.g. Annexin-V, that provide an inconsistent and late indication of apoptotic onset for human melanoma cells. Our motivation is to improve the detection of apoptosis by directly detecting apoptotic bodies in a label-free manner.

RESULTS

Our trained ResNet50 network identified nanowells containing apoptotic bodies with 92% accuracy and predicted the onset of apoptosis with an error of one frame (5 min/frame). Our apoptotic body segmentation yielded an IoU accuracy of 75%, allowing associative identification of apoptotic cells. Our method detected apoptosis events, 70% of which were not detected by Annexin-V staining.

AVAILABILITY AND IMPLEMENTATION

Open-source code and sample data provided at https://github.com/kwu14victor/ApoBDproject.

Classification of stages in cervical cancer MRI by customized CNN and transfer learning

Cervical cancer is the common cancer among women, where early-stage diagnoses of cervical cancer lead to recovery from the deadly cervical cancer. Correct cervical cancer staging is predominant to decide the treatment. Hence, cervical cancer staging is an important problem in designing automatic detection and diagnosing applications of the medical field. Convolutional Neural Networks (CNNs) often plays a greater role in object identification and classification. The performance of CNN in medical image classification can already compete with radiologists. In this paper, we planned to build a deep Capsule Network (CapsNet) for medical image classification that can achieve high accuracy using cervical cancer Magnetic Resonance (MR) images. In this study, a customized deep CNN model is developed using CapsNet to automatically predict the cervical cancer from MR images. In CapsNet, each layer receives input from all preceding layers, which helps to classify the features. The hyper parameters are estimated and it controls the backpropagation gradient at the initial learning. To improve the CapsNet performance, residual blocks are included between dense layers. Training and testing are performed with around 12,771 T2-weighted MR images of the TCGA-CESC dataset publicly available for research work. The results show that the accuracy of Customized CNN using CapsNetis higher and behaves well in classifying the cervical cancer. Thus, it is evident that CNN models can be used in developing automatic image analysis tools in the medical field.

Regioselective synthesis and in vitro cytotoxicity evaluation of 3-thiooxindole derivatives: Tubulin polymerization inhibition and apoptosis inducing studies

Herein, regiospecific nucleophilic ring-opening of spiroaziridine oxindoles has been established to afford 3-substituted-thiooxindole derivatives as anticancer agents. Among the new series, compounds 7d and 9c exhibited promising cytotoxic activity toward HCT-116 cells with IC₅₀ values of 6.73 ± 0.36 and 6.64 ± 0.95 µM, respectively. Further, AO/EB, DCFDA, and DAPI staining studies were executed to establish the underlying apoptosis mechanism which displayed significant nuclear and morphological alterations. JC-1 staining and annexin V binding assay inferred the loss of mitochondrial membrane potential in HCT-116 cancer cells. Cell cycle analysis showed the treatment of 9c against HCT-116 cells, arrested the cell cycle in G2-M phase. In addition, tubulin binding assay revealed that compound 9c exhibited tubulin polymerase inhibition with IC₅₀ value of 9.73 ± 0.18 μM. This inhibition of tubulin polymerase was further supported by binding interactions of 9c with tubulin through docking studies on PDB ID: 3E22.

Evaluation of grouped capsule network for intracranial hemorrhage segmentation in CT scans

Intracranial hemorrhage is a cerebral vascular disease with high mortality. Automotive diagnosing and segmentation of intracranial hemorrhage in Computed Tomography (CT) could assist the neurosurgeon in making treatment plans, which improves the survival rate. In this paper, we design a grouped capsule network named GroupCapsNet to segment the hemorrhage region from a Non-contract CT scan. In grouped capsule network, we constrain the prediction capsules for output capsules produced from different groups of input capsules with various types in each layer. This method can reduce the number of intermediate prediction capsules and accelerate the capsule network. In addition, we modify the squashing function to further accelerate the forward procedure without sacrificing its performance. We evaluate our proposed method with a collected dataset containing 210 intracranial hemorrhage CT scan slices. In experiments, our proposed method achieves competitive results in intracranial hemorrhage area segmentation compared to the existing methods.

Machine-Learning-Aided Quantification of Area Coverage of Adherent Cells from Phase-Contrast Images

The advances in machine learning (ML) software availability, efficiency, and friendliness, combined with the increase in the computation power of personal computers, are harnessed to rapidly and (relatively) effortlessly analyze time-lapse image series of adherent cell cultures, taken with phase-contrast microscopy (PCM). Since PCM is arguably the most widely used technique to visualize adherent cells in a label-free, noninvasive, and nondisruptive manner, the ability to easily extract quantitative information on the area covered by cells, should provide a valuable tool for investigation. We demonstrate two cases, in one we monitor the shrinking of cells in response to a toxicant, and in the second we measure the proliferation curve of mesenchymal stem cells (MSCs).

Deep Phenotypic Cell Classification using Capsule Neural Network

Recent developments in ultra-high-throughput microscopy have created a new generation of cell classification methodologies focused solely on image-based cell phenotypes. These image-based analyses enable morphological profiling and screening of thousands or even millions of single cells at a fraction of the cost. They have been shown to demonstrate the statistical significance required for understanding the role of cell heterogeneity in diverse biologists. However, these single-cell analysis techniques are slow and require expensive genetic/epigenetic analysis. This treatise proposes an innovative DL system based on the newly created capsule networks (CapsNet) architecture. The proposed deep CapsNet model employs "Capsules" for high-level feature abstraction relevant to the cell category. Experiments demonstrate that our proposed system can accurately classify different types of cells based on phenotypic label-free bright-field images with over 98.06% accuracy and that deep CapsNet models outperform CNN models in the prior art.

Memory-Augmented Capsule Network for Adaptable Lung Nodule Classification

Computer-aided diagnosis (CAD) systems must constantly cope with the perpetual changes in data distribution caused by different sensing technologies, imaging protocols, and patient populations. Adapting these systems to new domains often requires significant amounts of labeled data for re-training. This process is labor-intensive and time-consuming. We propose a memory-augmented capsule network for the rapid adaptation of CAD models to new domains. It consists of a capsule network that is meant to extract feature embeddings from some high-dimensional input, and a memory-augmented task network meant to exploit its stored knowledge from the target domains. Our network is able to efficiently adapt to unseen domains using only a few annotated samples. We evaluate our method using a large-scale public lung nodule dataset (LUNA), coupled with our own collected lung nodules and incidental lung nodules datasets. When trained on the LUNA dataset, our network requires only 30 additional samples from our collected lung nodule and incidental lung nodule datasets to achieve clinically relevant performance (0.925 and 0.891 area under receiving operating characteristic curves (AUROC), respectively). This result is equivalent to using two orders of magnitude less labeled training data while achieving the same performance. We further evaluate our method by introducing heavy noise, artifacts, and adversarial attacks. Under these severe conditions, our network's AUROC remains above 0.7 while the performance of state-of-the-art approaches reduce to chance level.

Efficient-CapsNet: capsule network with self-attention routing

Deep convolutional neural networks, assisted by architectural design strategies, make extensive use of data augmentation techniques and layers with a high number of feature maps to embed object transformations. That is highly inefficient and for large datasets implies a massive redundancy of features detectors. Even though capsules networks are still in their infancy, they constitute a promising solution to extend current convolutional networks and endow artificial visual perception with a process to encode more efficiently all feature affine transformations. Indeed, a properly working capsule network should theoretically achieve higher results with a considerably lower number of parameters count due to intrinsic capability to generalize to novel viewpoints. Nevertheless, little attention has been given to this relevant aspect. In this paper, we investigate the efficiency of capsule networks and, pushing their capacity to the limits with an extreme architecture with barely 160 K parameters, we prove that the proposed architecture is still able to achieve state-of-the-art results on three different datasets with only 2% of the original CapsNet parameters. Moreover, we replace dynamic routing with a novel non-iterative, highly parallelizable routing algorithm that can easily cope with a reduced number of capsules. Extensive experimentation with other capsule implementations has proved the effectiveness of our methodology and the capability of capsule networks to efficiently embed visual representations more prone to generalization.

Whole-brain tissue mapping toolkit using large-scale highly multiplexed immunofluorescence imaging and deep neural networks

Mapping biological processes in brain tissues requires piecing together numerous histological observations of multiple tissue samples. We present a direct method that generates readouts for a comprehensive panel of biomarkers from serial whole-brain slices, characterizing all major brain cell types, at scales ranging from subcellular compartments, individual cells, local multi-cellular niches, to whole-brain regions from each slice. We use iterative cycles of optimized 10-plex immunostaining with 10-color epifluorescence imaging to accumulate highly enriched image datasets from individual whole-brain slices, from which seamless signal-corrected mosaics are reconstructed. Specific fluorescent signals of interest are isolated computationally, rejecting autofluorescence, imaging noise, cross-channel bleed-through, and cross-labeling. Reliable large-scale cell detection and segmentation are achieved using deep neural networks. Cell phenotyping is performed by analyzing unique biomarker combinations over appropriate subcellular compartments. This approach can accelerate pre-clinical drug evaluation and system-level brain histology studies by simultaneously profiling multiple biological processes in their native anatomical context.

Kidney Level Lupus Nephritis Classification Using Uncertainty Guided Bayesian Convolutional Neural Networks

The kidney biopsy based diagnosis of Lupus Nephritis (LN) is characterized by low inter-observer agreement, with misdiagnosis being associated with increased patient morbidity and mortality. Although various Computer Aided Diagnosis (CAD) systems have been developed for other nephrohistopathological applications, little has been done to accurately classify kidneys based on their kidney level Lupus Glomerulonephritis (LGN) scores. The successful implementation of CAD systems has also been hindered by the diagnosing physician's perceived classifier strengths and weaknesses, which has been shown to have a negative effect on patient outcomes. We propose an Uncertainty-Guided Bayesian Classification (UGBC) scheme that is designed to accurately classify control, class I/II, and class III/IV LGN (3 class) at both the glomerular-level classification task (26,634 segmented glomerulus images) and the kidney-level classification task (87 MRL/lpr mouse kidney sections). Data annotation was performed using a high throughput, bulk labeling scheme that is designed to take advantage of Deep Neural Network's (or DNNs) resistance to label noise. Our augmented UGBC scheme achieved a 94.5% weighted glomerular-level accuracy while achieving a weighted kidney-level accuracy of 96.6%, improving upon the standard Convolutional Neural Network (CNN) architecture by 11.8% and 3.5% respectively.

Deciphering Cancer Cell Behavior From Motility and Shape Features: Peer Prediction and Dynamic Selection to Support Cancer Diagnosis and Therapy

Cell motility varies according to intrinsic features and microenvironmental stimuli, being a signature of underlying biological phenomena. The heterogeneity in cell response, due to multilevel cell diversity especially relevant in cancer, poses a challenge in identifying the biological scenario from cell trajectories. We propose here a novel peer prediction strategy among cell trajectories, deciphering cell state (tumor vs. nontumor), tumor stage, and response to the anticancer drug etoposide, based on morphology and motility features, solving the strong heterogeneity of individual cell properties. The proposed approach first barcodes cell trajectories, then automatically selects the good ones for optimal model construction (good teacher and test sample selection), and finally extracts a collective response from the heterogeneous populations via cooperative learning approaches, discriminating with high accuracy prostate noncancer vs. cancer cells of high vs. low malignancy. Comparison with standard classification methods validates our approach, which therefore represents a promising tool for addressing clinically relevant issues in cancer diagnosis and therapy, e.g., detection of potentially metastatic cells and anticancer drug screening.

DA-CapsNet: dual attention mechanism capsule network

A capsule network (CapsNet) is a recently proposed neural network model with a new structure. The purpose of CapsNet is to form activation capsules. In this paper, our team proposes a dual attention mechanism capsule network (DA-CapsNet). In DA-CapsNet, the first layer of the attention mechanism is added after the convolution layer and is referred to as Conv-Attention; the second layer is added after the PrimaryCaps and is referred to as Caps-Attention. The experimental results show that DA-CapsNet performs better than CapsNet. For MNIST, the trained DA-CapsNet is tested in the testset, the accuracy of the DA-CapsNet is 100% after 8 epochs, compared to 25 epochs for CapsNet. For SVHN, CIFAR10, FashionMNIST, smallNORB, and COIL-20, the highest accuracy of DA-CapsNet was 3.46%, 2.52%, 1.57%, 1.33% and 1.16% higher than that of CapsNet. And the results of image reconstruction in COIL-20 show that DA-CapsNet has a more competitive performance than CapsNet.

Risk-Aware Machine Learning Classifier for Skin Lesion Diagnosis

Knowing when a machine learning system is not confident about its prediction is crucial in medical domains where safety is critical. Ideally, a machine learning algorithm should make a prediction only when it is highly certain about its competency, and refer the case to physicians otherwise. In this paper, we investigate how Bayesian deep learning can improve the performance of the machine-physician team in the skin lesion classification task. We used the publicly available HAM10000 dataset, which includes samples from seven common skin lesion categories: Melanoma (MEL), Melanocytic Nevi (NV), Basal Cell Carcinoma (BCC), Actinic Keratoses and Intraepithelial Carcinoma (AKIEC), Benign Keratosis (BKL), Dermatofibroma (DF), and Vascular (VASC) lesions. Our experimental results show that Bayesian deep networks can boost the diagnostic performance of the standard DenseNet-169 model from 81.35% to 83.59% without incurring additional parameters or heavy computation. More importantly, a hybrid physician-machine workflow reaches a classification accuracy of 90 % while only referring 35 % of the cases to physicians. The findings are expected to generalize to other medical diagnosis applications. We believe that the availability of risk-aware machine learning methods will enable a wider adoption of machine learning technology in clinical settings.

Phasetime: Deep Learning Approach to Detect Nuclei in Time Lapse Phase Images

Time lapse microscopy is essential for quantifying the dynamics of cells, subcellular organelles and biomolecules. Biologists use different fluorescent tags to label and track the subcellular structures and biomolecules within cells. However, not all of them are compatible with time lapse imaging, and the labeling itself can perturb the cells in undesirable ways. We hypothesized that phase image has the requisite information to identify and track nuclei within cells. By utilizing both traditional blob detection to generate binary mask labels from the stained channel images and the deep learning Mask RCNN model to train a detection and segmentation model, we managed to segment nuclei based only on phase images. The detection average precision is 0.82 when the IoU threshold is to be set 0.5. And the mean IoU for masks generated from phase images and ground truth masks from experts is 0.735. Without any ground truth mask labels during the training time, this is good enough to prove our hypothesis. This result enables the ability to detect nuclei without the need for exogenous labeling.