Improved automatic detection of herpesvirus secondary envelopment stages in electron microscopy by augmenting training data with synthetic labelled images generated by a generative adversarial network

Machine learning for cross-scale microscopy of viruses

Despite advances in virological sciences and antiviral research, viruses continue to emerge, circulate, and threaten public health. We still lack a comprehensive understanding of how cells and individuals remain susceptible to infectious agents. This deficiency is in part due to the complexity of viruses, including the cell states controlling virus-host interactions. Microscopy samples distinct cellular infection stages in a multi-parametric, time-resolved manner at molecular resolution and is increasingly enhanced by machine learning and deep learning. Here we discuss how state-of-the-art artificial intelligence (AI) augments light and electron microscopy and advances virological research of cells. We describe current procedures for image denoising, object segmentation, tracking, classification, and super-resolution and showcase examples of how AI has improved the acquisition and analyses of microscopy data. The power of AI-enhanced microscopy will continue to help unravel virus infection mechanisms, develop antiviral agents, and improve viral vectors.

CardioVinci: building blocks for virtual cardiac cells using deep learning

Advances in electron microscopy (EM) such as electron tomography and focused ion-beam scanning electron microscopy provide unprecedented, three-dimensional views of cardiac ultrastructures within sample volumes ranging from hundreds of nanometres to hundreds of micrometres. The datasets from these samples are typically large, with file sizes ranging from gigabytes to terabytes and the number of image slices within the three-dimensional stack in the hundreds. A significant bottleneck with these large datasets is the time taken to extract and statistically analyse three-dimensional changes in cardiac ultrastructures. This is because of the inherently low contrast and the significant amount of structural detail that is present in EM images. These datasets often require manual annotation, which needs substantial person-hours and may result in only partial segmentation that makes quantitative analysis of the three-dimensional volumes infeasible. We present CardioVinci, a deep learning workflow to automatically segment and statistically quantify the morphologies and spatial assembly of mitochondria, myofibrils and Z-discs with minimal manual annotation. The workflow encodes a probabilistic model of the three-dimensional cardiomyocyte using a generative adversarial network. This generative model can be used to create new models of cardiomyocyte architecture that reflect variations in morphologies and cell architecture found in EM datasets.

This article is part of the theme issue ‘The cardiomyocyte: new revelations on the interplay between architecture and function in growth, health, and disease’.

Weighted average ensemble-based semantic segmentation in biological electron microscopy images

Semantic segmentation of electron microscopy images using deep learning methods is a valuable tool for the detailed analysis of organelles and cell structures. However, these methods require a large amount of labeled ground truth data that is often unavailable. To address this limitation, we present a weighted average ensemble model that can automatically segment biological structures in electron microscopy images when trained with only a small dataset. Thus, we exploit the fact that a combination of diverse base-learners is able to outperform one single segmentation model. Our experiments with seven different biological electron microscopy datasets demonstrate quantitative and qualitative improvements. We show that the Grad-CAM method can be used to interpret and verify the prediction of our model. Compared with a standard U-Net, the performance of our method is superior for all tested datasets. Furthermore, our model leverages a limited number of labeled training data to segment the electron microscopy images and therefore has a high potential for automated biological applications.

The unconventional way out-Egress of HCMV through multiviral bodies

Human cytomegalovirus (HCMV) is a ubiquitous herpesvirus and the leading cause of congenital disabilities as well as a significant cause of disease in immunocompromised patients. The envelopment and egress of HCMV particles is an essential step of the viral life cycle as it determines viral spread and potentially tropism. Here we review the current literature on HCMV envelopment and egress with a particular focus on the role of virus-containing multivesicular body-like vesicles for virus egress and spread. We discuss the difficulties of determining the cellular provenance of these structures in light of viral redistribution of cellular marker proteins and provide potential paths to illuminate their genesis. Finally, we discuss how divergent egress pathways could result in virions of different tropisms.

Generative Adversarial Networks for Morphological-Temporal Classification of Stem Cell Images

Frequently, neural network training involving biological images suffers from a lack of data, resulting in inefficient network learning. This issue stems from limitations in terms of time, resources, and difficulty in cellular experimentation and data collection. For example, when performing experimental analysis, it may be necessary for the researcher to use most of their data for testing, as opposed to model training. Therefore, the goal of this paper is to perform dataset augmentation using generative adversarial networks (GAN) to increase the classification accuracy of deep convolutional neural networks (CNN) trained on induced pluripotent stem cell microscopy images. The main challenges are: 1. modeling complex data using GAN and 2. training neural networks on augmented datasets that contain generated data. To address these challenges, a temporally constrained, hierarchical classification scheme that exploits domain knowledge is employed for model learning. First, image patches of cell colonies from gray-scale microscopy images are generated using GAN, and then these images are added to the real dataset and used to address class imbalances at multiple stages of training. Overall, a 2% increase in both true positive rate and F1-score is observed using this method as compared to a straightforward, imbalanced classification network, with some greater improvements on a classwise basis. This work demonstrates that synergistic model design involving domain knowledge is key for biological image analysis and improves model learning in high-throughput scenarios.