Structured illumination microscopy combined with machine learning enables the high throughput analysis and classification of virus structure

AI analysis of super-resolution microscopy: Biological discovery in the absence of ground truth

Super-resolution microscopy, or nanoscopy, enables the use of fluorescent-based molecular localization tools to study molecular structure at the nanoscale level in the intact cell, bridging the mesoscale gap to classical structural biology methodologies. Analysis of super-resolution data by artificial intelligence (AI), such as machine learning, offers tremendous potential for the discovery of new biology, that, by definition, is not known and lacks ground truth. Herein, we describe the application of weakly supervised paradigms to super-resolution microscopy and its potential to enable the accelerated exploration of the nanoscale architecture of subcellular macromolecules and organelles.

Machine learning for cross-scale microscopy of viruses

Despite advances in virological sciences and antiviral research, viruses continue to emerge, circulate, and threaten public health. We still lack a comprehensive understanding of how cells and individuals remain susceptible to infectious agents. This deficiency is in part due to the complexity of viruses, including the cell states controlling virus-host interactions. Microscopy samples distinct cellular infection stages in a multi-parametric, time-resolved manner at molecular resolution and is increasingly enhanced by machine learning and deep learning. Here we discuss how state-of-the-art artificial intelligence (AI) augments light and electron microscopy and advances virological research of cells. We describe current procedures for image denoising, object segmentation, tracking, classification, and super-resolution and showcase examples of how AI has improved the acquisition and analyses of microscopy data. The power of AI-enhanced microscopy will continue to help unravel virus infection mechanisms, develop antiviral agents, and improve viral vectors.

Fluorescence Microscopy in Adeno-Associated Virus Research

Research on adeno-associated virus (AAV) and its recombinant vectors as well as on fluorescence microscopy imaging is rapidly progressing driven by clinical applications and new technologies, respectively. The topics converge, since high and super-resolution microscopes facilitate the study of spatial and temporal aspects of cellular virus biology. Labeling methods also evolve and diversify. We review these interdisciplinary developments and provide information on the technologies used and the biological knowledge gained. The emphasis lies on the visualization of AAV proteins by chemical fluorophores, protein fusions and antibodies as well as on methods for the detection of adeno-associated viral DNA. We add a short overview of fluorescent microscope techniques and their advantages and challenges in detecting AAV.

On The Potential of Image Moments for Medical Diagnosis

Medical imaging is widely used for diagnosis and postoperative or post-therapy monitoring. The ever-increasing number of images produced has encouraged the introduction of automated methods to assist doctors or pathologists. In recent years, especially after the advent of convolutional neural networks, many researchers have focused on this approach, considering it to be the only method for diagnosis since it can perform a direct classification of images. However, many diagnostic systems still rely on handcrafted features to improve interpretability and limit resource consumption. In this work, we focused our efforts on orthogonal moments, first by providing an overview and taxonomy of their macrocategories and then by analysing their classification performance on very different medical tasks represented by four public benchmark data sets. The results confirmed that convolutional neural networks achieved excellent performance on all tasks. Despite being composed of much fewer features than those extracted by the networks, orthogonal moments proved to be competitive with them, showing comparable and, in some cases, better performance. In addition, Cartesian and harmonic categories provided a very low standard deviation, proving their robustness in medical diagnostic tasks. We strongly believe that the integration of the studied orthogonal moments can lead to more robust and reliable diagnostic systems, considering the performance obtained and the low variation of the results. Finally, since they have been shown to be effective on both magnetic resonance and computed tomography images, they can be easily extended to other imaging techniques.

Virus Detection and Identification in Minutes Using Single-Particle Imaging and Deep Learning

The increasing frequency and magnitude of viral outbreaks in recent decades, epitomized by the COVID-19 pandemic, has resulted in an urgent need for rapid and sensitive diagnostic methods. Here, we present a methodology for virus detection and identification that uses a convolutional neural network to distinguish between microscopy images of fluorescently labeled intact particles of different viruses. Our assay achieves labeling, imaging, and virus identification in less than 5 min and does not require any lysis, purification, or amplification steps. The trained neural network was able to differentiate SARS-CoV-2 from negative clinical samples, as well as from other common respiratory pathogens such as influenza and seasonal human coronaviruses. We were also able to differentiate closely related strains of influenza, as well as SARS-CoV-2 variants. Additional and novel pathogens can easily be incorporated into the test through software updates, offering the potential to rapidly utilize the technology in future infectious disease outbreaks or pandemics. Single-particle imaging combined with deep learning therefore offers a promising alternative to traditional viral diagnostic and genomic sequencing methods and has the potential for significant impact.

Fluorescent Nanoparticles for Super-Resolution Imaging

Super-resolution imaging techniques that overcome the diffraction limit of light have gained wide popularity for visualizing cellular structures with nanometric resolution. Following the pace of hardware developments, the availability of new fluorescent probes with superior properties is becoming ever more important. In this context, fluorescent nanoparticles (NPs) have attracted increasing attention as bright and photostable probes that address many shortcomings of traditional fluorescent probes. The use of NPs for super-resolution imaging is a recent development and this provides the focus for the current review. We give an overview of different super-resolution methods and discuss their demands on the properties of fluorescent NPs. We then review in detail the features, strengths, and weaknesses of each NP class to support these applications and provide examples from their utilization in various biological systems. Moreover, we provide an outlook on the future of the field and opportunities in material science for the development of probes for multiplexed subcellular imaging with nanometric resolution.

DeepBacs for multi-task bacterial image analysis using open-source deep learning approaches

This work demonstrates and guides how to use a range of state-of-the-art artificial neural-networks to analyse bacterial microscopy images using the recently developed ZeroCostDL4Mic platform. We generated a database of image datasets used to train networks for various image analysis tasks and present strategies for data acquisition and curation, as well as model training. We showcase different deep learning (DL) approaches for segmenting bright field and fluorescence images of different bacterial species, use object detection to classify different growth stages in time-lapse imaging data, and carry out DL-assisted phenotypic profiling of antibiotic-treated cells. To also demonstrate the ability of DL to enhance low-phototoxicity live-cell microscopy, we showcase how image denoising can allow researchers to attain high-fidelity data in faster and longer imaging. Finally, artificial labelling of cell membranes and predictions of super-resolution images allow for accurate mapping of cell shape and intracellular targets. Our purposefully-built database of training and testing data aids in novice users' training, enabling them to quickly explore how to analyse their data through DL. We hope this lays a fertile ground for the efficient application of DL in microbiology and fosters the creation of tools for bacterial cell biology and antibiotic research.

Virus morphology: Insights from super-resolution fluorescence microscopy

As epitomised by the COVID-19 pandemic, diseases caused by viruses are one of the greatest health and economic burdens to human society. Viruses are ‘nanostructures’, and their small size (typically less than 200 nm in diameter) can make it challenging to obtain images of their morphology and structure. Recent advances in fluorescence microscopy have given rise to super-resolution techniques, which have enabled the structure of viruses to be visualised directly at a resolution in the order of 20 nm. This mini-review discusses how recent state-of-the-art super-resolution imaging technologies are providing new nanoscale insights into virus structure.

Self-contained and modular structured illumination microscope

We present a modular implementation of structured illumination microscopy (SIM) that is fast, largely self-contained and that can be added onto existing fluorescence microscopes. Our strategy, which we call HIT-SIM, can theoretically deliver well over 50 super-resolved images per second and is readily compatible with existing acquisition software packages. We provide a full technical package consisting of schematics, a list of components and an alignment scheme that provides detailed specifications and assembly instructions. We illustrate the performance of the instrument by imaging optically large samples containing sequence-specifically stained DNA fragments.

Towards a Quantitative Single Particle Characterization by Super Resolution Microscopy: From Virus Structures to Antivirals Design

In the last year the COVID19 pandemic clearly illustrated the potential threat that viruses pose to our society. The characterization of viral structures and the identification of key proteins involved in each step of the cycle of infection are crucial to develop treatments. However, the small size of viruses, invisible under conventional fluorescence microscopy, make it difficult to study the organization of protein clusters within the viral particle. The applications of super-resolution microscopy have skyrocketed in the last years, converting this group into one of the leading techniques to characterize viruses and study the viral infection in cells, breaking the diffraction limit by achieving resolutions up to 10 nm using conventional probes such as fluorescent dyes and proteins. There are several super-resolution methods available and the selection of the right one it is crucial to study in detail all the steps involved in the viral infection, quantifying and creating models of infection for relevant viruses such as HIV-1, Influenza, herpesvirus or SARS-CoV-1. Here we review the use of super-resolution microscopy (SRM) to study all steps involved in the viral infection and antiviral design. In light of the threat of new viruses, these studies could inspire future assays to unveil the viral mechanism of emerging viruses and further develop successful antivirals against them.

Challenges facing quantitative large-scale optical super-resolution, and some simple solutions

Optical super-resolution microscopy (SRM) has enabled biologists to visualize cellular structures with near-molecular resolution, giving unprecedented access to details about the amounts, sizes, and spatial distributions of macromolecules in the cell. Precisely quantifying these molecular details requires large datasets of high-quality, reproducible SRM images. In this review, we discuss the unique set of challenges facing quantitative SRM, giving particular attention to the shortcomings of conventional specimen preparation techniques and the necessity for optimal labeling of molecular targets. We further discuss the obstacles to scaling SRM methods, such as lengthy image acquisition and complex SRM data analysis. For each of these challenges, we review the recent advances in the field that circumvent these pitfalls and provide practical advice to biologists for optimizing SRM experiments.

Application of Super-Resolution and Advanced Quantitative Microscopy to the Spatio-Temporal Analysis of Influenza Virus Replication

With an estimated three to five million human cases annually and the potential to infect domestic and wild animal populations, influenza viruses are one of the greatest health and economic burdens to our society, and pose an ongoing threat of large-scale pandemics. Despite our knowledge of many important aspects of influenza virus biology, there is still much to learn about how influenza viruses replicate in infected cells, for instance, how they use entry receptors or exploit host cell trafficking pathways. These gaps in our knowledge are due, in part, to the difficulty of directly observing viruses in living cells. In recent years, advances in light microscopy, including super-resolution microscopy and single-molecule imaging, have enabled many viral replication steps to be visualised dynamically in living cells. In particular, the ability to track single virions and their components, in real time, now allows specific pathways to be interrogated, providing new insights to various aspects of the virus-host cell interaction. In this review, we discuss how state-of-the-art imaging technologies, notably quantitative live-cell and super-resolution microscopy, are providing new nanoscale and molecular insights into influenza virus replication and revealing new opportunities for developing antiviral strategies.

Automatic classification and segmentation of single-molecule fluorescence time traces with deep learning

Traces from single-molecule fluorescence microscopy (SMFM) experiments exhibit photophysical artifacts that typically necessitate human expert screening, which is time-consuming and introduces potential for user-dependent expectation bias. Here, we use deep learning to develop a rapid, automatic SMFM trace selector, termed AutoSiM, that improves the sensitivity and specificity of an assay for a DNA point mutation based on single-molecule recognition through equilibrium Poisson sampling (SiMREPS). The improved performance of AutoSiM is based on accepting both more true positives and fewer false positives than the conventional approach of hidden Markov modeling (HMM) followed by hard thresholding. As a second application, the selector is used for automated screening of single-molecule Förster resonance energy transfer (smFRET) data to identify high-quality traces for further analysis, and achieves ~90% concordance with manual selection while requiring less processing time. Finally, we show that AutoSiM can be adapted readily to novel datasets, requiring only modest Transfer Learning.