Deep learning enables fast and dense single-molecule localization with high accuracy

Annotation and automated segmentation of single-molecule localisation microscopy data

Single Molecule Localisation Microscopy (SMLM) is becoming a widely used technique in cell biology. After processing the images, the molecular localisations are typically stored in a table as xy (or xyz) coordinates, with additional information, such as number of photons, etc. This set of coordinates can be used to generate an image to visualise the molecular distribution, for example, a 2D or 3D histogram of localisations. Many different methods have been devised to analyse SMLM data, among which cluster analysis of the localisations is popular. However, it can be useful to first segment the data, to extract the localisations in a specific region of a cell or in individual cells, prior to downstream analysis. Here we describe a pipeline for annotating localisations in an SMLM dataset in which we compared membrane segmentation approaches, including Otsu thresholding and machine learning models, and subsequent cell segmentation. We used an SMLM dataset derived from dSTORM images of sectioned cell pellets, stained for the membrane proteins EGFR (epidermal growth factor receptor) and EREG (epiregulin) as a test dataset. We found that a Cellpose model retrained on our data performed the best in the membrane segmentation task, allowing us to perform downstream cluster analysis of membrane versus cell interior localisations. We anticipate this will be generally useful for SMLM analysis.

Three-dimensional localization and tracking of chromosomal loci throughout the Escherichia coli cell cycle

The intracellular position of genes may impact their expression, but it has not been possible to accurately measure the 3D position of chromosomal loci. In 2D, loci can be tracked using arrays of DNA-binding sites for transcription factors (TFs) fused with fluorescent proteins. However, the same 2D data can result from different 3D trajectories. Here, we have developed a deep learning method for super-resolved astigmatism-based 3D localization of chromosomal loci in live E. coli cells which enables a precision better than 61 nm at a signal-to-background ratio of ~4 on a heterogeneous cell background. Determining the spatial localization of chromosomal loci, we find that some loci are at the periphery of the nucleoid for large parts of the cell cycle. Analyses of individual trajectories reveal that these loci are subdiffusive both longitudinally (x) and radially (r), but that individual loci explore the full radial width on a minute time scale.

Long-Axial-Range Double-Helix Point Spread Functions for 3D Volumetric Super-Resolution Imaging

Single-molecule localization microscopy (SMLM) is a powerful tool for observing structures beyond the diffraction limit of light. Combining SMLM with engineered point spread functions (PSFs) enables 3D imaging over an extended axial range, as has been demonstrated for super-resolution imaging of various cellular structures. However, super-resolving structures in 3D in thick samples, such as whole mammalian cells, remains challenging as it typically requires acquisition and postprocessing stitching of multiple slices to cover the entire sample volume or more complex analysis of the data. Here, we demonstrate how the imaging and analysis workflows can be simplified by 3D single-molecule super-resolution imaging with long-axial-range double-helix (DH)-PSFs. First, we experimentally benchmark the localization precisions of short- and long-axial-range DH-PSFs at different signal-to-background ratios by imaging fluorescent beads. The performance of the DH-PSFs in terms of achievable resolution and imaging speed was then quantified for 3D single-molecule super-resolution imaging of mammalian cells by DNA-PAINT imaging of nuclear lamina protein lamin B1 in U-2 OS cells. Furthermore, we demonstrate how the use of a deep-learning-based algorithm allows the localization of dense emitters, drastically improving the achievable imaging speed and resolution. Our data demonstrate that using long-axial-range DH-PSFs offers stitching-free, 3D super-resolution imaging of whole mammalian cells, simplifying the experimental and analysis procedures for obtaining volumetric nanoscale structural information.

Surpassing the Diffraction Limit in Label-Free Optical Microscopy

Super-resolution optical microscopy has enhanced our ability to visualize biological structures on the nanoscale. Fluorescence-based techniques are today irreplaceable in exploring the structure and dynamics of biological matter with high specificity and resolution. However, the fluorescence labeling concept narrows the range of observed interactions and fundamentally limits the spatiotemporal resolution. In contrast, emerging label-free imaging methods are not inherently limited by speed and have the potential to capture the entirety of complex biological processes and dynamics. While pushing a complex unlabeled microscopy image beyond the diffraction limit to single-molecule resolution and capturing dynamic processes at biomolecular time scales is widely regarded as unachievable, recent experimental strides suggest that elements of this vision might be already in place. These techniques derive signals directly from the sample using inherent optical phenomena, such as elastic and inelastic scattering, thereby enabling the measurement of additional properties, such as molecular mass, orientation, or chemical composition. This perspective aims to identify the cornerstones of future label-free super-resolution imaging techniques, discuss their practical applications and theoretical challenges, and explore directions that promise to enhance our understanding of complex biological systems through innovative optical advancements. Drawing on both traditional and emerging techniques, label-free super-resolution microscopy is evolving to offer detailed and dynamic imaging of living cells, surpassing the capabilities of conventional methods for visualizing biological complexities without the use of labels.

Machine learning in microscopy - insights, opportunities and challenges

Machine learning (ML) is transforming the field of image processing and analysis, from automation of laborious tasks to open-ended exploration of visual patterns. This has striking implications for image-driven life science research, particularly microscopy. In this Review, we focus on the opportunities and challenges associated with applying ML-based pipelines for microscopy datasets from a user point of view. We investigate the significance of different data characteristics - quantity, transferability and content - and how this determines which ML model(s) to use, as well as their output(s). Within the context of cell biological questions and applications, we further discuss ML utility range, namely data curation, exploration, prediction and explanation, and what they entail and translate to in the context of microscopy. Finally, we explore the challenges, common artefacts and risks associated with ML in microscopy. Building on insights from other fields, we propose how these pitfalls might be mitigated for in microscopy.

Neural network-assisted localization of clustered point spread functions in single-molecule localization microscopy

Single-molecule localization microscopy (SMLM), which has revolutionized nanoscale imaging, faces challenges in densely labelled samples due to fluorophore clustering, leading to compromised localization accuracy. In this paper, we propose a novel convolutional neural network (CNN)-assisted approach to address the issue of locating the clustered fluorophores. Our CNN is trained on a diverse data set of simulated SMLM images where it learns to predict point spread function (PSF) locations by generating Gaussian blobs as output. Through rigorous evaluation, we demonstrate significant improvements in PSF localization accuracy, especially in densely labelled samples where traditional methods struggle. In addition, we employ blob detection as a post-processing technique to refine the predicted PSF locations and enhance localization precision. Our study underscores the efficacy of CNN in addressing clustering challenges in SMLM, thereby advancing spatial resolution and enabling deeper insights into complex biological structures.

Neural space-time model for dynamic multi-shot imaging

Computational imaging reconstructions from multiple measurements that are captured sequentially often suffer from motion artifacts if the scene is dynamic. We propose a neural space-time model (NSTM) that jointly estimates the scene and its motion dynamics, without data priors or pre-training. Hence, we can both remove motion artifacts and resolve sample dynamics from the same set of raw measurements used for the conventional reconstruction. We demonstrate NSTM in three computational imaging systems: differential phase-contrast microscopy, three-dimensional structured illumination microscopy and rolling-shutter DiffuserCam. We show that NSTM can recover subcellular motion dynamics and thus reduce the misinterpretation of living systems caused by motion artifacts.

Whole-cell multi-target single-molecule super-resolution imaging in 3D with microfluidics and a single-objective tilted light sheet

Multi-target single-molecule super-resolution fluorescence microscopy offers a powerful means of understanding the distributions and interplay between multiple subcellular structures at the nanoscale. However, single-molecule super-resolution imaging of whole mammalian cells is often hampered by high fluorescence background and slow acquisition speeds, especially when imaging multiple targets in 3D. In this work, we have mitigated these issues by developing a steerable, dithered, single-objective tilted light sheet for optical sectioning to reduce fluorescence background and a pipeline for 3D nanoprinting microfluidic systems for reflection of the light sheet into the sample. This easily adaptable novel microfluidic fabrication pipeline allows for the incorporation of reflective optics into microfluidic channels without disrupting efficient and automated solution exchange. By combining these innovations with point spread function engineering for nanoscale localization of individual molecules in 3D, deep learning for analysis of overlapping emitters, active 3D stabilization for drift correction and long-term imaging, and Exchange-PAINT for sequential multi-target imaging without chromatic offsets, we demonstrate whole-cell multi-target 3D single-molecule super-resolution imaging with improved precision and imaging speed.

Computational Methods for Image Analysis in Craniofacial Development and Disease

Observation is at the center of all biological sciences. Advances in imaging technologies are therefore essential to derive novel biological insights to better understand the complex workings of living systems. Recent high-throughput sequencing and imaging techniques are allowing researchers to simultaneously address complex molecular variations spatially and temporarily in tissues and organs. The availability of increasingly large dataset sizes has allowed for the evolution of robust deep learning models, designed to interrogate biomedical imaging data. These models are emerging as transformative tools in diagnostic medicine. Combined, these advances allow for dynamic, quantitative, and predictive observations of entire organisms and tissues. Here, we address 3 main tasks of bioimage analysis, image restoration, segmentation, and tracking and discuss new computational tools allowing for 3-dimensional spatial genomics maps. Finally, we demonstrate how these advances have been applied in studies of craniofacial development and oral disease pathogenesis.

The actin-spectrin submembrane scaffold restricts endocytosis along proximal axons

Clathrin-mediated endocytosis has characteristic features in neuronal dendrites and presynapses, but how membrane proteins are internalized along the axon shaft remains unclear. We focused on clathrin-coated structures and endocytosis along the axon initial segment (AIS) and their relationship to the periodic actin-spectrin scaffold that lines the axonal plasma membrane. A combination of super-resolution microscopy and platinum-replica electron microscopy on cultured neurons revealed that AIS clathrin-coated pits form within "clearings", circular areas devoid of actin-spectrin mesh. Actin-spectrin scaffold disorganization increased clathrin-coated pit formation. Cargo uptake and live-cell imaging showed that AIS clathrin-coated pits are particularly stable. Neuronal plasticity-inducing stimulation triggered internalization of the clathrin-coated pits through polymerization of branched actin around them. Thus, spectrin and actin regulate clathrin-coated pit formation and scission to control endocytosis at the AIS.

Visualizing DNA/RNA, Proteins, and Small Molecule Metabolites within Live Cells

Live cell imaging is essential for obtaining spatial and temporal insights into dynamic molecular events within heterogeneous individual cells, in situ intracellular networks, and in vivo organisms. Molecular tracking in live cells is also a critical and general requirement for studying dynamic physiological processes in cell biology, cancer, developmental biology, and neuroscience. Alongside this context, this review provides a comprehensive overview of recent research progress in live-cell imaging of RNAs, DNAs, proteins, and small-molecule metabolites, as well as their applications in molecular diagnosis, immunodiagnosis, and biochemical diagnosis. A series of advanced live-cell imaging techniques have been introduced and summarized, including high-precision live-cell imaging, high-resolution imaging, low-abundance imaging, multidimensional imaging, multipath imaging, rapid imaging, and computationally driven live-cell imaging methods, all of which offer valuable insights for disease prevention, diagnosis, and treatment. This review article also addresses the current challenges, potential solutions, and future development prospects in this field.

Long axial-range double-helix point spread functions for 3D volumetric super-resolution imaging

Single-molecule localization microscopy (SMLM) is a powerful tool for observing structures beyond the diffraction limit of light. Combining SMLM with engineered point spread functions (PSFs) enables 3D imaging over an extended axial range, as has been demonstrated for super-resolution imaging of various cellular structures. However, super-resolving structures in 3D in thick samples, such as whole mammalian cells, remains challenging as it typically requires acquisition and post-processing stitching of multiple slices to cover the entire sample volume or more complex analysis of the data. Here, we demonstrate how the imaging and analysis workflows can be simplified by 3D single-molecule super-resolution imaging with long axial-range double-helix (DH)-PSFs. First, we experimentally benchmark the localization precisions of short- and long axial-range DH-PSFs at different signal-to-background ratios by imaging of fluorescent beads. The performance of the DH-PSFs in terms of achievable resolution and imaging speed was then quantified for 3D single-molecule super-resolution imaging of mammalian cells by DNA-PAINT imaging of the nuclear lamina protein lamin B1 in U-2 OS cells. Furthermore, we demonstrate how the use of a deep learning-based algorithm allows the localization of dense emitters, drastically improving the achievable imaging speed and resolution. Our data demonstrate that using long axial-range DH-PSFs offers stitching-free, 3D super-resolution imaging of whole mammalian cells, simplifying the experimental and analysis procedures for obtaining volumetric nanoscale structural information.

Highly sensitive volumetric single-molecule imaging

Volumetric subcellular imaging has long been essential for studying structures and dynamics in cells and tissues. However, due to limited imaging speed and depth of field, it has been challenging to perform live-cell imaging and single-particle tracking. Here we report a 2.5D fluorescence microscopy combined with highly inclined illumination beams, which significantly reduce not only the image acquisition time but also the out-of-focus background by ∼2-fold compared to epi-illumination. Instead of sequential z-scanning, our method projects a certain depth of volumetric information onto a 2D plane in a single shot using multi-layered glass for incoherent wavefront splitting, enabling high photon detection efficiency. We apply our method to multi-color immunofluorescence imaging and volumetric super-resolution imaging, covering ∼3-4 µm thickness of samples without z-scanning. Additionally, we demonstrate that our approach can substantially extend the observation time of single-particle tracking in living cells.

Deep 3D histology powered by tissue clearing, omics and AI

To comprehensively understand tissue and organism physiology and pathophysiology, it is essential to create complete three-dimensional (3D) cellular maps. These maps require structural data, such as the 3D configuration and positioning of tissues and cells, and molecular data on the constitution of each cell, spanning from the DNA sequence to protein expression. While single-cell transcriptomics is illuminating the cellular and molecular diversity across species and tissues, the 3D spatial context of these molecular data is often overlooked. Here, I discuss emerging 3D tissue histology techniques that add the missing third spatial dimension to biomedical research. Through innovations in tissue-clearing chemistry, labeling and volumetric imaging that enhance 3D reconstructions and their synergy with molecular techniques, these technologies will provide detailed blueprints of entire organs or organisms at the cellular level. Machine learning, especially deep learning, will be essential for extracting meaningful insights from the vast data. Further development of integrated structural, molecular and computational methods will unlock the full potential of next-generation 3D histology.

Inspiring a convergent engineering approach to measure and model the tissue microenvironment

Understanding the molecular and physical complexity of the tissue microenvironment (TiME) in the context of its spatiotemporal organization has remained an enduring challenge. Recent advances in engineering and data science are now promising the ability to study the structure, functions, and dynamics of the TiME in unprecedented detail; however, many advances still occur in silos that rarely integrate information to study the TiME in its full detail. This review provides an integrative overview of the engineering principles underlying chemical, optical, electrical, mechanical, and computational science to probe, sense, model, and fabricate the TiME. In individual sections, we first summarize the underlying principles, capabilities, and scope of emerging technologies, the breakthrough discoveries enabled by each technology and recent, promising innovations. We provide perspectives on the potential of these advances in answering critical questions about the TiME and its role in various disease and developmental processes. Finally, we present an integrative view that appreciates the major scientific and educational aspects in the study of the TiME.

Nanoscale insights into hematology: super-resolved imaging on blood cell structure, function, and pathology

Fluorescence nanoscopy, also known as super-resolution microscopy, has transcended the conventional resolution barriers and enabled visualization of biological samples at nanometric resolutions. A series of super-resolution techniques have been developed and applied to investigate the molecular distribution, organization, and interactions in blood cells, as well as the underlying mechanisms of blood-cell-associated diseases. In this review, we provide an overview of various fluorescence nanoscopy technologies, outlining their current development stage and the challenges they are facing in terms of functionality and practicality. We specifically explore how these innovations have propelled forward the analysis of thrombocytes (platelets), erythrocytes (red blood cells) and leukocytes (white blood cells), shedding light on the nanoscale arrangement of subcellular components and molecular interactions. We spotlight novel biomarkers uncovered by fluorescence nanoscopy for disease diagnosis, such as thrombocytopathies, malignancies, and infectious diseases. Furthermore, we discuss the technological hurdles and chart out prospective avenues for future research directions. This review aims to underscore the significant contributions of fluorescence nanoscopy to the field of blood cell analysis and disease diagnosis, poised to revolutionize our approach to exploring, understanding, and managing disease at the molecular level.

Advanced Design of Soft Robots with Artificial Intelligence

A comprehensive review focused on the whole systems of the soft robotics with artificial intelligence, which can feel, think, react and interact with humans, is presented. The design strategies concerning about various aspects of the soft robotics, like component materials, device structures, prepared technologies, integrated method, and potential applications, are summarized. A broad outlook on the future considerations for the soft robots is proposed.

In recent years, breakthrough has been made in the field of artificial intelligence (AI), which has also revolutionized the industry of robotics. Soft robots featured with high-level safety, less weight, lower power consumption have always been one of the research hotspots. Recently, multifunctional sensors for perception of soft robotics have been rapidly developed, while more algorithms and models of machine learning with high accuracy have been optimized and proposed. Designs of soft robots with AI have also been advanced ranging from multimodal sensing, human–machine interaction to effective actuation in robotic systems. Nonetheless, comprehensive reviews concerning the new developments and strategies for the ingenious design of the soft robotic systems equipped with AI are rare. Here, the new development is systematically reviewed in the field of soft robots with AI. First, background and mechanisms of soft robotic systems are briefed, after which development focused on how to endow the soft robots with AI, including the aspects of feeling, thought and reaction, is illustrated. Next, applications of soft robots with AI are systematically summarized and discussed together with advanced strategies proposed for performance enhancement. Design thoughts for future intelligent soft robotics are pointed out. Finally, some perspectives are put forward.

Super-resolution microscopies, technological breakthrough to decipher mitochondrial structure and dynamic

Mitochondria are complex organelles with an outer membrane enveloping a second inner membrane that creates a vast matrix space partitioned by pockets or cristae that join the peripheral inner membrane with several thin junctions. Several micrometres long, mitochondria are generally close to 300 nm in diameter, with membrane layers separated by a few tens of nanometres. Ultrastructural data from electron microscopy revealed the structure of these mitochondria, while conventional optical microscopy revealed their extraordinary dynamics through fusion, fission, and migration processes but its limited resolution power restricted the possibility to go further. By overcoming the limits of light diffraction, Super-Resolution Microscopy (SRM) now offers the potential to establish the links between the ultrastructure and remodelling of mitochondrial membranes, leading to major advances in our understanding of mitochondria's structure-function. Here we review the contributions of SRM imaging to our understanding of the relationship between mitochondrial structure and function. What are the hopes for these new imaging approaches which are particularly important for mitochondrial pathologies?

Universal inverse modeling of point spread functions for SMLM localization and microscope characterization

The point spread function (PSF) of a microscope describes the image of a point emitter. Knowing the accurate PSF model is essential for various imaging tasks, including single-molecule localization, aberration correction and deconvolution. Here we present universal inverse modeling of point spread functions (uiPSF), a toolbox to infer accurate PSF models from microscopy data, using either image stacks of fluorescent beads or directly images of blinking fluorophores, the raw data in single-molecule localization microscopy (SMLM). Our modular framework is applicable to a variety of microscope modalities and the PSF model incorporates system- or sample-specific characteristics, for example, the bead size, field- and depth- dependent aberrations, and transformations among channels. We demonstrate its application in single or multiple channels or large field-of-view SMLM systems, 4Pi-SMLM, and lattice light-sheet microscopes using either bead data or single-molecule blinking data.

Live-cell imaging powered by computation

The proliferation of microscopy methods for live-cell imaging offers many new possibilities for users but can also be challenging to navigate. The prevailing challenge in live-cell fluorescence microscopy is capturing intra-cellular dynamics while preserving cell viability. Computational methods can help to address this challenge and are now shifting the boundaries of what is possible to capture in living systems. In this Review, we discuss these computational methods focusing on artificial intelligence-based approaches that can be layered on top of commonly used existing microscopies as well as hybrid methods that integrate computation and microscope hardware. We specifically discuss how computational approaches can improve the signal-to-noise ratio, spatial resolution, temporal resolution and multi-colour capacity of live-cell imaging.

Temporal analysis of relative distances (TARDIS) is a robust, parameter-free alternative to single-particle tracking

In single-particle tracking, individual particles are localized and tracked over time to probe their diffusion and molecular interactions. Temporal crossing of trajectories, blinking particles, and false-positive localizations present computational challenges that have remained difficult to overcome. Here we introduce a robust, parameter-free alternative to single-particle tracking: temporal analysis of relative distances (TARDIS). In TARDIS, an all-to-all distance analysis between localizations is performed with increasing temporal shifts. These pairwise distances represent either intraparticle distances originating from the same particle, or interparticle distances originating from unrelated particles, and are fitted analytically to obtain quantitative measures on particle dynamics. We showcase that TARDIS outperforms tracking algorithms, benchmarked on simulated and experimental data of varying complexity. We further show that TARDIS performs accurately in complex conditions characterized by high particle density, strong emitter blinking or false-positive localizations, and is in fact limited by the capabilities of localization algorithms. TARDIS' robustness enables fivefold shorter measurements without loss of information.

Deep learning-based spectroscopic single-molecule localization microscopy

Significance

Spectroscopic single-molecule localization microscopy (sSMLM) takes advantage of nanoscopy and spectroscopy, enabling sub-10 nm resolution as well as simultaneous multicolor imaging of multi-labeled samples. Reconstruction of raw sSMLM data using deep learning is a promising approach for visualizing the subcellular structures at the nanoscale.

Aim

Develop a novel computational approach leveraging deep learning to reconstruct both label-free and fluorescence-labeled sSMLM imaging data.

Approach

We developed a two-network-model based deep learning algorithm, termed DsSMLM, to reconstruct sSMLM data. The effectiveness of DsSMLM was assessed by conducting imaging experiments on diverse samples, including label-free single-stranded DNA (ssDNA) fiber, fluorescence-labeled histone markers on COS-7 and U2OS cells, and simultaneous multicolor imaging of synthetic DNA origami nanoruler.

Results

For label-free imaging, a spatial resolution of 6.22 nm was achieved on ssDNA fiber; for fluorescence-labeled imaging, DsSMLM revealed the distribution of chromatin-rich and chromatin-poor regions defined by histone markers on the cell nucleus and also offered simultaneous multicolor imaging of nanoruler samples, distinguishing two dyes labeled in three emitting points with a separation distance of 40 nm. With DsSMLM, we observed enhanced spectral profiles with 8.8% higher localization detection for single-color imaging and up to 5.05% higher localization detection for simultaneous two-color imaging.

Conclusions

We demonstrate the feasibility of deep learning-based reconstruction for sSMLM imaging applicable to label-free and fluorescence-labeled sSMLM imaging data. We anticipate our technique will be a valuable tool for high-quality super-resolution imaging for a deeper understanding of DNA molecules' photophysics and will facilitate the investigation of multiple nanoscopic cellular structures and their interactions.

Photophysical image analysis: Unsupervised probabilistic thresholding for images from electron-multiplying charge-coupled devices

We introduce the concept photophysical image analysis (PIA) and an associated pipeline for unsupervised probabilistic image thresholding for images recorded by electron-multiplying charge-coupled device (EMCCD) cameras. We base our approach on a closed-form analytic expression for the characteristic function (Fourier-transform of the probability mass function) for the image counts recorded in an EMCCD camera, which takes into account both stochasticity in the arrival of photons at the imaging camera and subsequent noise induced by the detection system of the camera. The only assumption in our method is that the background photon arrival to the imaging system is described by a stationary Poisson process (we make no assumption about the photon statistics for the signal). We estimate the background photon statistics parameter, λbg, from an image which contains both background and signal pixels by use of a novel truncated fit procedure with an automatically determined image count threshold. Prior to this, the camera noise model parameters are estimated using a calibration step. Utilizing the estimates for the camera parameters and λbg, we then introduce a probabilistic thresholding method, where, for the first time, the fraction of misclassified pixels can be determined a priori for a general image in an unsupervised way. We use synthetic images to validate our a priori estimates and to benchmark against the Otsu method, which is a popular unsupervised non-probabilistic image thresholding method (no a priori estimates for the error rates are provided). For completeness, we lastly present a simple heuristic general-purpose segmentation method based on the thresholding results, which we apply to segmentation of synthetic images and experimental images of fluorescent beads and lung cell nuclei. Our publicly available software opens up for fully automated, unsupervised, probabilistic photophysical image analysis.

Context-aware deep learning enables high-efficacy localization of high concentration microbubbles for super-resolution ultrasound localization microscopy

Ultrasound localization microscopy (ULM) enables deep tissue microvascular imaging by localizing and tracking intravenously injected microbubbles circulating in the bloodstream. However, conventional localization techniques require spatially isolated microbubbles, resulting in prolonged imaging time to obtain detailed microvascular maps. Here, we introduce LOcalization with Context Awareness (LOCA)-ULM, a deep learning-based microbubble simulation and localization pipeline designed to enhance localization performance in high microbubble concentrations. In silico, LOCA-ULM enhanced microbubble detection accuracy to 97.8% and reduced the missing rate to 23.8%, outperforming conventional and deep learning-based localization methods up to 17.4% in accuracy and 37.6% in missing rate reduction. In in vivo rat brain imaging, LOCA-ULM revealed dense cerebrovascular networks and spatially adjacent microvessels undetected by conventional ULM. We further demonstrate the superior localization performance of LOCA-ULM in functional ULM (fULM) where LOCA-ULM significantly increased the functional imaging sensitivity of fULM to hemodynamic responses invoked by whisker stimulations in the rat brain.

Large-FOV 3D localization microscopy by spatially variant point spread function generation

Accurate characterization of the microscopic point spread function (PSF) is crucial for achieving high-performance localization microscopy (LM). Traditionally, LM assumes a spatially invariant PSF to simplify the modeling of the imaging system. However, for large fields of view (FOV) imaging, it becomes important to account for the spatially variant nature of the PSF. Here, we propose an accurate and fast principal components analysis-based field-dependent 3D PSF generator (PPG3D) and localizer for LM. Through simulations and experimental three-dimensional (3D) single-molecule localization microscopy (SMLM), we demonstrate the effectiveness of PPG3D, enabling super-resolution imaging of mitochondria and microtubules with high fidelity over a large FOV. A comparison of PPG3D with a shift-variant PSF generator for 3D LM reveals a threefold improvement in accuracy. Moreover, PPG3D is approximately 100 times faster than existing PSF generators, when used in image plane-based interpolation mode. Given its user-friendliness, we believe that PPG3D holds great potential for widespread application in SMLM and other imaging modalities.

High-density volumetric super-resolution microscopy

Volumetric super-resolution microscopy typically encodes the 3D position of single-molecule fluorescence into a 2D image by changing the shape of the point spread function (PSF) as a function of depth. However, the resulting large and complex PSF spatial footprints reduce biological throughput and applicability by requiring lower labeling densities to avoid overlapping fluorescent signals. We quantitatively compare the density dependence of single-molecule light field microscopy (SMLFM) to other 3D PSFs (astigmatism, double helix and tetrapod) showing that SMLFM enables an order-of-magnitude speed improvement compared to the double helix PSF by resolving overlapping emitters through parallax. We demonstrate this optical robustness experimentally with high accuracy ( > 99.2 ± 0.1%, 0.1 locs μm-2) and sensitivity ( > 86.6 ± 0.9%, 0.1 locs μm-2) through whole-cell (scan-free) imaging and tracking of single membrane proteins in live primary B cells. We also exemplify high-density volumetric imaging (0.15 locs μm-2) in dense cytosolic tubulin datasets.

HormoNet: a deep learning approach for hormone-drug interaction prediction

Several experimental evidences have shown that the human endogenous hormones can interact with drugs in many ways and affect drug efficacy. The hormone drug interactions (HDI) are essential for drug treatment and precision medicine; therefore, it is essential to understand the hormone-drug associations. Here, we present HormoNet to predict the HDI pairs and their risk level by integrating features derived from hormone and drug target proteins. To the best of our knowledge, this is one of the first attempts to employ deep learning approach for prediction of HDI prediction. Amino acid composition and pseudo amino acid composition were applied to represent target information using 30 physicochemical and conformational properties of the proteins. To handle the imbalance problem in the data, we applied synthetic minority over-sampling technique technique. Additionally, we constructed novel datasets for HDI prediction and the risk level of their interaction. HormoNet achieved high performance on our constructed hormone-drug benchmark datasets. The results provide insights into the understanding of the relationship between hormone and a drug, and indicate the potential benefit of reducing risk levels of interactions in designing more effective therapies for patients in drug treatments. Our benchmark datasets and the source codes for HormoNet are available in: https://github.com/EmamiNeda/HormoNet .

Harnessing artificial intelligence to reduce phototoxicity in live imaging

Fluorescence microscopy is essential for studying living cells, tissues and organisms. However, the fluorescent light that switches on fluorescent molecules also harms the samples, jeopardizing the validity of results - particularly in techniques such as super-resolution microscopy, which demands extended illumination. Artificial intelligence (AI)-enabled software capable of denoising, image restoration, temporal interpolation or cross-modal style transfer has great potential to rescue live imaging data and limit photodamage. Yet we believe the focus should be on maintaining light-induced damage at levels that preserve natural cell behaviour. In this Opinion piece, we argue that a shift in role for AIs is needed - AI should be used to extract rich insights from gentle imaging rather than recover compromised data from harsh illumination. Although AI can enhance imaging, our ultimate goal should be to uncover biological truths, not just retrieve data. It is essential to prioritize minimizing photodamage over merely pushing technical limits. Our approach is aimed towards gentle acquisition and observation of undisturbed living systems, aligning with the essence of live-cell fluorescence microscopy.

Image restoration of degraded time-lapse microscopy data mediated by near-infrared imaging

Time-lapse fluorescence microscopy is key to unraveling biological development and function; however, living systems, by their nature, permit only limited interrogation and contain untapped information that can only be captured by more invasive methods. Deep-tissue live imaging presents a particular challenge owing to the spectral range of live-cell imaging probes/fluorescent proteins, which offer only modest optical penetration into scattering tissues. Herein, we employ convolutional neural networks to augment live-imaging data with deep-tissue images taken on fixed samples. We demonstrate that convolutional neural networks may be used to restore deep-tissue contrast in GFP-based time-lapse imaging using paired final-state datasets acquired using near-infrared dyes, an approach termed InfraRed-mediated Image Restoration (IR2). Notably, the networks are remarkably robust over a wide range of developmental times. We employ IR2 to enhance the information content of green fluorescent protein time-lapse images of zebrafish and Drosophila embryo/larval development and demonstrate its quantitative potential in increasing the fidelity of cell tracking/lineaging in developing pescoids. Thus, IR2 is poised to extend live imaging to depths otherwise inaccessible.

Noise learning of instruments for high-contrast, high-resolution and fast hyperspectral microscopy and nanoscopy

The low scattering efficiency of Raman scattering makes it challenging to simultaneously achieve good signal-to-noise ratio (SNR), high imaging speed, and adequate spatial and spectral resolutions. Here, we report a noise learning (NL) approach that estimates the intrinsic noise distribution of each instrument by statistically learning the noise in the pixel-spatial frequency domain. The estimated noise is then removed from the noisy spectra. This enhances the SNR by ca. 10 folds, and suppresses the mean-square error by almost 150 folds. NL allows us to improve the positioning accuracy and spatial resolution and largely eliminates the impact of thermal drift on tip-enhanced Raman spectroscopic nanoimaging. NL is also applicable to enhance SNR in fluorescence and photoluminescence imaging. Our method manages the ground truth spectra and the instrumental noise simultaneously within the training dataset, which bypasses the tedious labelling of huge dataset required in conventional deep learning, potentially shifting deep learning from sample-dependent to instrument-dependent.

Super-Resolution Microscopy of the Bacterial Cell Wall Labeled by Fluorescent D-Amino Acids

Fluorescent D-amino acids (FDAAs) enable in situ visualization of bacterial cell wall synthesis via their incorporation into peptidoglycan (PG) crosslinks. When combined with super-resolution microscopy, FDAAs allow the details of cell wall synthesis to be resolved beyond the diffraction limit of visible light. Here, we describe using the super-resolution method of single-molecule localization microscopy (SMLM) in conjunction with two newly synthesized FDAAs (sCy5DA and sCy5DL_amide) to resolve bacterial PG at the nanoscale in a variety of species, including Gram-negative, Gram-positive, and mycobacteria.

Advanced optical imaging for the rational design of nanomedicines

Despite the enormous potential of nanomedicines to shape the future of medicine, their clinical translation remains suboptimal. Translational challenges are present in every step of the development pipeline, from a lack of understanding of patient heterogeneity to insufficient insights on nanoparticle properties and their impact on material-cell interactions. Here, we discuss how the adoption of advanced optical microscopy techniques, such as super-resolution optical microscopies, correlative techniques, and high-content modalities, could aid the rational design of nanocarriers, by characterizing the cell, the nanomaterial, and their interaction with unprecedented spatial and/or temporal detail. In this nanomedicine arena, we will discuss how the implementation of these techniques, with their versatility and specificity, can yield high volumes of multi-parametric data; and how machine learning can aid the rapid advances in microscopy: from image acquisition to data interpretation.

Multiplexed and Millimeter-Scale Fluorescence Nanoscopy of Cells and Tissue Sections via Prism-Illumination and Microfluidics-Enhanced DNA-PAINT

Fluorescence nanoscopy has become increasingly powerful for biomedical research, but it has historically afforded a small field-of-view (FOV) of around 50 μm × 50 μm at once and more recently up to ∼200 μm × 200 μm. Efforts to further increase the FOV in fluorescence nanoscopy have thus far relied on the use of fabricated waveguide substrates, adding cost and sample constraints to the applications. Here we report PRism-Illumination and Microfluidics-Enhanced DNA-PAINT (PRIME-PAINT) for multiplexed fluorescence nanoscopy across millimeter-scale FOVs. Built upon the well-established prism-type total internal reflection microscopy, PRIME-PAINT achieves robust single-molecule localization with up to ∼520 μm × 520 μm single FOVs and 25-40 nm lateral resolutions. Through stitching, nanoscopic imaging over mm2 sample areas can be completed in as little as 40 min per target. An on-stage microfluidics chamber facilitates probe exchange for multiplexing and enhances image quality, particularly for formalin-fixed paraffin-embedded (FFPE) tissue sections. We demonstrate the utility of PRIME-PAINT by analyzing ∼106 caveolae structures in ∼1,000 cells and imaging entire pancreatic cancer lesions from patient tissue biopsies. By imaging from nanometers to millimeters with multiplexity and broad sample compatibility, PRIME-PAINT will be useful for building multiscale, Google-Earth-like views of biological systems.

Live-cell imaging in the deep learning era

Live imaging is a powerful tool, enabling scientists to observe living organisms in real time. In particular, when combined with fluorescence microscopy, live imaging allows the monitoring of cellular components with high sensitivity and specificity. Yet, due to critical challenges (i.e., drift, phototoxicity, dataset size), implementing live imaging and analyzing the resulting datasets is rarely straightforward. Over the past years, the development of bioimage analysis tools, including deep learning, is changing how we perform live imaging. Here we briefly cover important computational methods aiding live imaging and carrying out key tasks such as drift correction, denoising, super-resolution imaging, artificial labeling, tracking, and time series analysis. We also cover recent advances in self-driving microscopy.

High-fidelity 3D live-cell nanoscopy through data-driven enhanced super-resolution radial fluctuation

Live-cell super-resolution microscopy enables the imaging of biological structure dynamics below the diffraction limit. Here we present enhanced super-resolution radial fluctuations (eSRRF), substantially improving image fidelity and resolution compared to the original SRRF method. eSRRF incorporates automated parameter optimization based on the data itself, giving insight into the trade-off between resolution and fidelity. We demonstrate eSRRF across a range of imaging modalities and biological systems. Notably, we extend eSRRF to three dimensions by combining it with multifocus microscopy. This realizes live-cell volumetric super-resolution imaging with an acquisition speed of ~1 volume per second. eSRRF provides an accessible super-resolution approach, maximizing information extraction across varied experimental conditions while minimizing artifacts. Its optimal parameter prediction strategy is generalizable, moving toward unbiased and optimized analyses in super-resolution microscopy.

DBlink: dynamic localization microscopy in super spatiotemporal resolution via deep learning

Single-molecule localization microscopy (SMLM) has revolutionized biological imaging, improving the spatial resolution of traditional microscopes by an order of magnitude. However, SMLM techniques require long acquisition times, typically a few minutes, to yield a single super-resolved image, because they depend on accumulation of many localizations over thousands of recorded frames. Hence, the capability of SMLM to observe dynamics at high temporal resolution has always been limited. In this work, we present DBlink, a deep-learning-based method for super spatiotemporal resolution reconstruction from SMLM data. The input to DBlink is a recorded video of SMLM data and the output is a super spatiotemporal resolution video reconstruction. We use a convolutional neural network combined with a bidirectional long short-term memory network architecture, designed for capturing long-term dependencies between different input frames. We demonstrate DBlink performance on simulated filaments and mitochondria-like structures, on experimental SMLM data under controlled motion conditions and on live-cell dynamic SMLM. DBlink's spatiotemporal interpolation constitutes an important advance in super-resolution imaging of dynamic processes in live cells.

LocNet: deep learning-based localization on a rotating point spread function with applications to telescope imaging

Three-dimensional (3D) point source recovery from two-dimensional (2D) data is a challenging problem with wide-ranging applications in single-molecule localization microscopy and space-debris localization telescops. Point spread function (PSF) engineering is a promising technique to solve this 3D localization problem. Specifically, we consider the problem of 3D localization of space debris from a 2D image using a rotating PSF where the depth information is encoded in the angle of rotation of a single-lobe PSF for each point source. Instead of applying a model-based optimization, we introduce a convolution neural network (CNN)-based approach to localize space debris in full 3D space automatically. A hard sample training strategy is proposed to improve the performance of CNN further. Contrary to the traditional model-based methods, our technique is efficient and outperforms the current state-of-the-art method by more than 11% in the precision rate with a comparable improvement in the recall rate.

Polarization differential interference contrast microscopy with physics-inspired plug-and-play denoiser for single-shot high-performance quantitative phase imaging

We present single-shot high-performance quantitative phase imaging with a physics-inspired plug-and-play denoiser for polarization differential interference contrast (PDIC) microscopy. The quantitative phase is recovered by the alternating direction method of multipliers (ADMM), balancing total variance regularization and a pre-trained dense residual U-net (DRUNet) denoiser. The custom DRUNet uses the Tanh activation function to guarantee the symmetry requirement for phase retrieval. In addition, we introduce an adaptive strategy accelerating convergence and explicitly incorporating measurement noise. After validating this deep denoiser-enhanced PDIC microscopy on simulated data and phantom experiments, we demonstrated high-performance phase imaging of histological tissue sections. The phase retrieval by the denoiser-enhanced PDIC microscopy achieves significantly higher quality and accuracy than the solution based on Fourier transforms or the iterative solution with total variance regularization alone.

An integrated platform for high-throughput nanoscopy

Single-molecule localization microscopy enables three-dimensional fluorescence imaging at tens-of-nanometer resolution, but requires many camera frames to reconstruct a super-resolved image. This limits the typical throughput to tens of cells per day. While frame rates can now be increased by over an order of magnitude, the large data volumes become limiting in existing workflows. Here we present an integrated acquisition and analysis platform leveraging microscopy-specific data compression, distributed storage and distributed analysis to enable an acquisition and analysis throughput of 10,000 cells per day. The platform facilitates graphically reconfigurable analyses to be automatically initiated from the microscope during acquisition and remotely executed, and can even feed back and queue new acquisition tasks on the microscope. We demonstrate the utility of this framework by imaging hundreds of cells per well in multi-well sample formats. Our platform, implemented within the PYthon-Microscopy Environment (PYME), is easily configurable to control custom microscopes, and includes a plugin framework for user-defined extensions.

Universal inverse modelling of point spread functions for SMLM localization and microscope characterization

The point spread function (PSF) of a microscope describes the image of a point emitter. Knowing the accurate PSF model is essential for various imaging tasks, including single molecule localization, aberration correction and deconvolution. Here we present uiPSF (universal inverse modelling of Point Spread Functions), a toolbox to infer accurate PSF models from microscopy data, using either image stacks of fluorescent beads or directly images of blinking fluorophores, the raw data in single molecule localization microscopy (SMLM). The resulting PSF model enables accurate 3D super-resolution imaging using SMLM. Additionally, uiPSF can be used to characterize and optimize a microscope system by quantifying the aberrations, including field-dependent aberrations, and resolutions. Our modular framework is applicable to a variety of microscope modalities and the PSF model incorporates system or sample specific characteristics, e.g., the bead size, depth dependent aberrations and transformations among channels. We demonstrate its application in single or multiple channels or large field-of-view SMLM systems, 4Pi-SMLM, and lattice light-sheet microscopes using either bead data or single molecule blinking data.

Presynapses contain distinct actin nanostructures

The architecture of the actin cytoskeleton that concentrates at presynapses remains poorly known, hindering our understanding of its roles in synaptic physiology. In this work, we measure and visualize presynaptic actin by diffraction-limited and super-resolution microscopy, thanks to a validated model of bead-induced presynapses in cultured neurons. We identify a major population of actin-enriched presynapses that concentrates more presynaptic components and shows higher synaptic vesicle cycling than their non-enriched counterparts. Pharmacological perturbations point to an optimal actin amount and the presence of distinct actin structures within presynapses. We directly visualize these nanostructures using Single Molecule Localization Microscopy (SMLM), defining three distinct types: an actin mesh at the active zone, actin rails between the active zone and deeper reserve pools, and actin corrals around the whole presynaptic compartment. Finally, CRISPR-tagging of endogenous actin allows us to validate our results in natural synapses between cultured neurons, confirming the role of actin enrichment and the presence of three types of presynaptic actin nanostructures.

Machine learning for cross-scale microscopy of viruses

Despite advances in virological sciences and antiviral research, viruses continue to emerge, circulate, and threaten public health. We still lack a comprehensive understanding of how cells and individuals remain susceptible to infectious agents. This deficiency is in part due to the complexity of viruses, including the cell states controlling virus-host interactions. Microscopy samples distinct cellular infection stages in a multi-parametric, time-resolved manner at molecular resolution and is increasingly enhanced by machine learning and deep learning. Here we discuss how state-of-the-art artificial intelligence (AI) augments light and electron microscopy and advances virological research of cells. We describe current procedures for image denoising, object segmentation, tracking, classification, and super-resolution and showcase examples of how AI has improved the acquisition and analyses of microscopy data. The power of AI-enhanced microscopy will continue to help unravel virus infection mechanisms, develop antiviral agents, and improve viral vectors.

Simultaneous spectral differentiation of multiple fluorophores in super-resolution imaging using a glass phase plate

By engineering the point-spread function (PSF) of single molecules, different fluorophore species can be imaged simultaneously and distinguished by their unique PSF patterns. Here, we insert a silicon-dioxide phase plate at the Fourier plane of the detection path of a wide-field fluorescence microscope to produce distinguishable PSFs (X-PSFs) at different wavelengths. We demonstrate that the resulting PSFs can be localized spatially and spectrally using a maximum-likelihood estimation algorithm and can be utilized for hyper-spectral super-resolution microscopy of biological samples. We produced superresolution images of fixed U2OS cells using X-PSFs for dSTORM imaging with simultaneous illumination of up to three fluorophore species. The species were distinguished only by the PSF pattern. We achieved ∼21-nm lateral localization precision (FWHM) and ∼17-nm axial precision (FWHM) with an average of 1,800 - 3,500 photons per PSF and a background as high as 130 - 400 photons per pixel. The modified PSF distinguished fluorescent probes with ∼80 nm separation between spectral peaks.

Are Deep Neural Networks Adequate Behavioral Models of Human Visual Perception?

Deep neural networks (DNNs) are machine learning algorithms that have revolutionized computer vision due to their remarkable successes in tasks like object classification and segmentation. The success of DNNs as computer vision algorithms has led to the suggestion that DNNs may also be good models of human visual perception. In this article, we review evidence regarding current DNNs as adequate behavioral models of human core object recognition. To this end, we argue that it is important to distinguish between statistical tools and computational models and to understand model quality as a multidimensional concept in which clarity about modeling goals is key. Reviewing a large number of psychophysical and computational explorations of core object recognition performance in humans and DNNs, we argue that DNNs are highly valuable scientific tools but that, as of today, DNNs should only be regarded as promising-but not yet adequate-computational models of human core object recognition behavior. On the way, we dispel several myths surrounding DNNs in vision science.

Neural network-assisted single-molecule localization microscopy with a weak-affinity protein tag

Single-molecule localization microscopy achieves nanometer spatial resolution by localizing single fluorophores separated in space and time. A major challenge of single-molecule localization microscopy is the long acquisition time, leading to low throughput, as well as to a poor temporal resolution that limits its use to visualize the dynamics of cellular structures in live cells. Another challenge is photobleaching, which reduces information density over time and limits throughput and the available observation time in live-cell applications. To address both challenges, we combine two concepts: first, we integrate the neural network DeepSTORM to predict super-resolution images from high-density imaging data, which increases acquisition speed. Second, we employ a direct protein label, HaloTag7, in combination with exchangeable ligands (xHTLs), for fluorescence labeling. This labeling method bypasses photobleaching by providing a constant signal over time and is compatible with live-cell imaging. The combination of both a neural network and a weak-affinity protein label reduced the acquisition time up to ∼25-fold. Furthermore, we demonstrate live-cell imaging with increased temporal resolution, and capture the dynamics of the endoplasmic reticulum over extended time without signal loss.

Physics-informed neural networks (PINNs) for 4D hemodynamics prediction: An investigation of optimal framework based on vascular morphology

Hemodynamic parameters are of great significance in the clinical diagnosis and treatment of cardiovascular diseases. However, noninvasive, real-time and accurate acquisition of hemodynamics remains a challenge for current invasive detection and simulation algorithms. Here, we integrate computational fluid dynamics with our customized analysis framework based on a multi-attribute point cloud dataset and physics-informed neural networks (PINNs)-aided deep learning modules. This combination is implemented by our workflow that generates flow field datasets within two types of patient personalized models - aorta with fine coronary branches and abdominal aorta. Deep learning modules with or without an antecedent hierarchical structure model the flow field development and complete the mapping from spatial and temporal dimensions to 4D hemodynamics. 88,000 cases on 4 randomized partitions in 16 controlled trials reveal the hemodynamic landscape of spatio-temporal anisotropy within two types of personalized models, which demonstrates the effectiveness of PINN in predicting the space-time behavior of flow fields and gives the optimal deep learning framework for different blood vessels in terms of balancing the training cost and accuracy dimensions. The proposed framework shows intentional performance in computational cost, accuracy and visualization compared to currently prevalent methods, and has the potential for generalization to model flow fields and corresponding clinical metrics within vessels at different locations. We expect our framework to push the 4D hemodynamic predictions to the real-time level, and in statistically significant fashion, applicable to morphologically variable vessels.

Deep-MSIM: Fast Image Reconstruction with Deep Learning in Multifocal Structured Illumination Microscopy

Fast and precise reconstruction algorithm is desired for for multifocal structured illumination microscopy (MSIM) to obtain the super-resolution image. This work proposes a deep convolutional neural network (CNN) to learn a direct mapping from raw MSIM images to super-resolution image, which takes advantage of the computational advances of deep learning to accelerate the reconstruction. The method is validated on diverse biological structures and in vivo imaging of zebrafish at a depth of 100 µm. The results show that high-quality, super-resolution images can be reconstructed in one-third of the runtime consumed by conventional MSIM method, without compromising spatial resolution. Last but not least, a fourfold reduction in the number of raw images required for reconstruction is achieved by using the same network architecture, yet with different training data.

Switchable and Functional Fluorophores for Multidimensional Single-Molecule Localization Microscopy

Multidimensional single-molecule localization microscopy (mSMLM) represents a paradigm shift in the realm of super-resolution microscopy techniques. It affords the simultaneous detection of single-molecule spatial locations at the nanoscale and functional information by interrogating the emission properties of switchable fluorophores. The latter is finely tuned to report its local environment through carefully manipulated laser illumination and single-molecule detection strategies. This Perspective highlights recent strides in mSMLM with a focus on fluorophore designs and their integration into mSMLM imaging systems. Particular interests are the accomplishments in simultaneous multiplexed super-resolution imaging, nanoscale polarity and hydrophobicity mapping, and single-molecule orientational imaging. Challenges and prospects in mSMLM are also discussed, which include the development of more vibrant and functional fluorescent probes, the optimization of optical implementation to judiciously utilize the photon budget, and the advancement of imaging analysis and machine learning techniques.

Advancing Spectrally-Resolved Single Molecule Localization Microscopy with Deep Learning

Spectrally-resolved single molecule localization microscopy (srSMLM) is a recent technique enriching single molecule localization microscopy with the simultaneous recording of spectra of the single emitters. srSMLM resolution is limited by the number of photons collected per emitters. Sharing a photon budget to record the localization and the spectroscopic information results in a loss of spatial and spectral resolution-or forces the sacrifice of one at the expense of the other. Here, srUnet-a deep-learning Unet-based image processing routine trained to increase the spectral and spatial signals to compensate for the resolution loss inherent in additionally recording the spectral component is reported. Both localization and spectral precision are improved by srUnet-particularly for the low-emitting species. srUnet increases the fraction of localization whose signal can be both spatially and spectrally characterized. It preserves spectral shifts and the linearity of the dispersion of light. It strongly facilitates wavelength assignment in multicolor experiments. srUnet is a simple post-processing add-on boosting srSMLM performance close to conventional SMLM with the potential to turn srSMLM into the new standard for multicolor single molecule imaging.

Advanced fluorescence microscopy in respiratory virus cell biology

Respiratory viruses are a major public health burden across all age groups around the globe, and are associated with high morbidity and mortality rates. They can be transmitted by multiple routes, including physical contact or droplets and aerosols, resulting in efficient spreading within the human population. Investigations of the cell biology of virus replication are thus of utmost importance to gain a better understanding of virus-induced pathogenicity and the development of antiviral countermeasures. Light and fluorescence microscopy techniques have revolutionized investigations of the cell biology of virus infection by allowing the study of the localization and dynamics of viral or cellular components directly in infected cells. Advanced microscopy including high- and super-resolution microscopy techniques available today can visualize biological processes at the single-virus and even single-molecule level, thus opening a unique view on virus infection. We will highlight how fluorescence microscopy has supported investigations on virus cell biology by focusing on three major respiratory viruses: respiratory syncytial virus (RSV), Influenza A virus (IAV) and SARS-CoV-2. We will review our current knowledge of virus replication and highlight how fluorescence microscopy has helped to improve our state of understanding. We will start by introducing major imaging and labeling modalities and conclude the chapter with a perspective discussion on remaining challenges and potential opportunities.