Deep Learning Automates the Quantitative Analysis of Individual Cells in Live-Cell Imaging Experiments

In silico labeling in cell biology: Potential and limitations

In silico labeling is the computational cross-modality image translation where the output modality is a subcellular marker that is not specifically encoded in the input image, for example, in silico localization of organelles from transmitted light images. In principle, in silico labeling has the potential to facilitate rapid live imaging of multiple organelles with reduced photobleaching and phototoxicity, a technology enabling a major leap toward understanding the cell as an integrated complex system. However, five years have passed since feasibility was attained, without any demonstration of using in silico labeling to uncover new biological insight. In here, we discuss the current state of in silico labeling, the limitations preventing it from becoming a practical tool, and how we can overcome these limitations to reach its full potential.

Multi-parametric atlas of the pre-metastatic liver for prediction of metastatic outcome in early-stage pancreatic cancer

Metastasis occurs frequently after resection of pancreatic cancer (PaC). In this study, we hypothesized that multi-parametric analysis of pre-metastatic liver biopsies would classify patients according to their metastatic risk, timing and organ site. Liver biopsies obtained during pancreatectomy from 49 patients with localized PaC and 19 control patients with non-cancerous pancreatic lesions were analyzed, combining metabolomic, tissue and single-cell transcriptomics and multiplex imaging approaches. Patients were followed prospectively (median 3 years) and classified into four recurrence groups; early (<6 months after resection) or late (>6 months after resection) liver metastasis (LiM); extrahepatic metastasis (EHM); and disease-free survivors (no evidence of disease (NED)). Overall, PaC livers exhibited signs of augmented inflammation compared to controls. Enrichment of neutrophil extracellular traps (NETs), Ki-67 upregulation and decreased liver creatine significantly distinguished those with future metastasis from NED. Patients with future LiM were characterized by scant T cell lobular infiltration, less steatosis and higher levels of citrullinated H3 compared to patients who developed EHM, who had overexpression of interferon target genes (MX1 and NR1D1) and an increase of CD11B+ natural killer (NK) cells. Upregulation of sortilin-1 and prominent NETs, together with the lack of T cells and a reduction in CD11B+ NK cells, differentiated patients with early-onset LiM from those with late-onset LiM. Liver profiles of NED closely resembled those of controls. Using the above parameters, a machine-learning-based model was developed that successfully predicted the metastatic outcome at the time of surgery with 78% accuracy. Therefore, multi-parametric profiling of liver biopsies at the time of PaC diagnosis may determine metastatic risk and organotropism and guide clinical stratification for optimal treatment selection.

A systematic evaluation of computational methods for cell segmentation

Cell segmentation is a fundamental task in analyzing biomedical images. Many computational methods have been developed for cell segmentation and instance segmentation, but their performances are not well understood in various scenarios. We systematically evaluated the performance of 18 segmentation methods to perform cell nuclei and whole cell segmentation using light microscopy and fluorescence staining images. We found that general-purpose methods incorporating the attention mechanism exhibit the best overall performance. We identified various factors influencing segmentation performances, including image channels, choice of training data, and cell morphology, and evaluated the generalizability of methods across image modalities. We also provide guidelines for choosing the optimal segmentation methods in various real application scenarios. We developed Seggal, an online resource for downloading segmentation models already pre-trained with various tissue and cell types, substantially reducing the time and effort for training cell segmentation models.

Spatial Dissection of the Immune Landscape of Solid Tumors to Advance Precision Medicine

Chemotherapeutics, radiation, targeted therapeutics, and immunotherapeutics each demonstrate clinical benefits for a small subset of patients with solid malignancies. Immune cells infiltrating the tumor and the surrounding stroma play a critical role in shaping cancer progression and modulating therapy response. They do this by interacting with the other cellular and molecular components of the tumor microenvironment. Spatial multi-omics technologies are rapidly evolving. Currently, such technologies allow high-throughput RNA and protein profiling and retain geographical information about the tumor microenvironment cellular architecture and the functional phenotype of tumor, immune, and stromal cells. An in-depth spatial characterization of the heterogeneous tumor immune landscape can improve not only the prognosis but also the prediction of therapy response, directing cancer patients to more tailored and efficacious treatments. This review highlights recent advancements in spatial transcriptomics and proteomics profiling technologies and the ways these technologies are being applied for the dissection of the immune cell composition in solid malignancies in order to further both basic research in oncology and the implementation of precision treatments in the clinic.

Deep 3D histology powered by tissue clearing, omics and AI

To comprehensively understand tissue and organism physiology and pathophysiology, it is essential to create complete three-dimensional (3D) cellular maps. These maps require structural data, such as the 3D configuration and positioning of tissues and cells, and molecular data on the constitution of each cell, spanning from the DNA sequence to protein expression. While single-cell transcriptomics is illuminating the cellular and molecular diversity across species and tissues, the 3D spatial context of these molecular data is often overlooked. Here, I discuss emerging 3D tissue histology techniques that add the missing third spatial dimension to biomedical research. Through innovations in tissue-clearing chemistry, labeling and volumetric imaging that enhance 3D reconstructions and their synergy with molecular techniques, these technologies will provide detailed blueprints of entire organs or organisms at the cellular level. Machine learning, especially deep learning, will be essential for extracting meaningful insights from the vast data. Further development of integrated structural, molecular and computational methods will unlock the full potential of next-generation 3D histology.

Mechanism-based and data-driven modeling in cell-free synthetic biology

Cell-free systems have emerged as a versatile platform in synthetic biology, finding applications in various areas such as prototyping synthetic circuits, biosensor development, and biomanufacturing. To streamline the prototyping process, cell-free systems often incorporate a modeling step that predicts the outcomes of various experimental scenarios, providing a deeper insight into the underlying mechanisms and functions. There are two recognized approaches for modeling these systems: mechanism-based modeling, which models the underlying reaction mechanisms; and data-driven modeling, which makes predictions based on data without preconceived interactions between system components. In this highlight, we focus on the latest advancements in both modeling approaches for cell-free systems, exploring their potential for the design and optimization of synthetic genetic circuits.

Low-frequency ERK and Akt activity dynamics are predictive of stochastic cell division events

Understanding the dynamics of intracellular signaling pathways, such as ERK1/2 (ERK) and Akt1/2 (Akt), in the context of cell fate decisions is important for advancing our knowledge of cellular processes and diseases, particularly cancer. While previous studies have established associations between ERK and Akt activities and proliferative cell fate, the heterogeneity of single-cell responses adds complexity to this understanding. This study employed a data-driven approach to address this challenge, developing machine learning models trained on a dataset of growth factor-induced ERK and Akt activity time courses in single cells, to predict cell division events. The most predictive models were developed by applying discrete wavelet transforms (DWTs) to extract low-frequency features from the time courses, followed by using Ensemble Integration, a data integration and predictive modeling framework. The results demonstrated that these models effectively predicted cell division events in MCF10A cells (F-measure=0.524, AUC=0.726). ERK dynamics were found to be more predictive than Akt, but the combination of both measurements further enhanced predictive performance. The ERK model`s performance also generalized to predicting division events in RPE cells, indicating the potential applicability of these models and our data-driven methodology for predicting cell division across different biological contexts. Interpretation of these models suggested that ERK dynamics throughout the cell cycle, rather than immediately after growth factor stimulation, were associated with the likelihood of cell division. Overall, this work contributes insights into the predictive power of intra-cellular signaling dynamics for cell fate decisions, and highlights the potential of machine learning approaches in unraveling complex cellular behaviors.

Live-cell imaging powered by computation

The proliferation of microscopy methods for live-cell imaging offers many new possibilities for users but can also be challenging to navigate. The prevailing challenge in live-cell fluorescence microscopy is capturing intra-cellular dynamics while preserving cell viability. Computational methods can help to address this challenge and are now shifting the boundaries of what is possible to capture in living systems. In this Review, we discuss these computational methods focusing on artificial intelligence-based approaches that can be layered on top of commonly used existing microscopies as well as hybrid methods that integrate computation and microscope hardware. We specifically discuss how computational approaches can improve the signal-to-noise ratio, spatial resolution, temporal resolution and multi-colour capacity of live-cell imaging.

A Review of Nuclei Detection and Segmentation on Microscopy Images Using Deep Learning With Applications to Unbiased Stereology Counting

The detection and segmentation of stained cells and nuclei are essential prerequisites for subsequent quantitative research for many diseases. Recently, deep learning has shown strong performance in many computer vision problems, including solutions for medical image analysis. Furthermore, accurate stereological quantification of microscopic structures in stained tissue sections plays a critical role in understanding human diseases and developing safe and effective treatments. In this article, we review the most recent deep learning approaches for cell (nuclei) detection and segmentation in cancer and Alzheimer's disease with an emphasis on deep learning approaches combined with unbiased stereology. Major challenges include accurate and reproducible cell detection and segmentation of microscopic images from stained sections. Finally, we discuss potential improvements and future trends in deep learning applied to cell detection and segmentation.

Deep Learning Insights into the Dynamic Effects of Photodynamic Therapy on Cancer Cells

Photodynamic therapy (PDT) shows promise in tumor treatment, particularly when combined with nanotechnology. This study examines the impact of deep learning, particularly the Cellpose algorithm, on the comprehension of cancer cell responses to PDT. The Cellpose algorithm enables robust morphological analysis of cancer cells, while logistic growth modelling predicts cellular behavior post-PDT. Rigorous model validation ensures the accuracy of the findings. Cellpose demonstrates significant morphological changes after PDT, affecting cellular proliferation and survival. The reliability of the findings is confirmed by model validation. This deep learning tool enhances our understanding of cancer cell dynamics after PDT. Advanced analytical techniques, such as morphological analysis and growth modeling, provide insights into the effects of PDT on hepatocellular carcinoma (HCC) cells, which could potentially improve cancer treatment efficacy. In summary, the research examines the role of deep learning in optimizing PDT parameters to personalize oncology treatment and improve efficacy.

Accurate single-molecule spot detection for image-based spatial transcriptomics with weakly supervised deep learning

Image-based spatial transcriptomics methods enable transcriptome-scale gene expression measurements with spatial information but require complex, manually tuned analysis pipelines. We present Polaris, an analysis pipeline for image-based spatial transcriptomics that combines deep-learning models for cell segmentation and spot detection with a probabilistic gene decoder to quantify single-cell gene expression accurately. Polaris offers a unifying, turnkey solution for analyzing spatial transcriptomics data from multiplexed error-robust FISH (MERFISH), sequential fluorescence in situ hybridization (seqFISH), or in situ RNA sequencing (ISS) experiments. Polaris is available through the DeepCell software library (https://github.com/vanvalenlab/deepcell-spots) and https://www.deepcell.org.

"Keep on ROCKIn": Repurposed ROCK inhibitors to boost corneal endothelial regeneration

The global shortage of corneal endothelial graft tissue necessitates the exploration of alternative therapeutic strategies. Rho-associated protein kinase inhibitors (ROCKi), recognized for their regenerative potential in cardiology, oncology, and neurology, have shown promise in corneal endothelial regeneration. This study investigates the repurposing potential of additional ROCKi compounds. Through screening a self-assembled library of ROCKi on B4G12 corneal endothelial cells, we evaluated their dose-dependent effects on proliferation, migration, and toxicity using live-cell imaging. Nine ROCKi candidates significantly enhanced B4G12 proliferation compared to the basal growth rate. These candidates were further assessed for their potential to accelerate wound closure as another indicator for tissue regeneration capacity, with most demonstrating notable efficacy. To assess the potential impact of candidate ROCKi on key corneal endothelial cell markers related to cell proliferation, leaky tight junctions and ion efflux capacity, we analyzed the protein expression of cyclin E1, CDK2, p16, ZO-1 and Na+/K+-ATPase, respectively. Immunocytochemistry and western blot analysis confirmed the preservation of corneal endothelial markers post-treatment with ROCKi hits. However, notable cytoplasm enlargement and nuclear fragmentation were detected after the treatment with SR-3677 and Thiazovivin, indicating possible cellular stress. In compared parameters, Chroman-1 at a concentration of 10 nM outperformed other ROCKi, requiring significantly 1000-fold lower effective concentration than established ROCKi Y-27632 and Fasudil. Altogether, this study underscores the potential of repurposing ROCKi for treating corneal endothelial dysfunctions, offering a viable alternative to conventional grafting methods, and highlights Chroman-1 as a promising candidate structure for hit-to-lead development.

Plasmid-mediated phenotypic noise leads to transient antibiotic resistance in bacteria

The rise of antibiotic resistance is a critical public health concern, requiring an understanding of mechanisms that enable bacteria to tolerate antimicrobial agents. Bacteria use diverse strategies, including the amplification of drug-resistance genes. In this paper, we showed that multicopy plasmids, often carrying antibiotic resistance genes in clinical bacteria, can rapidly amplify genes, leading to plasmid-mediated phenotypic noise and transient antibiotic resistance. By combining stochastic simulations of a computational model with high-throughput single-cell measurements of blaTEM-1 expression in Escherichia coli MG1655, we showed that plasmid copy number variability stably maintains populations composed of cells with both low and high plasmid copy numbers. This diversity in plasmid copy number enhances the probability of bacterial survival in the presence of antibiotics, while also rapidly reducing the burden of carrying multiple plasmids in drug-free environments. Our results further support the tenet that multicopy plasmids not only act as vehicles for the horizontal transfer of genetic information between cells but also as drivers of bacterial adaptation, enabling rapid modulation of gene copy numbers. Understanding the role of multicopy plasmids in antibiotic resistance is critical, and our study provides insights into how bacteria can transiently survive lethal concentrations of antibiotics.

DANCE: a deep learning library and benchmark platform for single-cell analysis

DANCE is the first standard, generic, and extensible benchmark platform for accessing and evaluating computational methods across the spectrum of benchmark datasets for numerous single-cell analysis tasks. Currently, DANCE supports 3 modules and 8 popular tasks with 32 state-of-art methods on 21 benchmark datasets. People can easily reproduce the results of supported algorithms across major benchmark datasets via minimal efforts, such as using only one command line. In addition, DANCE provides an ecosystem of deep learning architectures and tools for researchers to facilitate their own model development. DANCE is an open-source Python package that welcomes all kinds of contributions.

The evolution of metastatic upper tract urothelial carcinoma through genomic-transcriptomic and single-cell protein markers analysis

The molecular characteristics of metastatic upper tract urothelial carcinoma (UTUC) are not well understood, and there is a lack of knowledge regarding the genomic and transcriptomic differences between primary and metastatic UTUC. To address these gaps, we integrate whole-exome sequencing, RNA sequencing, and Imaging Mass Cytometry using lanthanide metal-conjugated antibodies of 44 tumor samples from 28 patients with high-grade primary and metastatic UTUC. We perform a spatially-resolved single-cell analysis of cancer, immune, and stromal cells to understand the evolution of primary to metastatic UTUC. We discover that actionable genomic alterations are frequently discordant between primary and metastatic UTUC tumors in the same patient. In contrast, molecular subtype membership and immune depletion signature are stable across primary and matched metastatic UTUC. Molecular and immune subtypes are consistent between bulk RNA-sequencing and mass cytometry of protein markers from 340,798 single cells. Molecular subtypes at the single-cell level are highly conserved between primary and metastatic UTUC tumors within the same patient.

BetaBuddy: An automated end-to-end computer vision pipeline for analysis of calcium fluorescence dynamics in β-cells

Insulin secretion from pancreatic β-cells is integral in maintaining the delicate equilibrium of blood glucose levels. Calcium is known to be a key regulator and triggers the release of insulin. This sub-cellular process can be monitored and tracked through live-cell imaging and subsequent cell segmentation, registration, tracking, and analysis of the calcium level in each cell. Current methods of analysis typically require the manual outlining of β-cells, involve multiple software packages, and necessitate multiple researchers-all of which tend to introduce biases. Utilizing deep learning algorithms, we have therefore created a pipeline to automatically segment and track thousands of cells, which greatly reduces the time required to gather and analyze a large number of sub-cellular images and improve accuracy. Tracking cells over a time-series image stack also allows researchers to isolate specific calcium spiking patterns and spatially identify those of interest, creating an efficient and user-friendly analysis tool. Using our automated pipeline, a previous dataset used to evaluate changes in calcium spiking activity in β-cells post-electric field stimulation was reanalyzed. Changes in spiking activity were found to be underestimated previously with manual segmentation. Moreover, the machine learning pipeline provides a powerful and rapid computational approach to examine, for example, how calcium signaling is regulated by intracellular interactions.

A Hitchhiker's guide to high-dimensional tissue imaging with multiplexed ion beam imaging

Advancements in multiplexed tissue imaging technologies are vital in shaping our understanding of tissue microenvironmental influences in disease contexts. These technologies now allow us to relate the phenotype of individual cells to their higher-order roles in tissue organization and function. Multiplexed Ion Beam Imaging (MIBI) is one of such technologies, which uses metal isotope-labeled antibodies and secondary ion mass spectrometry (SIMS) to image more than 40 protein markers simultaneously within a single tissue section. Here, we describe an optimized MIBI workflow for high-plex analysis of Formalin-Fixed Paraffin-Embedded (FFPE) tissues following antigen retrieval, metal isotope-conjugated antibody staining, imaging using the MIBI instrument, and subsequent data processing and analysis. While this workflow is focused on imaging human FFPE samples using the MIBI, this workflow can be easily extended to model systems, biological questions, and multiplexed imaging modalities.

Epstein-Barr Virus Orchestrates Spatial Reorganization and Immunomodulation within the Classic Hodgkin Lymphoma Tumor Microenvironment

Classic Hodgkin Lymphoma (cHL) is a tumor composed of rare malignant Hodgkin and Reed-Sternberg (HRS) cells nested within a T-cell rich inflammatory immune infiltrate. cHL is associated with Epstein-Barr Virus (EBV) in 25% of cases. The specific contributions of EBV to the pathogenesis of cHL remain largely unknown, in part due to technical barriers in dissecting the tumor microenvironment (TME) in high detail. Herein, we applied multiplexed ion beam imaging (MIBI) spatial pro-teomics on 6 EBV-positive and 14 EBV-negative cHL samples. We identify key TME features that distinguish between EBV-positive and EBV-negative cHL, including the relative predominance of memory CD8 T cells and increased T-cell dysfunction as a function of spatial proximity to HRS cells. Building upon a larger multi-institutional cohort of 22 EBV-positive and 24 EBV-negative cHL samples, we orthogonally validated our findings through a spatial multi-omics approach, coupling whole transcriptome capture with antibody-defined cell types for tu-mor and T-cell populations within the cHL TME. We delineate contrasting transcriptomic immunological signatures between EBV-positive and EBV-negative cases that differently impact HRS cell proliferation, tumor-immune interactions, and mecha-nisms of T-cell dysregulation and dysfunction. Our multi-modal framework enabled a comprehensive dissection of EBV-linked reorganization and immune evasion within the cHL TME, and highlighted the need to elucidate the cellular and molecular fac-tors of virus-associated tumors, with potential for targeted therapeutic strategies.

Machine Learning and Deep Learning in Synthetic Biology: Key Architectures, Applications, and Challenges

Machine learning (ML), particularly deep learning (DL), has made rapid and substantial progress in synthetic biology in recent years. Biotechnological applications of biosystems, including pathways, enzymes, and whole cells, are being probed frequently with time. The intricacy and interconnectedness of biosystems make it challenging to design them with the desired properties. ML and DL have a synergy with synthetic biology. Synthetic biology can be employed to produce large data sets for training models (for instance, by utilizing DNA synthesis), and ML/DL models can be employed to inform design (for example, by generating new parts or advising unrivaled experiments to perform). This potential has recently been brought to light by research at the intersection of engineering biology and ML/DL through achievements like the design of novel biological components, best experimental design, automated analysis of microscopy data, protein structure prediction, and biomolecular implementations of ANNs (Artificial Neural Networks). I have divided this review into three sections. In the first section, I describe predictive potential and basics of ML along with myriad applications in synthetic biology, especially in engineering cells, activity of proteins, and metabolic pathways. In the second section, I describe fundamental DL architectures and their applications in synthetic biology. Finally, I describe different challenges causing hurdles in the progress of ML/DL and synthetic biology along with their solutions.

Nondestructive, quantitative viability analysis of 3D tissue cultures using machine learning image segmentation

Ascertaining the collective viability of cells in different cell culture conditions has typically relied on averaging colorimetric indicators and is often reported out in simple binary readouts. Recent research has combined viability assessment techniques with image-based deep-learning models to automate the characterization of cellular properties. However, further development of viability measurements to assess the continuity of possible cellular states and responses to perturbation across cell culture conditions is needed. In this work, we demonstrate an image processing algorithm for quantifying features associated with cellular viability in 3D cultures without the need for assay-based indicators. We show that our algorithm performs similarly to a pair of human experts in whole-well images over a range of days and culture matrix compositions. To demonstrate potential utility, we perform a longitudinal study investigating the impact of a known therapeutic on pancreatic cancer spheroids. Using images taken with a high content imaging system, the algorithm successfully tracks viability at the individual spheroid and whole-well level. The method we propose reduces analysis time by 97% in comparison with the experts. Because the method is independent of the microscope or imaging system used, this approach lays the foundation for accelerating progress in and for improving the robustness and reproducibility of 3D culture analysis across biological and clinical research.

Software Tools for 2D Cell Segmentation

Cell segmentation is an important task in the field of image processing, widely used in the life sciences and medical fields. Traditional methods are mainly based on pixel intensity and spatial relationships, but have limitations. In recent years, machine learning and deep learning methods have been widely used, providing more-accurate and efficient solutions for cell segmentation. The effort to develop efficient and accurate segmentation software tools has been one of the major focal points in the field of cell segmentation for years. However, each software tool has unique characteristics and adaptations, and no universal cell-segmentation software can achieve perfect results. In this review, we used three publicly available datasets containing multiple 2D cell-imaging modalities. Common segmentation metrics were used to evaluate the performance of eight segmentation tools to compare their generality and, thus, find the best-performing tool.

Accurate single-molecule spot detection for image-based spatial transcriptomics with weakly supervised deep learning

Image-based spatial transcriptomics methods enable transcriptome-scale gene expression measurements with spatial information but require complex, manually-tuned analysis pipelines. We present Polaris, an analysis pipeline for image-based spatial transcriptomics that combines deep learning models for cell segmentation and spot detection with a probabilistic gene decoder to quantify single-cell gene expression accurately. Polaris offers a unifying, turnkey solution for analyzing spatial transcriptomics data from MERFSIH, seqFISH, or ISS experiments. Polaris is available through the DeepCell software library (https://github.com/vanvalenlab/deepcell-spots) and https://www.deepcell.org.

From pixels to insights: Machine learning and deep learning for bioimage analysis

Bioimage analysis plays a critical role in extracting information from biological images, enabling deeper insights into cellular structures and processes. The integration of machine learning and deep learning techniques has revolutionized the field, enabling the automated, reproducible, and accurate analysis of biological images. Here, we provide an overview of the history and principles of machine learning and deep learning in the context of bioimage analysis. We discuss the essential steps of the bioimage analysis workflow, emphasizing how machine learning and deep learning have improved preprocessing, segmentation, feature extraction, object tracking, and classification. We provide examples that showcase the application of machine learning and deep learning in bioimage analysis. We examine user-friendly software and tools that enable biologists to leverage these techniques without extensive computational expertise. This review is a resource for researchers seeking to incorporate machine learning and deep learning in their bioimage analysis workflows and enhance their research in this rapidly evolving field.

A systematic evaluation of computation methods for cell segmentation

Cell segmentation is a fundamental task in analyzing biomedical images. Many computational methods have been developed for cell segmentation, but their performances are not well understood in various scenarios. We systematically evaluated the performance of 18 segmentation methods to perform cell nuclei and whole cell segmentation using light microscopy and fluorescence staining images. We found that general-purpose methods incorporating the attention mechanism exhibit the best overall performance. We identified various factors influencing segmentation performances, including training data and cell morphology, and evaluated the generalizability of methods across image modalities. We also provide guidelines for choosing the optimal segmentation methods in various real application scenarios. We developed Seggal, an online resource for downloading segmentation models already pre-trained with various tissue and cell types, which substantially reduces the time and effort for training cell segmentation models.

MAPS: pathologist-level cell type annotation from tissue images through machine learning

Highly multiplexed protein imaging is emerging as a potent technique for analyzing protein distribution within cells and tissues in their native context. However, existing cell annotation methods utilizing high-plex spatial proteomics data are resource intensive and necessitate iterative expert input, thereby constraining their scalability and practicality for extensive datasets. We introduce MAPS (Machine learning for Analysis of Proteomics in Spatial biology), a machine learning approach facilitating rapid and precise cell type identification with human-level accuracy from spatial proteomics data. Validated on multiple in-house and publicly available MIBI and CODEX datasets, MAPS outperforms current annotation techniques in terms of speed and accuracy, achieving pathologist-level precision even for typically challenging cell types, including tumor cells of immune origin. By democratizing rapidly deployable and scalable machine learning annotation, MAPS holds significant potential to expedite advances in tissue biology and disease comprehension.

Unveiling Macrophage Heterogeneity and Their Spatial Distribution Using Multiplexed Tissue Imaging

Macrophages display a high degree of phenotypic diversity and plasticity, which is influenced by their location within the tissue microenvironment. Co-Detection by Indexing (CODEX), a multiplexed imaging technique, allows the simultaneous detection of multiple membrane and cellular markers that enable the accurate identification of tissue-resident hematopoietic and non-hematopoietic cells, while conferring spatial information at a single-cell level. Here we describe the use of CODEX to visualize the phenotypic and spatial heterogeneity of murine tissue-resident macrophages in several organs, and a pipeline to characterize their cellular microenvironments and interactions.

A guide to ERK dynamics, part 1: mechanisms and models

Extracellular signal-regulated kinase (ERK) has long been studied as a key driver of both essential cellular processes and disease. A persistent question has been how this single pathway is able to direct multiple cell behaviors, including growth, proliferation, and death. Modern biosensor studies have revealed that the temporal pattern of ERK activity is highly variable and heterogeneous, and critically, that these dynamic differences modulate cell fate. This two-part review discusses the current understanding of dynamic activity in the ERK pathway, how it regulates cellular decisions, and how these cell fates lead to tissue regulation and pathology. In part 1, we cover the optogenetic and live-cell imaging technologies that first revealed the dynamic nature of ERK, as well as current challenges in biosensor data analysis. We also discuss advances in mathematical models for the mechanisms of ERK dynamics, including receptor-level regulation, negative feedback, cooperativity, and paracrine signaling. While hurdles still remain, it is clear that higher temporal and spatial resolution provide mechanistic insights into pathway circuitry. Exciting new algorithms and advanced computational tools enable quantitative measurements of single-cell ERK activation, which in turn inform better models of pathway behavior. However, the fact that current models still cannot fully recapitulate the diversity of ERK responses calls for a deeper understanding of network structure and signal transduction in general.

A framework for immunofluorescence image augmentation and classification based on unsupervised attention mechanism

Autoimmune encephalitis (AE) is a common neurological disorder. As a standard method for neuroautoantibody detection, pathologists use tissue matrix assays (TBA) for initial disease screening. In this study, microscopic fluorescence imaging was combined with deep learning to improve AE diagnostic accuracy. Due to the inter-class imbalance of medical data, we propose an innovative generative adversarial network supplemented with attention mechanisms to highlight key regions in images to synthesize high-quality fluorescence images. However, securing annotated medical data is both time-consuming and costly. To circumvent this problem, we employ a self-supervised learning approach that utilizes unlabeled fluorescence data to support downstream classification tasks. To better understand the fluorescence properties in the data, we introduce a multichannel input convolutional neural network that adds additional channels of fluorescence intensity. This study builds an AE immunofluorescence dataset and obtains the classification accuracy of 88.5% using our method, thus confirming the effectiveness of the proposed method.

Point-Supervised Single-Cell Segmentation via Collaborative Knowledge Sharing

Despite their superior performance, deep-learning methods often suffer from the disadvantage of needing large-scale well-annotated training data. In response, recent literature has seen a proliferation of efforts aimed at reducing the annotation burden. This paper focuses on a weakly-supervised training setting for single-cell segmentation models, where the only available training label is the rough locations of individual cells. The specific problem is of practical interest due to the widely available nuclei counter-stain data in biomedical literature, from which the cell locations can be derived programmatically. Of more general interest is a proposed self-learning method called collaborative knowledge sharing, which is related to but distinct from the more well-known consistency learning methods. This strategy achieves self-learning by sharing knowledge between a principal model and a very light-weight collaborator model. Importantly, the two models are entirely different in their architectures, capacities, and model outputs: In our case, the principal model approaches the segmentation problem from an object-detection perspective, whereas the collaborator model a sematic segmentation perspective. We assessed the effectiveness of this strategy by conducting experiments on LIVECell, a large single-cell segmentation dataset of bright-field images, and on A431 dataset, a fluorescence image dataset in which the location labels are generated automatically from nuclei counter-stain data. Implementing code is available at https://github.com/jiyuuchc/lacss.

Emerging technologies in adipose tissue research

Technologies are transforming the understanding of adipose tissue as a complex and dynamic tissue that plays a critical role in energy homoeostasis and metabolic health. This mini-review provides a brief overview of the potential impact of novel technologies in biomedical research and aims to identify areas where these technologies can make the most significant contribution to adipose tissue research. It discusses the impact of cutting-edge technologies such as single-cell sequencing, multi-omics analyses, spatial transcriptomics, live imaging, 3D tissue engineering, microbiome analysis, in vivo imaging, and artificial intelligence/machine learning. As these technologies continue to evolve, we can expect them to play an increasingly important role in advancing our understanding of adipose tissue and improving the treatment of related diseases.

Opportunities and challenges for deep learning in cell dynamics research

The growth of artificial intelligence (AI) has led to an increase in the adoption of computer vision and deep learning (DL) techniques for the evaluation of microscopy images and movies. This adoption has not only addressed hurdles in quantitative analysis of dynamic cell biological processes but has also started to support advances in drug development, precision medicine, and genome-phenome mapping. We survey existing AI-based techniques and tools, as well as open-source datasets, with a specific focus on the computational tasks of segmentation, classification, and tracking of cellular and subcellular structures and dynamics. We summarise long-standing challenges in microscopy video analysis from a computational perspective and review emerging research frontiers and innovative applications for DL-guided automation in cell dynamics research.

Single cell spatial analysis reveals inflammatory foci of immature neutrophil and CD8 T cells in COVID-19 lungs

Single cell spatial interrogation of the immune-structural interactions in COVID -19 lungs is challenging, mainly because of the marked cellular infiltrate and architecturally distorted microstructure. To address this, we develop a suite of mathematical tools to search for statistically significant co-locations amongst immune and structural cells identified using 37-plex imaging mass cytometry. This unbiased method reveals a cellular map interleaved with an inflammatory network of immature neutrophils, cytotoxic CD8 T cells, megakaryocytes and monocytes co-located with regenerating alveolar progenitors and endothelium. Of note, a highly active cluster of immature neutrophils and CD8 T cells, is found spatially linked with alveolar progenitor cells, and temporally with the diffuse alveolar damage stage. These findings offer further insights into how immune cells interact in the lungs of severe COVID-19 disease. We provide our pipeline [Spatial Omics Oxford Pipeline (SpOOx)] and visual-analytical tool, Multi-Dimensional Viewer (MDV) software, as a resource for spatial analysis.

An exhaustive multiple knockout approach to understanding cell wall hydrolase function in Bacillus subtilis

IMPORTANCE

In order to grow, bacterial cells must both create and break down their cell wall. The enzymes that are responsible for these processes are the target of some of our best antibiotics. Our understanding of the proteins that break down the wall- cell wall hydrolases-has been limited by redundancy among the large number of hydrolases many bacteria contain. To solve this problem, we identified 42 cell wall hydrolases in Bacillus subtilis and created a strain lacking 40 of them. We show that cells can survive using only a single cell wall hydrolase; this means that to understand the growth of B. subtilis in standard laboratory conditions, it is only necessary to study a very limited number of proteins, simplifying the problem substantially. We additionally show that the ∆40 strain is a research tool to characterize hydrolases, using it to identify three "helper" hydrolases that act in certain stress conditions.

GeneSegNet: a deep learning framework for cell segmentation by integrating gene expression and imaging

When analyzing data from in situ RNA detection technologies, cell segmentation is an essential step in identifying cell boundaries, assigning RNA reads to cells, and studying the gene expression and morphological features of cells. We developed a deep-learning-based method, GeneSegNet, that integrates both gene expression and imaging information to perform cell segmentation. GeneSegNet also employs a recursive training strategy to deal with noisy training labels. We show that GeneSegNet significantly improves cell segmentation performances over existing methods that either ignore gene expression information or underutilize imaging information.

Unbiased image segmentation assessment toolkit for quantitative differentiation of state-of-the-art algorithms and pipelines

BACKGROUND

Image segmentation pipelines are commonly used in microscopy to identify cellular compartments like nucleus and cytoplasm, but there are few standards for comparing segmentation accuracy across pipelines. The process of selecting a segmentation assessment pipeline can seem daunting to researchers due to the number and variety of metrics available for evaluating segmentation quality.

RESULTS

Here we present automated pipelines to obtain a comprehensive set of 69 metrics to evaluate segmented data and propose a selection methodology for models based on quantitative analysis, dimension reduction or unsupervised classification techniques and informed selection criteria.

CONCLUSION

We show that the metrics used here can often be reduced to a small number of metrics that give a more complete understanding of segmentation accuracy, with different groups of metrics providing sensitivity to different types of segmentation error. These tools are delivered as easy to use python libraries, command line tools, Common Workflow Language Tools, and as Web Image Processing Pipeline interactive plugins to ensure a wide range of users can access and use them. We also present how our evaluation methods can be used to observe the changes in segmentations across modern machine learning/deep learning workflows and use cases.

LOCATOR: feature extraction and spatial analysis of the cancer tissue microenvironment using mass cytometry imaging technologies

Motivation

Recent advances in highly multiplexed imaging have provided unprecedented insights into the complex cellular organization of tissues, with many applications in translational medicine. However, downstream analyses of multiplexed imaging data face several technical limitations, and although some computational methods and bioinformatics tools are available, deciphering the complex spatial organization of cellular ecosystems remains a challenging problem.

Results

To mitigate this problem, we develop a novel computational tool, LOCATOR (anaLysis Of CAncer Tissue micrOenviRonment), for spatial analysis of cancer tissue microenvironments using data acquired from mass cytometry imaging technologies. LOCATOR introduces a graph-based representation of tissue images to describe features of the cellular organization and deploys downstream analysis and visualization utilities that can be used for data-driven patient-risk stratification. Our case studies using mass cytometry imaging data from two well-annotated breast cancer cohorts re-confirmed that the spatial organization of the tumour-immune microenvironment is strongly associated with the clinical outcome in breast cancer. In addition, we report interesting potential associations between the spatial organization of macrophages and patients' survival. Our work introduces an automated and versatile analysis tool for mass cytometry imaging data with many applications in future cancer research projects.

Availability and implementation

Datasets and codes of LOCATOR are publicly available at https://github.com/RezvanEhsani/LOCATOR.

Addressing annotation and data scarcity when designing machine learning strategies for neurophotonics

Machine learning has revolutionized the way data are processed, allowing information to be extracted in a fraction of the time it would take an expert. In the field of neurophotonics, machine learning approaches are used to automatically detect and classify features of interest in complex images. One of the key challenges in applying machine learning methods to the field of neurophotonics is the scarcity of available data and the complexity associated with labeling them, which can limit the performance of data-driven algorithms. We present an overview of various strategies, such as weakly supervised learning, active learning, and domain adaptation that can be used to address the problem of labeled data scarcity in neurophotonics. We provide a comprehensive overview of the strengths and limitations of each approach and discuss their potential applications to bioimaging datasets. In addition, we highlight how different strategies can be combined to increase model performance on those datasets. The approaches we describe can help to improve the accessibility of machine learning-based analysis with limited number of annotated images for training and can enable researchers to extract more meaningful insights from small datasets.

Plantorganelle Hunter is an effective deep-learning-based method for plant organelle phenotyping in electron microscopy

Accurate delineation of plant cell organelles from electron microscope images is essential for understanding subcellular behaviour and function. Here we develop a deep-learning pipeline, called the organelle segmentation network (OrgSegNet), for pixel-wise segmentation to identify chloroplasts, mitochondria, nuclei and vacuoles. OrgSegNet was evaluated on a large manually annotated dataset collected from 19 plant species and achieved state-of-the-art segmentation performance. We defined three digital traits (shape complexity, electron density and cross-sectional area) to track the quantitative features of individual organelles in 2D images and released an open-source web tool called Plantorganelle Hunter for quantitatively profiling subcellular morphology. In addition, the automatic segmentation method was successfully applied to a serial-sectioning scanning microscope technique to create a 3D cell model that offers unique views of the morphology and distribution of these organelles. The functionalities of Plantorganelle Hunter can be easily operated, which will increase efficiency and productivity for the plant science community, and enhance understanding of subcellular biology.

Long-term, real-time and label-free live cell image processing and analysis based on a combined algorithm of CellPose and watershed segmentation

Developing a rapid and quantitative method to accurately evaluate the physiological abilities of living cells is critical for tumor control. Many experiments have been conducted in the field of biology in an attempt to measure the proliferation and movement abilities of cells, but existing methods cannot provide real-time and objective data for label-free cells. The quantitative imaging technique, including an automatic segmentation algorithm for individual label-free cells, has been a breakthrough in this regard. In this study, we develop a combined automatic image processing algorithm of CellPose and watershed segmentation for the long-term and real-time imaging of label-free cells. This method shows strong reliability in cell identification regardless of cell densities, allowing us to obtain accurate information about the number and proliferation ability of the target cells. Additionally, our results also suggest that this method is a reliable way to assess real-time data on drug cytotoxicity, cell morphology, and cell movement ability.

Tracking bacteria at high density with FAST, the Feature-Assisted Segmenter/Tracker

Most bacteria live attached to surfaces in densely-packed communities. While new experimental and imaging techniques are beginning to provide a window on the complex processes that play out in these communities, resolving the behaviour of individual cells through time and space remains a major challenge. Although a number of different software solutions have been developed to track microorganisms, these typically require users either to tune a large number of parameters or to groundtruth a large volume of imaging data to train a deep learning model-both manual processes which can be very time consuming for novel experiments. To overcome these limitations, we have developed FAST, the Feature-Assisted Segmenter/Tracker, which uses unsupervised machine learning to optimise tracking while maintaining ease of use. Our approach, rooted in information theory, largely eliminates the need for users to iteratively adjust parameters manually and make qualitative assessments of the resulting cell trajectories. Instead, FAST measures multiple distinguishing 'features' for each cell and then autonomously quantifies the amount of unique information each feature provides. We then use these measurements to determine how data from different features should be combined to minimize tracking errors. Comparing our algorithm with a naïve approach that uses cell position alone revealed that FAST produced 4 to 10 fold fewer tracking errors. The modular design of FAST combines our novel tracking method with tools for segmentation, extensive data visualisation, lineage assignment, and manual track correction. It is also highly extensible, allowing users to extract custom information from images and seamlessly integrate it into downstream analyses. FAST therefore enables high-throughput, data-rich analyses with minimal user input. It has been released for use either in Matlab or as a compiled stand-alone application, and is available at https://bit.ly/3vovDHn, along with extensive tutorials and detailed documentation.

Evolution and impact of high content imaging

The field of high content imaging has steadily evolved and expanded substantially across many industry and academic research institutions since it was first described in the early 1990's. High content imaging refers to the automated acquisition and analysis of microscopic images from a variety of biological sample types. Integration of high content imaging microscopes with multiwell plate handling robotics enables high content imaging to be performed at scale and support medium- to high-throughput screening of pharmacological, genetic and diverse environmental perturbations upon complex biological systems ranging from 2D cell cultures to 3D tissue organoids to small model organisms. In this perspective article the authors provide a collective view on the following key discussion points relevant to the evolution of high content imaging: • Evolution and impact of high content imaging: An academic perspective • Evolution and impact of high content imaging: An industry perspective • Evolution of high content image analysis • Evolution of high content data analysis pipelines towards multiparametric and phenotypic profiling applications • The role of data integration and multiomics • The role and evolution of image data repositories and sharing standards • Future perspective of high content imaging hardware and software.

A microwell platform for high-throughput longitudinal phenotyping and selective retrieval of organoids

Organoids are powerful experimental models for studying the ontogeny and progression of various diseases including cancer. Organoids are conventionally cultured in bulk using an extracellular matrix mimic. However, bulk-cultured organoids physically overlap, making it impossible to track the growth of individual organoids over time in high throughput. Moreover, local spatial variations in bulk matrix properties make it difficult to assess whether observed phenotypic heterogeneity between organoids results from intrinsic cell differences or differences in the microenvironment. Here, we developed a microwell-based method that enables high-throughput quantification of image-based parameters for organoids grown from single cells, which can further be retrieved from their microwells for molecular profiling. Coupled with a deep learning image-processing pipeline, we characterized phenotypic traits including growth rates, cellular movement, and apical-basal polarity in two CRISPR-engineered human gastric organoid models, identifying genomic changes associated with increased growth rate and changes in accessibility and expression correlated with apical-basal polarity. A record of this paper's transparent peer review process is included in the supplemental information.

Mechanism of outer membrane destabilization by global reduction of protein content

The outer membrane (OM) of Gram-negative bacteria such as Escherichia coli is an asymmetric bilayer with the glycolipid lipopolysaccharide (LPS) in the outer leaflet and glycerophospholipids in the inner. Nearly all integral OM proteins (OMPs) have a characteristic β-barrel fold and are assembled in the OM by the BAM complex, which contains one essential β-barrel protein (BamA), one essential lipoprotein (BamD), and three non-essential lipoproteins (BamBCE). A gain-of-function mutation in bamA enables survival in the absence of BamD, showing that the essential function of this protein is regulatory. Here, we demonstrate that the global reduction in OMPs caused by BamD loss weakens the OM, altering cell shape and causing OM rupture in spent medium. To fill the void created by OMP loss, phospholipids (PLs) flip into the outer leaflet. Under these conditions, mechanisms that remove PLs from the outer leaflet create tension between the OM leaflets, which contributes to membrane rupture. Rupture is prevented by suppressor mutations that release the tension by halting PL removal from the outer leaflet. However, these suppressors do not restore OM stiffness or normal cell shape, revealing a possible connection between OM stiffness and cell shape.

Metabolomic rearrangement controls the intrinsic microbial response to temperature changes

Temperature is one of the key determinants of microbial behavior and survival, whose impact is typically studied under heat- or cold-shock conditions that elicit specific regulation to combat lethal stress. At intermediate temperatures, cellular growth rate varies according to the Arrhenius law of thermodynamics without stress responses, a behavior whose origins have not yet been elucidated. Using single-cell microscopy during temperature perturbations, we show that bacteria exhibit a highly conserved, gradual response to temperature upshifts with a time scale of ~1.5 doublings at the higher temperature, regardless of initial/final temperature or nutrient source. We find that this behavior is coupled to a temperature memory, which we rule out as being neither transcriptional, translational, nor membrane dependent. Instead, we demonstrate that an autocatalytic enzyme network incorporating temperature-sensitive Michaelis-Menten kinetics recapitulates all temperature-shift dynamics through metabolome rearrangement, which encodes a temperature memory and successfully predicts alterations in the upshift response observed under simple-sugar, low-nutrient conditions, and in fungi. This model also provides a mechanistic framework for both Arrhenius-dependent growth and the classical Monod Equation through temperature-dependent metabolite flux.

A mutant fitness compendium in Bifidobacteria reveals molecular determinants of colonization and host-microbe interactions

Bifidobacteria commonly represent a dominant constituent of human gut microbiomes during infancy, influencing nutrition, immune development, and resistance to infection. Despite interest as a probiotic therapy, predicting the nutritional requirements and health-promoting effects of Bifidobacteria is challenging due to major knowledge gaps. To overcome these deficiencies, we used large-scale genetics to create a compendium of mutant fitness in Bifidobacterium breve (Bb). We generated a high density, randomly barcoded transposon insertion pool in Bb, and used this pool to determine Bb fitness requirements during colonization of germ-free mice and chickens with multiple diets and in response to hundreds of in vitro perturbations. To enable mechanistic investigation, we constructed an ordered collection of insertion strains covering 1462 genes. We leveraged these tools to improve models of metabolic pathways, reveal unexpected host- and diet-specific requirements for colonization, and connect the production of immunomodulatory molecules to growth benefits. These resources will greatly reduce the barrier to future investigations of this important beneficial microbe.

Contributions of deep learning to automated numerical modelling of the interaction of electric fields and cartilage tissue based on 3D images

Electric fields find use in tissue engineering but also in sensor applications besides the broad classical application range. Accurate numerical models of electrical stimulation devices can pave the way for effective therapies in cartilage regeneration. To this end, the dielectric properties of the electrically stimulated tissue have to be known. However, knowledge of the dielectric properties is scarce. Electric field-based methods such as impedance spectroscopy enable determining the dielectric properties of tissue samples. To develop a detailed understanding of the interaction of the employed electric fields and the tissue, fine-grained numerical models based on tissue-specific 3D geometries are considered. A crucial ingredient in this approach is the automated generation of numerical models from biomedical images. In this work, we explore classical and artificial intelligence methods for volumetric image segmentation to generate model geometries. We find that deep learning, in particular the StarDist algorithm, permits fast and automatic model geometry and discretisation generation once a sufficient amount of training data is available. Our results suggest that already a small number of 3D images (23 images) is sufficient to achieve 80% accuracy on the test data. The proposed method enables the creation of high-quality meshes without the need for computer-aided design geometry post-processing. Particularly, the computational time for the geometrical model creation was reduced by half. Uncertainty quantification as well as a direct comparison between the deep learning and the classical approach reveal that the numerical results mainly depend on the cell volume. This result motivates further research into impedance sensors for tissue characterisation. The presented approach can significantly improve the accuracy and computational speed of image-based models of electrical stimulation for tissue engineering applications.

Comparison of particle image velocimetry and the underlying agents dynamics in collectively moving self propelled particles

Collective migration of cells is a fundamental behavior in biology. For the quantitative understanding of collective cell migration, live-cell imaging techniques have been used using e.g., phase contrast or fluorescence images. Particle tracking velocimetry (PTV) is a common recipe to quantify cell motility with those image data. However, the precise tracking of cells is not always feasible. Particle image velocimetry (PIV) is an alternative to PTV, corresponding to Eulerian picture of fluid dynamics, which derives the average velocity vector of an aggregate of cells. However, the accuracy of PIV in capturing the underlying cell motility and what values of the parameters should be chosen is not necessarily well characterized, especially for cells that do not adhere to a viscous flow. Here, we investigate the accuracy of PIV by generating images of simulated cells by the Vicsek model using trajectory data of agents at different noise levels. It was found, using an alignment score, that the direction of the PIV vectors coincides with the direction of nearby agents with appropriate choices of PIV parameters. PIV is found to accurately measure the underlying motion of individual agents for a wide range of noise level, and its condition is addressed.

A review on deep learning applications in highly multiplexed tissue imaging data analysis

Since its introduction into the field of oncology, deep learning (DL) has impacted clinical discoveries and biomarker predictions. DL-driven discoveries and predictions in oncology are based on a variety of biological data such as genomics, proteomics, and imaging data. DL-based computational frameworks can predict genetic variant effects on gene expression, as well as protein structures based on amino acid sequences. Furthermore, DL algorithms can capture valuable mechanistic biological information from several spatial "omics" technologies, such as spatial transcriptomics and spatial proteomics. Here, we review the impact that the combination of artificial intelligence (AI) with spatial omics technologies has had on oncology, focusing on DL and its applications in biomedical image analysis, encompassing cell segmentation, cell phenotype identification, cancer prognostication, and therapy prediction. We highlight the advantages of using highly multiplexed images (spatial proteomics data) compared to single-stained, conventional histopathological ("simple") images, as the former can provide deep mechanistic insights that cannot be obtained by the latter, even with the aid of explainable AI. Furthermore, we provide the reader with the advantages/disadvantages of DL-based pipelines used in preprocessing highly multiplexed images (cell segmentation, cell type annotation). Therefore, this review also guides the reader to choose the DL-based pipeline that best fits their data. In conclusion, DL continues to be established as an essential tool in discovering novel biological mechanisms when combined with technologies such as highly multiplexed tissue imaging data. In balance with conventional medical data, its role in clinical routine will become more important, supporting diagnosis and prognosis in oncology, enhancing clinical decision-making, and improving the quality of care for patients. Since its introduction into the field of oncology, deep learning (DL) has impacted clinical discoveries and biomarker predictions. DL-driven discoveries and predictions in oncology are based on a variety of biological data such as genomics, proteomics, and imaging data. DL-based computational frameworks can predict genetic variant effects on gene expression, as well as protein structures based on amino acid sequences. Furthermore, DL algorithms can capture valuable mechanistic biological information from several spatial "omics" technologies, such as spatial transcriptomics and spatial proteomics. Here, we review the impact that the combination of artificial intelligence (AI) with spatial omics technologies has had on oncology, focusing on DL and its applications in biomedical image analysis, encompassing cell segmentation, cell phenotype identification, cancer prognostication, and therapy prediction. We highlight the advantages of using highly multiplexed images (spatial proteomics data) compared to single-stained, conventional histopathological ("simple") images, as the former can provide deep mechanistic insights that cannot be obtained by the latter, even with the aid of explainable AI. Furthermore, we provide the reader with the advantages/disadvantages of the DL-based pipelines used in preprocessing the highly multiplexed images (cell segmentation, cell type annotation). Therefore, this review also guides the reader to choose the DL-based pipeline that best fits their data. In conclusion, DL continues to be established as an essential tool in discovering novel biological mechanisms when combined with technologies such as highly multiplexed tissue imaging data. In balance with conventional medical data, its role in clinical routine will become more important, supporting diagnosis and prognosis in oncology, enhancing clinical decision-making, and improving the quality of care for patients.

Fast detection of slender bodies in high density microscopy data

Computer-aided analysis of biological microscopy data has seen a massive improvement with the utilization of general-purpose deep learning techniques. Yet, in microscopy studies of multi-organism systems, the problem of collision and overlap remains challenging. This is particularly true for systems composed of slender bodies such as swimming nematodes, swimming spermatozoa, or the beating of eukaryotic or prokaryotic flagella. Here, we develop a end-to-end deep learning approach to extract precise shape trajectories of generally motile and overlapping slender bodies. Our method works in low resolution settings where feature keypoints are hard to define and detect. Detection is fast and we demonstrate the ability to track thousands of overlapping organisms simultaneously. While our approach is agnostic to area of application, we present it in the setting of and exemplify its usability on dense experiments of swimming Caenorhabditis elegans. The model training is achieved purely on synthetic data, utilizing a physics-based model for nematode motility, and we demonstrate the model's ability to generalize from simulations to experimental videos.