Enabling reactive microscopy with MicroMator

The rise of data-driven microscopy powered by machine learning

Optical microscopy is an indispensable tool in life sciences research, but conventional techniques require compromises between imaging parameters like speed, resolution, field of view and phototoxicity. To overcome these limitations, data-driven microscopes incorporate feedback loops between data acquisition and analysis. This review overviews how machine learning enables automated image analysis to optimise microscopy in real time. We first introduce key data-driven microscopy concepts and machine learning methods relevant to microscopy image analysis. Subsequently, we highlight pioneering works and recent advances in integrating machine learning into microscopy acquisition workflows, including optimising illumination, switching modalities and acquisition rates, and triggering targeted experiments. We then discuss the remaining challenges and future outlook. Overall, intelligent microscopes that can sense, analyse and adapt promise to transform optical imaging by opening new experimental possibilities.

Live-cell imaging powered by computation

The proliferation of microscopy methods for live-cell imaging offers many new possibilities for users but can also be challenging to navigate. The prevailing challenge in live-cell fluorescence microscopy is capturing intra-cellular dynamics while preserving cell viability. Computational methods can help to address this challenge and are now shifting the boundaries of what is possible to capture in living systems. In this Review, we discuss these computational methods focusing on artificial intelligence-based approaches that can be layered on top of commonly used existing microscopies as well as hybrid methods that integrate computation and microscope hardware. We specifically discuss how computational approaches can improve the signal-to-noise ratio, spatial resolution, temporal resolution and multi-colour capacity of live-cell imaging.

Design and Model-Driven Analysis of Synthetic Circuits with the Staphylococcus aureus Dead-Cas9 (sadCas9) as a Programmable Transcriptional Regulator in Bacteria

Synthetic circuit design is crucial for engineering microbes that process environmental cues and provide biologically relevant outputs. To reliably scale-up circuit complexity, the availability of parts toolkits is central. Streptococcus pyogenes (sp)-derived CRISPR interference/dead-Cas9 (CRISPRi/spdCas9) is widely adopted for implementing programmable regulations in synthetic circuits, and alternative CRISPRi systems will further expand our toolkits of orthogonal components. Here, we showcase the potential of CRISPRi using the engineered dCas9 from Staphylococcus aureus (sadCas9), not previously used in bacterial circuits, that is attractive for its low size and high specificity. We designed a collection of ∼20 increasingly complex circuits and variants in Escherichia coli, including circuits with static function like one-/two-input logic gates (NOT, NAND), circuits with dynamic behavior like incoherent feedforward loops (iFFLs), and applied sadCas9 to fix a T7 polymerase-based cascade. Data demonstrated specific and efficient target repression (100-fold) and qualitatively successful functioning for all circuits. Other advantageous features included low sadCas9-borne cell load and orthogonality with spdCas9. However, different circuit variants showed quantitatively unexpected and previously unreported steady-state responses: the dynamic range, switch point, and slope of NOT/NAND gates changed for different output promoters, and a multiphasic behavior was observed in iFFLs, differing from the expected bell-shaped or sigmoidal curves. Model analysis explained the observed curves by complex interplays among components, due to reporter gene-borne cell load and regulator competition. Overall, CRISPRi/sadCas9 successfully expanded the available toolkit for bacterial engineering. Analysis of our circuit collection depicted the impact of generally neglected effects modulating the shape of component dose-response curves, to avoid drawing wrong conclusions on circuit functioning.

Deep model predictive control of gene expression in thousands of single cells

Gene expression is inherently dynamic, due to complex regulation and stochastic biochemical events. However, the effects of these dynamics on cell phenotypes can be difficult to determine. Researchers have historically been limited to passive observations of natural dynamics, which can preclude studies of elusive and noisy cellular events where large amounts of data are required to reveal statistically significant effects. Here, using recent advances in the fields of machine learning and control theory, we train a deep neural network to accurately predict the response of an optogenetic system in Escherichia coli cells. We then use the network in a deep model predictive control framework to impose arbitrary and cell-specific gene expression dynamics on thousands of single cells in real time, applying the framework to generate complex time-varying patterns. We also showcase the framework's ability to link expression patterns to dynamic functional outcomes by controlling expression of the tetA antibiotic resistance gene. This study highlights how deep learning-enabled feedback control can be used to tailor distributions of gene expression dynamics with high accuracy and throughput without expert knowledge of the biological system.

Harnessing artificial intelligence to reduce phototoxicity in live imaging

Fluorescence microscopy is essential for studying living cells, tissues and organisms. However, the fluorescent light that switches on fluorescent molecules also harms the samples, jeopardizing the validity of results - particularly in techniques such as super-resolution microscopy, which demands extended illumination. Artificial intelligence (AI)-enabled software capable of denoising, image restoration, temporal interpolation or cross-modal style transfer has great potential to rescue live imaging data and limit photodamage. Yet we believe the focus should be on maintaining light-induced damage at levels that preserve natural cell behaviour. In this Opinion piece, we argue that a shift in role for AIs is needed - AI should be used to extract rich insights from gentle imaging rather than recover compromised data from harsh illumination. Although AI can enhance imaging, our ultimate goal should be to uncover biological truths, not just retrieve data. It is essential to prioritize minimizing photodamage over merely pushing technical limits. Our approach is aimed towards gentle acquisition and observation of undisturbed living systems, aligning with the essence of live-cell fluorescence microscopy.

Live-cell imaging in the deep learning era

Live imaging is a powerful tool, enabling scientists to observe living organisms in real time. In particular, when combined with fluorescence microscopy, live imaging allows the monitoring of cellular components with high sensitivity and specificity. Yet, due to critical challenges (i.e., drift, phototoxicity, dataset size), implementing live imaging and analyzing the resulting datasets is rarely straightforward. Over the past years, the development of bioimage analysis tools, including deep learning, is changing how we perform live imaging. Here we briefly cover important computational methods aiding live imaging and carrying out key tasks such as drift correction, denoising, super-resolution imaging, artificial labeling, tracking, and time series analysis. We also cover recent advances in self-driving microscopy.

Advances in automated and reactive flow cytometry for synthetic biotechnology

Automated flow cytometry (FC) has been initially considered for bioprocess monitoring and optimization. More recently, new physical and software interfaces have been made available, facilitating the access to this technology for labs and industries. It also comes with new capabilities, such as being able to act on the cultivation conditions based on population data. This approach, known as reactive FC, extended the range of applications of automated FC to bioprocess control and the stabilization of cocultures, but also to the broad field of synthetic and systems biology for the characterization of gene circuits. However, several issues must be addressed before automated and reactive FC can be considered standard and modular technologies.

Analysis and design of single-cell experiments to harvest fluctuation information while rejecting measurement noise

Introduction: Despite continued technological improvements, measurement errors always reduce or distort the information that any real experiment can provide to quantify cellular dynamics. This problem is particularly serious for cell signaling studies to quantify heterogeneity in single-cell gene regulation, where important RNA and protein copy numbers are themselves subject to the inherently random fluctuations of biochemical reactions. Until now, it has not been clear how measurement noise should be managed in addition to other experiment design variables (e.g., sampling size, measurement times, or perturbation levels) to ensure that collected data will provide useful insights on signaling or gene expression mechanisms of interest. Methods: We propose a computational framework that takes explicit consideration of measurement errors to analyze single-cell observations, and we derive Fisher Information Matrix (FIM)-based criteria to quantify the information value of distorted experiments. Results and Discussion: We apply this framework to analyze multiple models in the context of simulated and experimental single-cell data for a reporter gene controlled by an HIV promoter. We show that the proposed approach quantitatively predicts how different types of measurement distortions affect the accuracy and precision of model identification, and we demonstrate that the effects of these distortions can be mitigated through explicit consideration during model inference. We conclude that this reformulation of the FIM could be used effectively to design single-cell experiments to optimally harvest fluctuation information while mitigating the effects of image distortion.

Light-sheets and smart microscopy, an exciting future is dawning

Light-sheet fluorescence microscopy has transformed our ability to visualize and quantitatively measure biological processes rapidly and over long time periods. In this review, we discuss current and future developments in light-sheet fluorescence microscopy that we expect to further expand its capabilities. This includes smart and adaptive imaging schemes to overcome traditional imaging trade-offs, i.e., spatiotemporal resolution, field of view and sample health. In smart microscopy, a microscope will autonomously decide where, when, what and how to image. We further assess how image restoration techniques provide avenues to overcome these tradeoffs and how "open top" light-sheet microscopes may enable multi-modal imaging with high throughput. As such, we predict that light-sheet microscopy will fulfill an important role in biomedical and clinical imaging in the future.

Recent advances, opportunities and challenges in cybergenetic identification and control of biomolecular networks

Cybergenetics is a new area of research aimed at developing digital and biological controllers for living systems. Synthetic biologists have begun exploiting cybergenetic tools and platforms to both accelerate the development of mathematical models and develop control strategies for complex biological phenomena. Here, we review the state of the art in cybergenetic identification and control. Our aim is to lower the entry barrier to this field and foster the adoption of methods and technologies that will accelerate the pace at which Synthetic Biology progresses toward applications.

Data-driven microscopy allows for automated context-specific acquisition of high-fidelity image data

Light microscopy is a powerful single-cell technique that allows for quantitative spatial information at subcellular resolution. However, unlike flow cytometry and single-cell sequencing techniques, microscopy has issues achieving high-quality population-wide sample characterization while maintaining high resolution. Here, we present a general framework, data-driven microscopy (DDM) that uses real-time population-wide object characterization to enable data-driven high-fidelity imaging of relevant phenotypes based on the population context. DDM combines data-independent and data-dependent steps to synergistically enhance data acquired using different imaging modalities. As a proof of concept, we develop and apply DDM with plugins for improved high-content screening and live adaptive microscopy for cell migration and infection studies that capture events of interest, rare or common, with high precision and resolution. We propose that DDM can reduce human bias, increase reproducibility, and place single-cell characteristics in the context of the sample population when interpreting microscopy data, leading to an increase in overall data fidelity.

An open-source microscopy framework for simultaneous control of image acquisition, reconstruction, and analysis

We present a computational framework to simultaneously perform image acquisition, reconstruction, and analysis in the context of open-source microscopy automation. The setup features multiple computer units intersecting software with hardware devices and achieves automation using python scripts. In practice, script files are executed in the acquisition computer and can perform any experiment by modifying the state of the hardware devices and accessing experimental data. The presented framework achieves concurrency by using multiple instances of ImSwitch and napari working simultaneously. ImSwitch is a flexible and modular open-source software package for microscope control, and napari is a multidimensional image viewer for scientific image analysis. The presented framework implements a system based on file watching, where multiple units monitor a filesystem that acts as the synchronization primitive. The proposed solution is valid for any microscope setup, supporting various biological applications. The only necessary element is a shared filesystem, common in any standard laboratory, even in resource-constrained settings. The file watcher functionality in Python can be easily integrated into other python-based software. We demonstrate the proposed solution by performing tiling experiments using the molecular nanoscale live imaging with sectioning ability (MoNaLISA) microscope, a high-throughput super-resolution microscope based on reversible saturable optical fluorescence transitions (RESOLFT).

An optogenetic toolkit for light-inducible antibiotic resistance

Antibiotics are a key control mechanism for synthetic biology and microbiology. Resistance genes are used to select desired cells and regulate bacterial populations, however their use to-date has been largely static. Precise spatiotemporal control of antibiotic resistance could enable a wide variety of applications that require dynamic control of susceptibility and survival. Here, we use light-inducible Cre recombinase to activate expression of drug resistance genes in Escherichia coli. We demonstrate light-activated resistance to four antibiotics: carbenicillin, kanamycin, chloramphenicol, and tetracycline. Cells exposed to blue light survive in the presence of lethal antibiotic concentrations, while those kept in the dark do not. To optimize resistance induction, we vary promoter, ribosome binding site, and enzyme variant strength using chromosome and plasmid-based constructs. We then link inducible resistance to expression of a heterologous fatty acid enzyme to increase production of octanoic acid. These optogenetic resistance tools pave the way for spatiotemporal control of cell survival.

Fast4DReg - fast registration of 4D microscopy datasets

Unwanted sample drift is a common issue that plagues microscopy experiments, preventing accurate temporal visualization and quantification of biological processes. Although multiple methods and tools exist to correct images post acquisition, performing drift correction of three-dimensional (3D) videos using open-source solutions remains challenging and time consuming. Here, we present a new tool developed for ImageJ or Fiji called Fast4DReg that can quickly correct axial and lateral drift in 3D video-microscopy datasets. Fast4DReg works by creating intensity projections along multiple axes and estimating the drift between frames using two-dimensional cross-correlations. Using synthetic and acquired datasets, we demonstrate that Fast4DReg can perform better than other state-of-the-art open-source drift-correction tools and significantly outperforms them in speed. We also demonstrate that Fast4DReg can be used to register misaligned channels in 3D using either calibration slides or misaligned images directly. Altogether, Fast4DReg provides a quick and easy-to-use method to correct 3D imaging data before further visualization and analysis.

Platforms for Optogenetic Stimulation and Feedback Control

Harnessing the potential of optogenetics in biology requires methodologies from different disciplines ranging from biology, to mechatronics engineering, to control engineering. Light stimulation of a synthetic optogenetic construct in a given biological species can only be achieved via a suitable light stimulation platform. Emerging optogenetic applications entail a consistent, reproducible, and regulated delivery of light adapted to the application requirement. In this review, we explore the evolution of light-induction hardware-software platforms from simple illumination set-ups to sophisticated microscopy, microtiter plate and bioreactor designs, and discuss their respective advantages and disadvantages. Here, we examine design approaches followed in performing optogenetic experiments spanning different cell types and culture volumes, with induction capabilities ranging from single cell stimulation to entire cell culture illumination. The development of automated measurement and stimulation schemes on these platforms has enabled researchers to implement various in silico feedback control strategies to achieve computer-controlled living systems—a theme we briefly discuss in the last part of this review.