High-throughput fluorescence microscopy using multi-frame motion deblurring

GANscan: continuous scanning microscopy using deep learning deblurring

Most whole slide imaging (WSI) systems today rely on the "stop-and-stare" approach, where, at each field of view, the scanning stage is brought to a complete stop before the camera snaps a picture. This procedure ensures that each image is free of motion blur, which comes at the expense of long acquisition times. In order to speed up the acquisition process, especially for large scanning areas, such as pathology slides, we developed an acquisition method in which the data is acquired continuously while the stage is moving at high speeds. Using generative adversarial networks (GANs), we demonstrate this ultra-fast imaging approach, referred to as GANscan, which restores sharp images from motion blurred videos. GANscan allows us to complete image acquisitions at 30x the throughput of stop-and-stare systems. This method is implemented on a Zeiss Axio Observer Z1 microscope, requires no specialized hardware, and accomplishes successful reconstructions at stage speeds of up to 5000 μm/s. We validate the proposed method by imaging H&E stained tissue sections. Our method not only retrieves crisp images from fast, continuous scans, but also adjusts for defocusing that occurs during scanning within +/- 5 μm. Using a consumer GPU, the inference runs at <20 ms/ image.

Self-calibrated 3D differential phase contrast microscopy with optimized illumination

3D phase imaging recovers an object's volumetric refractive index from intensity and/or holographic measurements. Partially coherent methods, such as illumination-based differential phase contrast (DPC), are particularly simple to implement in a commercial brightfield microscope. 3D DPC acquires images at multiple focus positions and with different illumination source patterns in order to reconstruct 3D refractive index. Here, we present a practical extension of the 3D DPC method that does not require a precise motion stage for scanning the focus and uses optimized illumination patterns for improved performance. The user scans the focus by hand, using the microscope's focus knob, and the algorithm self-calibrates the axial position to solve for the 3D refractive index of the sample through a computational inverse problem. We further show that the illumination patterns can be optimized by an end-to-end learning procedure. Combining these two, we demonstrate improved 3D DPC with a commercial microscope whose only hardware modification is LED array illumination.

Probing pluripotency gene regulatory networks with quantitative live cell imaging

Live cell imaging uniquely enables the measurement of dynamic events in single cells, but it has not been used often in the study of gene regulatory networks. Network components can be examined in relation to one another by quantitative live cell imaging of fluorescent protein reporter cell lines that simultaneously report on more than one network component. A series of dual-reporter cell lines would allow different combinations of network components to be examined in individual cells. Dynamical information about interacting network components in individual cells is critical to predictive modeling of gene regulatory networks, and such information is not accessible through omics and other end point techniques. Achieving this requires that gene-edited cell lines are appropriately designed and adequately characterized to assure the validity of the biological conclusions derived from the expression of the reporters. In this brief review we discuss what is known about the importance of dynamics to network modeling and review some recent advances in optical microscopy methods and image analysis approaches that are making the use of quantitative live cell imaging for network analysis possible. We also discuss how strategies for genetic engineering of reporter cell lines can influence the biological relevance of the data.