Accelerating single molecule localization microscopy through parallel processing on a high-performance computing cluster

Novel tools to study cell-ECM interactions, cell adhesion dynamics and migration

Integrin-mediated cell adhesion is essential for cell migration, mechanotransduction and tissue integrity. In vivo, these processes are regulated by complex physicochemical signals from the extracellular matrix (ECM). These nuanced cues, including molecular composition, rigidity and topology, call for sophisticated systems to faithfully explore cell behaviour. Here, we discuss recent methodological advances in cell-ECM adhesion research and compile a toolbox of techniques that we expect to shape this field in future. We outline methodological breakthroughs facilitating the transition from rigid 2D substrates to more complex and dynamic 3D systems, as well as advances in super-resolution imaging for an in-depth understanding of adhesion nanostructure. Selected methods are exemplified with relevant biological findings to underscore their applicability in cell adhesion research. We expect this new "toolbox" of methods will allow for a closer approximation of in vitro experimental setups to in vivo conditions, providing deeper insights into physiological and pathophysiological processes associated with cell-ECM adhesion.

openFrame: A modular, sustainable, open microscopy platform with single-shot, dual-axis optical autofocus module providing high precision and long range of operation

'openFrame' is a modular, low-cost, open-hardware microscopy platform that can be configured or adapted to most light microscopy techniques and is easily upgradeable or expandable to multiple modalities. The ability to freely mix and interchange both open-source and proprietary hardware components or software enables low-cost, yet research-grade instruments to be assembled and maintained. It also enables rapid prototyping of advanced or novel microscope systems. For long-term time-lapse image data acquisition, slide-scanning or high content analysis, we have developed a novel optical autofocus incorporating orthogonal cylindrical optics to provide robust single-shot closed-loop focus lock, which we have demonstrated to accommodate defocus up to ±37 μm with <200 nm accuracy, and a two-step autofocus mode which we have shown can operate with defocus up to ±68 μm. We have used this to implement automated single molecule localisation microscopy (SMLM) in a relatively low-cost openFrame-based instrument using multimode diode lasers for excitation and cooled CMOS cameras.

Temporally resolved SMLM (with large PAR shift) enabled visualization of dynamic HA cluster formation and migration in a live cell

The blinking properties of a single molecule are critical for single-molecule localization microscopy (SMLM). Typically, SMLM techniques involve recording several frames of diffraction-limited bright spots of single-molecules with a detector exposure time close to the blinking period. This sets a limit on the temporal resolution of SMLM to a few tens of milliseconds. Realizing that a substantial fraction of single molecules emit photons for time scales much shorter than the average blinking period, we propose accelerating data collection to capture these fast emitters. Here, we put forward a short exposure-based SMLM (shortSMLM) method powered by sCMOS detector for understanding dynamical events (both at single molecule and ensemble level). The technique is demonstrated on an Influenza-A disease model, where NIH3T3 cells (both fixed and live cells) were transfected by Dendra2-HA plasmid DNA. Analysis shows a 2.76-fold improvement in the temporal resolution that comes with a sacrifice in spatial resolution, and a particle resolution shift PAR-shift (in terms of localization precision) of [Formula: see text] 11.82  nm compared to standard SMLM. We visualized dynamic HA cluster formation in transfected cells post 24 h of DNA transfection. It is noted that a reduction in spatial resolution does not substantially alter cluster characteristics (cluster density, [Formula: see text] molecules/cluster, cluster spread, etc.) and, indeed, preserves critical features. Moreover, the time-lapse imaging reveals the dynamic formation and migration of Hemagglutinin (HA) clusters in a live cell. This suggests that [Formula: see text] using a synchronized high QE sCMOS detector (operated at short exposure times) is excellent for studying temporal dynamics in cellular system.

Imaging Heterogeneous 3D Dynamics of Individual Solutes in a Polyelectrolyte Brush

Understanding molecular transport in polyelectrolyte brushes (PEBs) is crucial for applications such as separations, drug delivery, anti-fouling, and biosensors, where structural features of the polymer control intermolecular interactions. The complex structure and local heterogeneity of PEBs, while theoretically predicted, are not easily accessed with conventional experimental methods. In this work, we use 3D single-molecule tracking to understand transport behavior within a cationic poly(2-(N,N-dimethylamino)ethyl acrylate) (PDMAEA) brush using an anionic dye, Alexa Fluor 546, as the probe. The analysis is done by a parallelized, unbiased 3D tracking algorithm. Our results explicitly demonstrate that spatial heterogeneity within the brush manifests as heterogeneity of single-molecule displacements. Two distinct populations of probe motion are identified, with anticorrelated axial and lateral transport confinement, which we believe to correspond to intra- vs inter-chain probe motion.

Fluorescence microscopy imaging of a neurotransmitter receptor and its cell membrane lipid milieu

Hampered by the diffraction phenomenon, as expressed in 1873 by Abbe, applications of optical microscopy to image biological structures were for a long time limited to resolutions above the ∼200 nm barrier and restricted to the observation of stained specimens. The introduction of fluorescence was a game changer, and since its inception it became the gold standard technique in biological microscopy. The plasma membrane is a tenuous envelope of 4 nm-10 nm in thickness surrounding the cell. Because of its highly versatile spectroscopic properties and availability of suitable instrumentation, fluorescence techniques epitomize the current approach to study this delicate structure and its molecular constituents. The wide spectral range covered by fluorescence, intimately linked to the availability of appropriate intrinsic and extrinsic probes, provides the ability to dissect membrane constituents at the molecular scale in the spatial domain. In addition, the time resolution capabilities of fluorescence methods provide complementary high precision for studying the behavior of membrane molecules in the time domain. This review illustrates the value of various fluorescence techniques to extract information on the topography and motion of plasma membrane receptors. To this end I resort to a paradigmatic membrane-bound neurotransmitter receptor, the nicotinic acetylcholine receptor (nAChR). The structural and dynamic picture emerging from studies of this prototypic pentameric ligand-gated ion channel can be extrapolated not only to other members of this superfamily of ion channels but to other membrane-bound proteins. I also briefly discuss the various emerging techniques in the field of biomembrane labeling with new organic chemistry strategies oriented to applications in fluorescence nanoscopy, the form of fluorescence microscopy that is expanding the depth and scope of interrogation of membrane-associated phenomena.

Robust deep learning optical autofocus system applied to automated multiwell plate single molecule localization microscopy

We presenta robust, long-range optical autofocus system for microscopy utilizing machine learning. This can be useful for experiments with long image data acquisition times that may be impacted by defocusing resulting from drift of components, for example due to changes in temperature or mechanical drift. It is also useful for automated slide scanning or multiwell plate imaging where the sample(s) to be imaged may not be in the same horizontal plane throughout the image data acquisition. To address the impact of (thermal or mechanical) fluctuations over time in the optical autofocus system itself, we utilize a convolutional neural network (CNN) that is trained over multiple days to account for such fluctuations. To address the trade-off between axial precision and range of the autofocus, we implement orthogonal optical readouts with separate CNN training data, thereby achieving an accuracy well within the 600 nm depth of field of our 1.3 numerical aperture objective lens over a defocus range of up to approximately +/-100 μm. We characterize the performance of this autofocus system and demonstrate its application to automated multiwell plate single molecule localization microscopy.

Fast DNA-PAINT imaging using a deep neural network

DNA points accumulation for imaging in nanoscale topography (DNA-PAINT) is a super-resolution technique with relatively easy-to-implement multi-target imaging. However, image acquisition is slow as sufficient statistical data has to be generated from spatio-temporally isolated single emitters. Here, we train the neural network (NN) DeepSTORM to predict fluorophore positions from high emitter density DNA-PAINT data. This achieves image acquisition in one minute. We demonstrate multi-colour super-resolution imaging of structure-conserved semi-thin neuronal tissue and imaging of large samples. This improvement can be integrated into any single-molecule imaging modality to enable fast single-molecule super-resolution microscopy.

Accelerating multi-emitter localization in super-resolution localization microscopy with FPGA-GPU cooperative computation

The real-time multi-emitter localization method is essential for advancing high-throughput super-resolution localization microscopy (HT-SRLM). In the past decade, the graphics processing unit (GPU) computation has been dominantly used to accelerate the execution speed of the multi-emitter localization method. However, if HT-SRLM is combined with a scientific complementary metal-oxide-semiconductor (sCMOS) camera working at full frame rate, real-time image processing is still difficult to achieve using this acceleration approach, thus resulting in a massive data storage challenge and even system crash. Here we take advantage of the cooperative acceleration power of field programming gate array (FPGA) computation and GPU computation, and propose a method called HCP-STORM to enable real-time multi-emitter localization. Using simulated images, we verified that HCP-STORM is capable of providing real-time image processing for raw images from a representative Hamamatsu Flash 4 V3 sCMOS camera working at full frame rate (that is, 2048×2048 pixels @ 10 ms exposure time). Using experimental images, we prove that HCP-STORM is 25 times faster than QC-STORM and 295 times faster than ThunderSTORM, with a small but acceptable degradation in image quality. This study shows the potential of FPGA-GPU cooperative computation in accelerating multi-emitter localization, and pushes a significant step toward the maturity of HT-SRLM technology.

Accelerating 3D single-molecule localization microscopy using blind sparse inpainting

SIGNIFICANCE

Single-molecule localization-based super-resolution microscopy has enabled the imaging of microscopic objects beyond the diffraction limit. However, this technique is limited by the requirements of imaging an extremely large number of frames of biological samples to generate a super-resolution image, thus requiring a longer acquisition time. Additionally, the processing of such a large image sequence leads to longer data processing time. Therefore, accelerating image acquisition and processing in single-molecule localization microscopy (SMLM) has been of perennial interest.

AIM

To accelerate three-dimensional (3D) SMLM imaging by leveraging a computational approach without compromising the resolution.

APPROACH

We used blind sparse inpainting to reconstruct high-density 3D images from low-density ones. The low-density images are generated using much fewer frames than usually needed, thus requiring a shorter acquisition and processing time. Therefore, our technique will accelerate 3D SMLM without changing the existing standard SMLM hardware system and labeling protocol.

RESULTS

The performance of the blind sparse inpainting was evaluated on both simulation and experimental datasets. Superior reconstruction results of 3D SMLM images using up to 10-fold fewer frames in simulation and up to 50-fold fewer frames in experimental data were achieved.

CONCLUSIONS

We demonstrate the feasibility of fast 3D SMLM imaging leveraging a computational approach to reduce the number of acquired frames. We anticipate our technique will enable future real-time live-cell 3D imaging to investigate complex nanoscopic biological structures and their functions.

Super-resolution Microscopy with Single Molecules in Biology and Beyond-Essentials, Current Trends, and Future Challenges

Single-molecule super-resolution microscopy has developed from a specialized technique into one of the most versatile and powerful imaging methods of the nanoscale over the past two decades. In this perspective, we provide a brief overview of the historical development of the field, the fundamental concepts, the methodology required to obtain maximum quantitative information, and the current state of the art. Then, we will discuss emerging perspectives and areas where innovation and further improvement are needed. Despite the tremendous progress, the full potential of single-molecule super-resolution microscopy is yet to be realized, which will be enabled by the research ahead of us.

PySpark-Based Optimization of Microwave Image Reconstruction Algorithm for Head Imaging Big Data on High-Performance Computing and Google Cloud Platform

For processing large-scale medical imaging data, adopting high-performance computing and cloud-based resources are getting attention rapidly. Due to its low–cost and non-invasive nature, microwave technology is being investigated for breast and brain imaging. Microwave imaging via space-time algorithm and its extended versions are commonly used, as it provides high-quality images. However, due to intensive computation and sequential execution, these algorithms are not capable of producing images in an acceptable time. In this paper, a parallel microwave image reconstruction algorithm based on Apache Spark on high-performance computing and Google Cloud Platform is proposed. The input data is first converted to a resilient distributed data set and then distributed to multiple nodes on a cluster. The subset of pixel data is calculated in parallel on these nodes, and the results are retrieved to a master node for image reconstruction. Using Apache Spark, the performance of the parallel microwave image reconstruction algorithm is evaluated on high-performance computing and Google Cloud Platform, which shows an average speed increase of 28.56 times on four homogeneous computing nodes. Experimental results revealed that the proposed parallel microwave image reconstruction algorithm fully inherits the parallelism, resulting in fast reconstruction of images from radio frequency sensor’s data. This paper also illustrates that the proposed algorithm is generalized and can be deployed on any master-slave architecture.