A frequency domain SIM reconstruction algorithm using reduced number of images

Superresolution structured illumination microscopy reconstruction algorithms: a review

Structured illumination microscopy (SIM) has become the standard for next-generation wide-field microscopy, offering ultrahigh imaging speed, superresolution, a large field-of-view, and long-term imaging. Over the past decade, SIM hardware and software have flourished, leading to successful applications in various biological questions. However, unlocking the full potential of SIM system hardware requires the development of advanced reconstruction algorithms. Here, we introduce the basic theory of two SIM algorithms, namely, optical sectioning SIM (OS-SIM) and superresolution SIM (SR-SIM), and summarize their implementation modalities. We then provide a brief overview of existing OS-SIM processing algorithms and review the development of SR-SIM reconstruction algorithms, focusing primarily on 2D-SIM, 3D-SIM, and blind-SIM. To showcase the state-of-the-art development of SIM systems and assist users in selecting a commercial SIM system for a specific application, we compare the features of representative off-the-shelf SIM systems. Finally, we provide perspectives on the potential future developments of SIM.

High-speed spatially re-modulated structured illumination microscopy

Structured illumination microscopy (SIM) allows non-invasive visualization of nanoscale subcellular structures. However, image acquisition and reconstruction become the bottleneck to further improve the imaging speed. Here, we propose a method to accelerate SIM imaging by combining the spatial re-modulation principle with Fourier domain filtering and using measured illumination patterns. This approach enables high-speed, high-quality imaging of dense subcellular structures using a conventional nine-frame SIM modality without phase estimation of the patterns. In addition, seven-frame SIM reconstruction and additional hardware acceleration further improve the imaging speed using our method. Furthermore, our method is also applicable to other spatially uncorrelated illumination patterns, such as distorted sinusoidal, multifocal, and speckle patterns.

Small-molecule fluorogenic probes for mitochondrial nanoscale imaging

Mitochondria are inextricably linked to the development of diseases and cell metabolism disorders. Super-resolution imaging (SRI) is crucial in enhancing our understanding of mitochondrial ultrafine structures and functions. In addition to high-precision instruments, super-resolution microscopy relies heavily on fluorescent materials with unique photophysical properties. Small-molecule fluorogenic probes (SMFPs) have excellent properties that make them ideal for mitochondrial SRI. This paper summarizes recent advances in the field of SMFPs, with a focus on the chemical and spectroscopic properties required for mitochondrial SRI. Finally, we discuss future challenges in this field, including the design principles of SMFPs and nanoscopic techniques.

Electron-beam patterned calibration structures for structured illumination microscopy

Super-resolution fluorescence microscopy can be achieved by image reconstruction after spatially patterned illumination or sequential photo-switching and read-out. Reconstruction algorithms and microscope performance are typically tested using simulated image data, due to a lack of strategies to pattern complex fluorescent patterns with nanoscale dimension control. Here, we report direct electron-beam patterning of fluorescence nanopatterns as calibration standards for super-resolution fluorescence. Patterned regions are identified with both electron microscopy and fluorescence labelling of choice, allowing precise correlation of predefined pattern dimensions, a posteriori obtained electron images, and reconstructed super-resolution images.

High-speed super-resolution imaging with compressive imaging-based structured illumination microscopy

Structured illumination microscopy (SIM) has been widely applied to investigating fine structures of biological samples by breaking the optical diffraction limitation. So far, video-rate imaging has been obtained in SIM, but the imaging speed was still limited due to the reconstruction of a super-solution image through multi-sampling, which hindered the applications in high-speed biomedical imaging. To overcome this limitation, here we develop compressive imaging-based structured illumination microscopy (CISIM) by synergizing SIM and compressive sensing (CS). Compared with conventional SIM, CISIM can greatly improve the super-resolution imaging speed by extracting multiple super-resolution images from one compressed image. Based on CISIM, we successfully reconstruct the super-resolution images in biological dynamics, and analyze the effect factors of image reconstruction quality, which verify the feasibility of CISIM. CISIM paves a way for high-speed super-resolution imaging, which may bring technological breakthroughs and significant applications in biomedical imaging.

Answering some questions about structured illumination microscopy

Structured illumination microscopy (SIM) provides images of fluorescent objects at an enhanced resolution greater than that of conventional epifluorescence wide-field microscopy. Initially demonstrated in 1999 to enhance the lateral resolution twofold, it has since been extended to enhance axial resolution twofold (2008), applied to live-cell imaging (2009) and combined with myriad other techniques, including interferometric detection (2008), confocal microscopy (2010) and light sheet illumination (2012). Despite these impressive developments, SIM remains, perhaps, the most poorly understood 'super-resolution' method. In this article, we provide answers to the 13 questions regarding SIM proposed by Prakash et al. along with answers to a further three questions. After providing a general overview of the technique and its developments, we explain why SIM as normally used is still diffraction-limited. We then highlight the necessity for a non-polynomial, and not just nonlinear, response to the illuminating light in order to make SIM a true, diffraction-unlimited, super-resolution technique. In addition, we present a derivation of a real-space SIM reconstruction approach that can be used to process conventional SIM and image scanning microscopy (ISM) data and extended to process data with quasi-arbitrary illumination patterns. Finally, we provide a simple bibliometric analysis of SIM development over the past two decades and provide a short outlook on potential future work. This article is part of the Theo Murphy meeting issue 'Super-resolution structured illumination microscopy (part 2)'.

SIM reconstruction framework for high-speed multi-dimensional super-resolution imaging

Structured illumination microscopy (SIM) holds great promise for live cell imaging applications due to its potential to obtain multidimensional information such as intensity, spectrum and polarization (I, λ , p) at high spatial-temporal resolution, enabling the observation of more complex dynamic interactions between subcellular structures. However, the reconstruction results of polarized samples are prone to artifacts because all current SIM reconstruction frameworks use incomplete imaging models which neglect polarization modulation. Such polarization-related artifacts are especially prevalent for SIM reconstruction using a reduced number of raw images (RSIM) and severely undermine the ability of SIM to capture multi-dimensional information. Here, we report a new SIM reconstruction framework (PRSIM) that can recover multi-dimensional information (I, λ, p) using a reduced number of raw images. PRSIM adopts a complete imaging model that is versatile for normal and polarized samples and uses a frequency-domain iterative reconstruction algorithm for artifact-free super-resolution (SR) reconstruction. It can simultaneously obtain the SR spatial structure and polarization orientation of polarized samples using 6 raw SIM images and can perform SR reconstruction using 4 SIM images for normal samples. In addition, PRSIM has less spatial computational complexity and achieves reconstruction speeds tens of times higher than that of the state-of-the-art non-iterative RSIM, making it more suitable for large field-of-view imaging. Thus, PRSIM is expected to facilitate the development of SIM into an ultra-high-speed and multi-dimensional SR imaging tool.

Advances of super-resolution fluorescence polarization microscopy and its applications in life sciences

Fluorescence polarization microscopy (FPM) analyzes both intensity and orientation of fluorescence dipole, and reflects the structural specificity of target molecules. It has become an important tool for studying protein organization, orientational order, and structural changes in cells. However, suffering from optical diffraction limit, conventional FPM has low orientation resolution and observation accuracy, as the polarization information is averaged by multiple fluorescent molecules within a diffraction-limited volume. Recently, novel super-resolution FPMs have been developed to break the diffraction barrier. In this review, we will introduce the recent progress to achieve sub-diffraction determination of dipole orientation. Biological applications, based on polarization analysis of fluorescence dipole, are also summarized, with focus on chromophore-target molecule interaction and molecular organization.

Deep learning enables structured illumination microscopy with low light levels and enhanced speed

Structured illumination microscopy (SIM) surpasses the optical diffraction limit and offers a two-fold enhancement in resolution over diffraction limited microscopy. However, it requires both intense illumination and multiple acquisitions to produce a single high-resolution image. Using deep learning to augment SIM, we obtain a five-fold reduction in the number of raw images required for super-resolution SIM, and generate images under extreme low light conditions (at least 100× fewer photons). We validate the performance of deep neural networks on different cellular structures and achieve multi-color, live-cell super-resolution imaging with greatly reduced photobleaching.

Super-resolution microscopy typically requires high laser powers which can induce photobleaching and degrade image quality. Here the authors augment structured illumination microscopy (SIM) with deep learning to reduce the number of raw images required and boost its performance under low light conditions.

Deep learning enables structured illumination microscopy with low light levels and enhanced speed

Structured illumination microscopy (SIM) surpasses the optical diffraction limit and offers a two-fold enhancement in resolution over diffraction limited microscopy. However, it requires both intense illumination and multiple acquisitions to produce a single high-resolution image. Using deep learning to augment SIM, we obtain a five-fold reduction in the number of raw images required for super-resolution SIM, and generate images under extreme low light conditions (at least 100× fewer photons). We validate the performance of deep neural networks on different cellular structures and achieve multi-color, live-cell super-resolution imaging with greatly reduced photobleaching.

Group-Sparsity-Based Super-Resolution Dipole Orientation Mapping

The dipole orientation of fluorophores could be resolved by fluorescence polarization microscopy (FPM), which in turn reveals structural specificity for the labeled organelles. Conventional FPM can detect only the averaged fluorescence anisotropy collected from dipoles within the diffraction-limited volume. Super-resolution dipole orientation mapping (SDOM) method, which applies sparse deconvolution and least square estimation to fluorescence polarization modulation data, achieves the dipole orientation measurement within a sub-diffraction focal area. However, during SDOM analysis, some pixels with fluorescence signal are not resolved with orientation for relatively small adjusted R². Here we report group-sparsity-based SDOM (GS-SDOM), which utilizes the relevance of modulation sequences to effectively improve the SDOM reconstruction model. More credible resolved dipole orientations with higher adjusted R² can be mapped and false positive estimation for local dipole orientation is vitally corrected. In addition to achieving the same spatial super-resolution as SDOM does, GS-SDOM accesses more morphological information with more credible orientations and more accurate local dipole distribution estimation. During the GS-SDOM analysis of actin filaments in mammalian kidney cells, the dipole orientation of fluorescence is detected always parallel to the direction of the actin filaments. Also with dipole orientation information extracted by GS-SDOM, the reconstructed visual circle from intensity dimension is discerned as jointed by double close filaments and 3-dimensional co-localization is accomplished in the intersection of actin filaments.

Structured illumination imaging without grating rotation based on mirror operation on 1D Fourier spectrum

Structured illumination microscopy (SIM) is a rapidly developing a super-resolution optical microscopy technique. With SIM, the grating is needed in order to rotate several angles for illuminating the sample in different directions. Multiple rotations reduce the imaging speed and grating rotation angle errors damage the image recovery quality. We introduce mirror transformation on one-dimension (1D) Fourier spectrum to SIM for resolving the problems of low imaging speed and severe impact on image reconstruction quality by grating rotation angle errors. When mirror operation and SIM are combined, the grating is placed at an orientation for obtaining three shadow images. The three shadow images are acquired by CCD at three different phase shift for a direction of grating. Thus, the SIM imaging speed is faster and the effect on image reconstruction quality by grating rotation angle errors is greatly reduced.

Tunable-frequency three-dimensional structured illumination microscopy with reduced data-acquisition

The performance of a tunable three-dimensional (3D) structured illumination microscope (SIM) system and its ability to provide simultaneously super-resolution (SR) and optical-sectioning (OS) capabilities are investigated. Numerical results show that the performance of our 3D-SIM system is comparable with the one provided by a three-wave interference SIM, while requiring 40% fewer images for the reconstruction and providing frequency tunability in a cost-effective implementation. The performance of the system has been validated experimentally with images from test samples, which were also imaged with a commercial SIM based on incoherent-grid projection for comparison. Restored images from data acquired from an axially-thin fluorescent layer show a 1.6× improvement in OS capability compared to the commercial instrument while results from a fluorescent tilted USAF target show the OS and SR capabilities achieved by our system.