Light field microscopy based on structured light illumination

LYNSU: automated 3D neuropil segmentation of fluorescent images for Drosophila brains

The brain atlas, which provides information about the distribution of genes, proteins, neurons, or anatomical regions, plays a crucial role in contemporary neuroscience research. To analyze the spatial distribution of those substances based on images from different brain samples, we often need to warp and register individual brain images to a standard brain template. However, the process of warping and registration may lead to spatial errors, thereby severely reducing the accuracy of the analysis. To address this issue, we develop an automated method for segmenting neuropils in the Drosophila brain for fluorescence images from the FlyCircuit database. This technique allows future brain atlas studies to be conducted accurately at the individual level without warping and aligning to a standard brain template. Our method, LYNSU (Locating by YOLO and Segmenting by U-Net), consists of two stages. In the first stage, we use the YOLOv7 model to quickly locate neuropils and rapidly extract small-scale 3D images as input for the second stage model. This stage achieves a 99.4% accuracy rate in neuropil localization. In the second stage, we employ the 3D U-Net model to segment neuropils. LYNSU can achieve high accuracy in segmentation using a small training set consisting of images from merely 16 brains. We demonstrate LYNSU on six distinct neuropils or structures, achieving a high segmentation accuracy comparable to professional manual annotations with a 3D Intersection-over-Union (IoU) reaching up to 0.869. Our method takes only about 7 s to segment a neuropil while achieving a similar level of performance as the human annotators. To demonstrate a use case of LYNSU, we applied it to all female Drosophila brains from the FlyCircuit database to investigate the asymmetry of the mushroom bodies (MBs), the learning center of fruit flies. We used LYNSU to segment bilateral MBs and compare the volumes between left and right for each individual. Notably, of 8,703 valid brain samples, 10.14% showed bilateral volume differences that exceeded 10%. The study demonstrated the potential of the proposed method in high-throughput anatomical analysis and connectomics construction of the Drosophila brain.

Efficient high-resolution fluorescence projection imaging over an extended depth of field through optical hardware and deep learning optimizations

Optical microscopy has witnessed notable advancements but has also become more costly and complex. Conventional wide field microscopy (WFM) has low resolution and shallow depth-of-field (DOF), which limits its applications in practical biological experiments. Recently, confocal and light sheet microscopy become major workhorses for biology that incorporate high-precision scanning to perform imaging within an extended DOF but at the sacrifice of expense, complexity, and imaging speed. Here, we propose deep focus microscopy, an efficient framework optimized both in hardware and algorithm to address the tradeoff between resolution and DOF. Our deep focus microscopy achieves large-DOF and high-resolution projection imaging by integrating a deep focus network (DFnet) into light field microscopy (LFM) setups. Based on our constructed dataset, deep focus microscopy features a significantly enhanced spatial resolution of ∼260 nm, an extended DOF of over 30 µm, and broad generalization across diverse sample structures. It also reduces the computational costs by four orders of magnitude compared to conventional LFM technologies. We demonstrate the excellent performance of deep focus microscopy in vivo, including long-term observations of cell division and migrasome formation in zebrafish embryos and mouse livers at high resolution without background contamination.

Highly focused beam generated with a height tuned micro-optical structure for high contrast microscopic imaging

Light sheet illumination technology improves the signal-to-noise ratio, resolution, and reduces scattered backgrounds for biological microscopic detection system. Here, we developed a novel micro-optical structure to produce a focused and uniform beam for the enhancement of imaging contrast. The beam intensity and working distance can be modified by adjusting the height and period of the structure. Our experiments successfully recorded structured light illumination, demonstrating the ability of the structure to capture high-contrast imaging data. We compared the light fields generated with and without the structure to assess the imaging quality, revealing a maximum 4.78-fold improvement in the signal-to-noise ratio. This work provides a potential method for high-resolution and high-contrast light sheet fluorescence microscopic detection.

High-resolution 3D imaging in light-field microscopy through Stokes matrices and data fusion

The trade-off between the lateral and vertical resolution has long posed challenges to the efficient and widespread application of Fourier light-field microscopy, a highly scalable 3D imaging tool. Although existing methods for resolution enhancement can improve the measurement result to a certain extent, they come with limitations in terms of accuracy and applicable specimen types. To address these problems, this paper proposed a resolution enhancement scheme utilizing data fusion of polarization Stokes vectors and light-field information for Fourier light-field microscopy system. By introducing the surface normal vector information obtained from polarization measurement and integrating it with the light-field 3D point cloud data, 3D reconstruction results accuracy is highly improved in axial direction. Experimental results with a Fourier light-field 3D imaging microscope demonstrated a substantial enhancement of vertical resolution with a depth resolution to depth of field ratio of 0.19%. This represented approximately 44 times the improvement compared to the theoretical ratio before data fusion, enabling the system to access more detailed information with finer measurement accuracy for test samples. This work not only provides a feasible solution for breaking the limitations imposed by traditional light-field microscope hardware configurations but also offers superior 3D measurement approach in a more cost-effective and practical manner.

Multiplane HiLo microscopy with speckle illumination and non-local means denoising

Significance

HiLo microscopy synthesizes an optically sectioned image from two images, one obtained with uniform and another with patterned illumination, such as laser speckle. Speckle-based HiLo has the advantage of being robust to aberrations but is susceptible to residual speckle noise that is difficult to control. We present a computational method to reduce this residual noise without undermining resolution. In addition, we improve the versatility of HiLo microscopy by enabling simultaneous multiplane imaging (here nine planes).

Aim

Our goal is to perform fast, high-contrast, multiplane imaging with a conventional camera-based fluorescence microscope.

Approach

Multiplane HiLo imaging is achieved with the use of a single camera and z-splitter prism. Speckle noise reduction is based on the application of a non-local means (NLM) denoising method to perform ensemble averaging of speckle grains.

Results

We demonstrate the capabilities of multiplane HiLo with NLM denoising both with synthesized data and by imaging cardiac and brain activity in zebrafish larvae at 40 Hz frame rates.

Conclusions

Multiplane HiLo microscopy aided by NLM denoising provides a simple tool for fast optically sectioned volumetric imaging that can be of general utility for fluorescence imaging applications.

Background inhibited and speed-loss-free volumetric imaging in vivo based on structured-illumination Fourier light field microscopy

Benefiting from its advantages in fast volumetric imaging for recording biodynamics, Fourier light field microscopy (FLFM) has a wide range of applications in biomedical research, especially in neuroscience. However, the imaging quality of the FLFM is always deteriorated by both the out-of-focus background and the strong scattering in biological samples. Here we propose a structured-illumination and interleaved-reconstruction based Fourier light field microscopy (SI-FLFM), in which we can filter out the background fluorescence in FLFM without sacrificing imaging speed. We demonstrate the superiority of our SI-FLFM in high-speed, background-inhibited volumetric imaging of various biodynamics in larval zebrafish and mice in vivo. The signal-to-background ratio (SBR) is improved by tens of times. And the volumetric imaging speed can be up to 40 Hz, avoiding artifacts caused by temporal under-sampling in conventional structured illumination microscopy. These suggest that our SI-FLFM is suitable for applications of weak fluorescence signals but high imaging speed requirements.

Phase-unwrapping-free 3D reconstruction in structured light field system based on varied auxiliary point

Three-dimensional (3D) reconstruction is an essential task in structured light field (SLF) related techniques and applications. This paper presents a new method to reconstruct a 3D object point by using many auxiliary points adjacent to it. The relationship between two points in a SLF system is derived. Different from conventional "direct" methods that reconstruct 3D coordinates of the object point by using phase, slope, disparity etc., the proposed method is an "indirect" method as the 3D coordinates of auxiliary points are not needed. Based on the auxiliary point theory, the wrapped phase obtained by 4-step phase-shifting method is sufficient for 3D reconstruction, without the need for phase unwrapping. To the best of our knowledge, this is the first strategy that combines the intrinsic characteristics of structured light and light field for phase-unwrapping-free 3D reconstruction. This paper also analyzes the constraints between system architecture parameters and phase rectification, phase to depth ratio, and presents a relatively simple criterion to guide the system design. Experimental results show that, with an appropriate system architecture, the proposed method can realize accurate, unambiguous, and reliable 3D reconstruction without phase unwrapping.