All-optical microscope autofocus based on an electrically tunable lens and a totally internally reflected IR laser

A Rapid Nanofocusing Method for a Deep-Sea Gene Sequencing Microscope Based on Critical Illumination

In the deep-sea environment, the volume available for an in-situ gene sequencer is severely limited. In addition, optical imaging systems are subject to real-time, large-scale defocusing problems caused by ambient temperature fluctuations and vibrational perturbations. To address these challenges, we propose an edge detection algorithm for defocused images based on grayscale gradients and establish a defocus state detection model with nanometer resolution capabilities by relying on the inherent critical illumination light field. The model has been applied to a prototype deep-sea gene sequencing microscope with a 20× objective. It has demonstrated the ability to focus within a dynamic range of ±40 μm with an accuracy of 200 nm by a single iteration within 160 ms. By increasing the number of iterations and exposures, the focusing accuracy can be refined to 78 nm within a dynamic range of ±100 μm within 1.2 s. Notably, unlike conventional photoelectric hill-climbing, this method requires no additional hardware and meets the wide dynamic range, speed, and high-accuracy autofocusing requirements of deep-sea gene sequencing in a compact form factor.

Fast image-free autofocus method for passive FSPI microscopy

Autofocus is crucial for capturing sharp images with imaging devices for information acquisition. Traditional autofocus strategies based on post-processing become less efficient for passive FSPI microscopy of yet low temporal resolution. In this Letter, a fast and image-free autofocus system is proposed for passive FSPI microscopy. Based on the complementary design of an optical path, the system can measure the focus degree at 5000 fps while maintaining a high light efficiency for imaging. The proposed system can be easily combined with existing trinocular microscopes, which provides a welcomed boost to the practicability of passive FSPI microscopy.

Astigmatism-based active focus stabilisation with universal objective lens compatibility, extended operating range and nanometer precision

Focus stabilisation is vital for long-term fluorescence imaging, particularly in the case of high-resolution imaging techniques. Current stabilisation solutions either rely on fiducial markers that can be perturbative, or on beam reflection monitoring that is limited to high-numerical aperture objective lenses, making multimodal and large-scale imaging challenging. We introduce a beam-based method that relies on astigmatism, which offers advantages in terms of precision and the range over which focus stabilisation is effective. This approach is shown to be compatible with a wide range of objective lenses (10x-100x), typically achieving <10 nm precision with >10 μm operating range. Notably, our technique is largely unaffected by pointing stability errors, which in combination with implementation through a standalone Raspberry Pi architecture, offers a versatile focus stabilisation unit that can be added onto most existing microscope setups.

High-robustness autofocusing method in the microscope with laser-based arrayed spots

Accurate and rapid autofocus technology plays a crucial role in various fields, including automatic optical inspection technology, bio-chips scanning, and semiconductor manufacturing. The current photoelectric autofocus methods have limitations because of detecting the focal plane solely at the center of the microscope field of view. In the application of Stereo-seq the risk of autofocus errors will be increased, which have reduced the robustness of the system, like when the surface of the tested samples are wrinkling and inconsistent thickness, or the detection spot is at the edge of the sample. To enhance the robustness of the autofocus system and mitigate the constraints of the photoelectric autofocus methods, the laser-based arrayed spots photoelectric autofocus method has been proposed. To achieve the uniform light splitting, a 2D-Dammann grating is incorporated into the optical path of the autofocus system, resulting in the formation of an n × n arrayed spots on the surface of the sample. Through experimental verification, it has been demonstrated that this method can achieve the autofocus range of ±100μm and the autofocus accuracy of ±1/4 DOF when applied to a microscope equipped with a 10× objective lens, thereby satisfying the requirements for microscopic focusing. The arrayed light autofocus method devised in this study presents what we believe is a novel research concept for active autofocus detection and holds significant application value.

Autofocus Fourier single-pixel microscopy

Single-pixel microscopy enables observation of micro samples in invisible wave bands. Finding the focus position is essential to capture a clear image of a sample but could be difficult for single-pixel microscopy particularly in invisible wave bands. It is because the structured patterns projected onto the sample would be invisible and searching for the focus position manually could be exhausting. Here, we report an autofocus method for Fourier single-pixel microscopy. The reported method allows one to find the focus position without recording or reconstructing a complete image. The focus position is determined by the magnitude summation of a small number of Fourier coefficients, which enables fast autofocus. The reported method is experimentally demonstrated in imaging various objects in both visible and near-infrared wave bands. The method adds practicability to a single-pixel microscopy.

Simple, non-mechanical and automatic calibration approach for axial-scanning microscopy with an electrically tunable lens

We describe a simple and robust calibration approach for axial-scanning microscopy that realizes axial focus shifts with an electrically tunable lens (ETL). We demonstrate the calibration approach based on a microscope with an ETL placed close to the rear stop of the objective lens. By introducing a target-consisted of repeating lines at one known frequency and placed at a ~45° angle to the imaging path, the calibration method captures multiple images at different ETL currents and calibrates the dependence of the axial focus shift on the ETL current by evaluating the sharpness of the captured images. It calibrates the dependence of the magnification of the microscope on the ETL current by measuring the period of the repeating lines in the captured images. The experimental results show that different from the conventional calibration procedure, the proposed scheme does not involve any mechanical scanning and can simultaneously calibrate the dependence of the axial focus shift and the magnification on the ETL current. This might facilitate imaging studies that require the measurement of fine structures in a 3D volume. We also show the calibration procedure can be used to estimate the radius of a conner-arc sample, fabricated using laser micromachining. We believe that this easy-to-use calibration approach may facilitate use of ETLs for a variety of imaging platforms. It may also provide new insights for the development of novel 3D surface measurement methods. RESEARCH HIGHLIGHTS: The proposed calibration scheme does not involve any mechanical scanning and can simultaneously calibrate the dependence of the axial focus shift and the magnification on the electrically tunable lens (ETL) current. It might facilitate imaging studies that require the measurement of fine structures in a 3D volume, and the use of ETLs for a variety of imaging platforms. It may also provide new insights for the development of novel 3D surface measurement methods.

Rectangular Amplitude Mask-Based Auto-Focus Method with a Large Range and High Precision for a Micro-LED Wafer Defects Detection System

Auto-focus technology plays an important role in the Micro-LED wafer defects detection system. How to accurately measure the defocus amount and the defocus direction of the Micro-LED wafer sample in a large linear range is one of the keys to realizing wafer defects detection. In this paper, a large range and high-precision auto-focus method based on a rectangular amplitude mask is proposed. A rectangular amplitude mask without a long edge is used to modulate the shape of the incident laser beams so that the spot shape distribution of the reflected laser beam on the sensor changes with the defocus amount of the wafer sample. By calculating the shape of the light spots, the defocus amount and the defocus direction can be obtained at the same time. The experimental results show that under the 20× microscopy objective, the linear range of the auto-focus system is 480 μm and the accuracy can reach 1 μm. It can be seen that the automatic focusing method proposed in this paper has the advantages of large linear range, high accuracy, and compact structure, which can meet the requirements of the Micro-LED wafer defects detection equipment.

An autofocusing method for dynamic surface-enhanced Raman spectroscopy detection realized by optimized hill-climbing algorithm with long time stable hotspots

In the manual dynamic surface-enhanced Raman spectroscopy (D-SERS) detection process, it is difficult to focus on sample drop due to the constantly changing hotspot and easy judgment method. In this paper, we proposed an automatic focusing method based on long time stable hotspot with aid of optimization of hill-climbing algorithm and achieved on a designed device. First, set up a high temperature accelerating evaporation process to obtain hotspot and then cool to a low temperature rapidly to maintain it. Then, the spectral intensity was used as a focus of feedback signal in optimized hill-climbing algorithm to drive the sample stage to move up and down to adjust the depth of the laser on the samples to realize automatic focusing. As a result, the hotspot can be maintained for 5 min, and the autofocusing result can be achieved within 9 s, while the sensitivity was improved with two orders of magnitude in D-SERS detection of crystal violet (CV) compared with manual focusing.

Automated Camera Lucida Method with Colored Images through Integration of Hardware and Software in Microscopic Zooming

Automating the camera Lucida method which is a standard way for focusing microscopic images is a very challenging study for many scientists. Hence, actually combining hardware and software to automate microscopic imaging systems is one of the most important issues in the field of medicine as well. This idea reduces scanning time and increases the accuracy of user's results in this field. Closed-loop control system has been designed and implemented in the hardware part to move the stage in predefined limits of 15°. This system produces 50 consecutive images from parasites at the mentioned spatial distances in two directions of the z-axis. Then, by introducing our proposed relational software with combining images, a high-contrast image can be presented. This colored image is focused on many subparts of the sample even with different ruggedness. After implementing the closed-loop controller, stages movement was repeated eight times with an average step displacement of 20 μm which were measured in two directions of the z-axis by a digital micrometer. On average, the movement's error was 1 μm. In software, the edge intensity energy index has been calculated for image quality evaluation. The standard camera Lucida method has been simulated with acceptable results based on experts' opinions and also mean squared error parameters. Mechanical movement in stage has an accuracy of about 95% which will meet the expectations of laboratory user. Although output-focused colored images from our combining software can be replaced by the traditional fully accepted Camera Lucida method.

Experimental characterization, modelling and compensation of temperature effects in optotunable lenses

Most tunable lenses (TLs) are affected by deviations in optical power induced by external temperature changes or due to internal heating while in use. This study proposes: (1) An experimental characterization method to evaluate the magnitude of the optical power deviations due to internal temperature shifts; (2) three different mathematical models (experimental, polynomial, and optimized) to describe the response of the lens with temperature; (3) predictions of the internal temperature shifts while using the lens in time frames of minutes, seconds, and milliseconds and; (4) a real time optical power compensation tool based on the implementation of the models on a custom voltage electronic driver. The compensation methods were successfully applied to two TL samples in static and dynamic experiments and in hysteresis cycles. After 40 min at a static nominal power of 5 diopters (dpt), the internal temperature exponentially increased by 17 °C, producing an optical power deviation of 1.0 dpt (1.5 dpt when the lens cools down), representing a 20% distortion for heating and 30% for cooling. Modelling and compensation reduced the deviations to 0.2 dpt when heating (0.35 dpt when cooling) and the distortions to 4% and 7%. Similar levels of improvement were obtained in dynamic and hysteresis experiments. Compensation reduced temperature effects by more than 75%, representing a significant improvement in the performance of the lens.

Closed-loop experimental optimization of tunable lenses

Tunable lenses (TLs) are optical devices that can change their optical power in response to an electrical signal. In many applications, they are often pushed to or beyond their temporal limits. Fast periodic and/or abrupt variations of the optical power induce undesired distortions in their transient response and produce a decrease in their performance. A low-cost focimetry system, along with a custom closed-loop iterative optimization algorithm, was developed to (1) characterize a TL's response at high speed and (2) optimize their performance in realistic TL working conditions. A significant lens performance improvement was found in about 23 iterations with a decrease in the area under the error curve and an improved effective time. Applying the closed-loop optimization algorithm in a depth scanning experiment enhanced the image quality. Quantitatively, the image quality was evaluated using the structural similarity index metric that improves in individual frames, on average, from 0.345 to 0.895.

Fast autofocusing based on pixel difference with the Tanimoto coefficient between images

Focusing objects accurately over short time scales is an essential and nontrivial task for a variety of microscopy applications. In this Letter, an autofocusing algorithm using pixel difference with the Tanimoto coefficient (PDTC) is described to predict the focus. Our method can robustly distinguish differences in clarity among datasets. The generated auto-focusing curves have extremely high sensitivity. A dataset of a defocused stack acquired by an Olympus microscope demonstrates the feasibility of our technique. This work can be applied in full-color microscopic imaging systems and is also valid for single-color imaging.

Quantized spiral-phase-modulation based deep learning for real-time defocusing distance prediction

Whole slide imaging (WSI) has become an essential tool in pathological diagnosis, owing to its convenience on remote and collaborative review. However, how to bring the sample at the optimal position in the axial direction and image without defocusing artefacts is still a challenge, as traditional methods are either not universal or time-consuming. Until recently, deep learning has been shown to be effective in the autofocusing task in predicting defocusing distance. Here, we apply quantized spiral phase modulation on the Fourier domain of the captured images before feeding them into a light-weight neural network. It can significantly reduce the average predicting error to be lower than any previous work on an open dataset. Also, the high predicting speed strongly supports it can be applied on an edge device for real-time tasks with limited computational source and memory footprint.

Illumination angle correction during image acquisition in light-sheet fluorescence microscopy using deep learning

Light-sheet fluorescence microscopy (LSFM) is a high-speed imaging technique that provides optical sectioning with reduced photodamage. LSFM is routinely used in life sciences for live cell imaging and for capturing large volumes of cleared tissues. LSFM has a unique configuration, in which the illumination and detection paths are separated and perpendicular to each other. As such, the image quality, especially at high resolution, largely depends on the degree of overlap between the detection focal plane and the illuminating beam. However, spatial heterogeneity within the sample, curved specimen boundaries, and mismatch of refractive index between tissues and immersion media can refract the well-aligned illumination beam. This refraction can cause extensive blur and non-uniform image quality over the imaged field-of-view. To address these issues, we tested a deep learning-based approach to estimate the angular error of the illumination beam relative to the detection focal plane. The illumination beam was then corrected using a pair of galvo scanners, and the correction significantly improved the image quality across the entire field-of-view. The angular estimation was based on calculating the defocus level on a pixel level within the image using two defocused images. Overall, our study provides a framework that can correct the angle of the light-sheet and improve the overall image quality in high-resolution LSFM 3D image acquisition.

Deep learning-based single-shot autofocus method for digital microscopy

Digital pathology is being transformed by artificial intelligence (AI)-based pathological diagnosis. One major challenge for correct AI diagnoses is to ensure the focus quality of captured images. Here, we propose a deep learning-based single-shot autofocus method for microscopy. We use a modified MobileNetV3, a lightweight network, to predict the defocus distance with a single-shot microscopy image acquired at an arbitrary image plane without secondary camera or additional optics. The defocus prediction takes only 9 ms with a focusing error of only ∼1/15 depth of field. We also provide implementation examples for the augmented reality microscope and the whole slide imaging (WSI) system. Our proposed technique can perform real-time and accurate autofocus which will not only support pathologists in their daily work, but also provide potential applications in the life sciences, material research, and industrial automatic detection.

Fluorescent toys 'n' tools lighting the way in fungal research

Although largely overlooked compared to bacterial infections, fungal infections pose a significant threat to the health of humans and other organisms. Many pathogenic fungi, especially Candida species, are extremely versatile and flexible in adapting to various host niches and stressful situations. This leads to high pathogenicity and increasing resistance to existing drugs. Due to the high level of conservation between fungi and mammalian cells, it is hard to find fungus-specific drug targets for novel therapy development. In this respect, it is vital to understand how these fungi function on a molecular, cellular as well as organismal level. Fluorescence imaging allows for detailed analysis of molecular mechanisms, cellular structures and interactions on different levels. In this manuscript, we provide researchers with an elaborate and contemporary overview of fluorescence techniques that can be used to study fungal pathogens. We focus on the available fluorescent labelling techniques and guide our readers through the different relevant applications of fluorescent imaging, from subcellular events to multispecies interactions and diagnostics. As well as cautioning researchers for potential challenges and obstacles, we offer hands-on tips and tricks for efficient experimentation and share our expert-view on future developments and possible improvements.

Auto-focusing and quantitative phase imaging using deep learning for the incoherent illumination microscopy system

It is well known that the quantitative phase information which is vital in the biomedical study is hard to be directly obtained with bright-field microscopy under incoherent illumination. In addition, it is impossible to maintain the living sample in focus over long-term observation. Therefore, both the autofocusing and quantitative phase imaging techniques have to be solved in microscopy simultaneously. Here, we propose a lightweight deep learning-based framework, which is constructed by residual structure and is constrained by a novel loss function model, to realize both autofocusing and quantitative phase imaging. It outputs the corresponding in-focus amplitude and phase information at high speed (10fps) from a single-shot out-of-focus bright-field image. The training data were captured with a designed system under a hybrid incoherent and coherent illumination system. The experimental results verify that the focused and quantitative phase images of non-biological samples and biological samples can be reconstructed by using the framework. It provides a versatile quantitative technique for continuous monitoring of living cells in long-term and label-free imaging by using a traditional incoherent illumination microscopy system.

Spatial Location in Integrated Circuits through Infrared Microscopy

In this paper, we present an infrared microscopy based approach for structures' location in integrated circuits, to automate their secure characterization. The use of an infrared sensor is the key device for internal integrated circuit inspection. Two main issues are addressed. The first concerns the scan of integrated circuits using a motorized optical system composed of an infrared uncooled camera combined with an optical microscope. An automated system is required to focus the conductive tracks under the silicon layer. It is solved by an autofocus system analyzing the infrared images through a discrete polynomial image transform which allows an accurate features detection to build a focus metric robust against specific image degradation inherent to the acquisition context. The second issue concerns the location of structures to be characterized on the conductive tracks. Dealing with a large amount of redundancy and noise, a graph-matching method is presented-discriminating graph labels are developed to overcome the redundancy, while a flexible assignment optimizer solves the inexact matching arising from noises on graphs. The resulting automated location system brings reproducibility for secure characterization of integrated systems, besides accuracy and time speed increase.

Challenges facing quantitative large-scale optical super-resolution, and some simple solutions

Optical super-resolution microscopy (SRM) has enabled biologists to visualize cellular structures with near-molecular resolution, giving unprecedented access to details about the amounts, sizes, and spatial distributions of macromolecules in the cell. Precisely quantifying these molecular details requires large datasets of high-quality, reproducible SRM images. In this review, we discuss the unique set of challenges facing quantitative SRM, giving particular attention to the shortcomings of conventional specimen preparation techniques and the necessity for optimal labeling of molecular targets. We further discuss the obstacles to scaling SRM methods, such as lengthy image acquisition and complex SRM data analysis. For each of these challenges, we review the recent advances in the field that circumvent these pitfalls and provide practical advice to biologists for optimizing SRM experiments.

A Single-Shot Autofocus Approach for Surface Plasmon Resonance Microscopy

Surface plasmon resonance microscopy (SPRM) has been widely used as a sensitive imaging platform for chemical and biological analysis. The SPRM system inevitably suffers from focus inhomogeneity and drifts, especially in long-term recordings, leading to distorted images and inaccurate quantification. Traditional focus correction approaches require additional optical parts to detect and adjust focal conditions. Herein, we propose a deep-learning-based image processing method to gain autofocused SPRM images, without increasing the complexity of the optical systems. We trained a generative adversarial network (GAN) model with thousands of SPRM images of nanoparticles acquired at different focal distances. The trained model was able to directly generate focused SPRM images from single-shot defocused images, with no prior knowledge of the focus conditions during recording. Experiments using Au nanoparticles show that this method is effective in both static and time-lapse monitoring. The proposed autofocus technique thus provides an approach for improving the consistency among SPRM studies and for long-term monitoring.

A micromirror array with annular partitioning for high-speed random-access axial focusing

Dynamic axial focusing functionality has recently experienced widespread incorporation in microscopy, augmented/virtual reality (AR/VR), adaptive optics and material processing. However, the limitations of existing varifocal tools continue to beset the performance capabilities and operating overhead of the optical systems that mobilize such functionality. The varifocal tools that are the least burdensome to operate (e.g. liquid crystal, elastomeric or optofluidic lenses) suffer from low (≈100 Hz) refresh rates. Conversely, the fastest devices sacrifice either critical capabilities such as their dwelling capacity (e.g. acoustic gradient lenses or monolithic micromechanical mirrors) or low operating overhead (e.g. deformable mirrors). Here, we present a general-purpose random-access axial focusing device that bridges these previously conflicting features of high speed, dwelling capacity and lightweight drive by employing low-rigidity micromirrors that exploit the robustness of defocusing phase profiles. Geometrically, the device consists of an 8.2 mm diameter array of piston-motion and 48-μm-pitch micromirror pixels that provide 2π phase shifting for wavelengths shorter than 1100 nm with 10-90% settling in 64.8 μs (i.e., 15.44 kHz refresh rate). The pixels are electrically partitioned into 32 rings for a driving scheme that enables phase-wrapped operation with circular symmetry and requires <30 V per channel. Optical experiments demonstrated the array's wide focusing range with a measured ability to target 29 distinct resolvable depth planes. Overall, the features of the proposed array offer the potential for compact, straightforward methods of tackling bottlenecked applications, including high-throughput single-cell targeting in neurobiology and the delivery of dense 3D visual information in AR/VR.

Development and evaluation of a hand-held fiber-optic Raman probe with an integrated autofocus unit

Current implementations of fiber-optic Raman spectroscopy probes are frequently based on non-contact probes with a fixed focus and thus and have to precisely maintain the probe-to-sample distance to ensure a sufficient signal collection. We propose and experimentally demonstrate a novel hand-held fiber-optic Raman probe design, which is based on a liquid lens autofocusing unit, combined with a distance sensor and an in-house developed algorithm to precisely determine the probe-to-sample distance. The reported probe significantly improves the signal stability even for hand-held operation, while reducing distance-dependent artifacts for the acquisition of Raman spectra and can improve the acquisition of Raman spectra in a variety of applications.

Optical performance evaluation and chromatic aberration correction of a focus tunable lens used for 3D microscopy

Recently, the development of motion-free 3D microscopy utilizing focus tunable lenses (FTL) has been rapid. However, the downgrade of optical performance due to FTL and its gravity effect are rarely discussed in detail. Also, color dispersion is usually maintained purely depending on the FTL material without further correction. In this manuscript, we provide a quantitative evaluation of the impact of FTL on the optical performance of the microscope. The evaluation is based on both optical ray tracing simulations and lab experiments. In addition, we derive the first order conditions to correct axial color aberration of FTL through its entire power tuning range. Secondary spectrum correction is also possible and an apochromatic motion-free 3D microscope with 2 additional doublets is demonstrated. This study will serve a guidance in utilizing FTL as a motion-free element for 3D microscopy.

VIEW-MOD: a versatile illumination engine with a modular optical design for fluorescence microscopy

We developed VIEW-MOD (Versatile Illumination Engine with a Modular Optical Design): a compact, multi-modality microscope, which accommodates multiple illumination schemes including variable angle total internal reflection, point scanning and vertical/horizontal light sheet. This system allows combining and flexibly switching between different illuminations and imaging modes by employing three electrically tunable lenses and two fast-steering mirrors. This versatile optics design provides control of 6 degrees of freedom of the illumination source (3 translation, 2 tilt, and beam shape) plus the axial position of the imaging plane. We also developed standalone software with an easy-to-use GUI to calibrate and control the microscope. We demonstrate the applications of this system and software in biosensor imaging, optogenetics and fast 3D volume imaging. This system is ready to fit into complex imaging circumstances requiring precise control of illumination and detection paths, and has a broad scope of usability for a myriad of biological applications.

Using time-lapse fluorescence microscopy to study gene regulation

Time-lapse fluorescence microscopy is a powerful tool to study gene regulation. By probing fluorescent signals in single cells over extended period of time, this method can be used to study the dynamics, noise, movement, memory, inheritance, and coordination, of gene expression during cell growth, development, and differentiation. In combination with a flow-cell device, it can also measure gene regulation by external stimuli. Due to the single cell nature and the spatial/temporal capacity, this method can often provide information that is hard to get using other methods. Here, we review the standard experimental procedures and new technical developments in this field.