Through-focus or volumetric type of optical imaging methods: a review

Machine Learning Enhanced Optical Microscopy for the Rapid Morphology Characterization of Silver Nanoparticles

The rapid characterization of nanoparticles for morphological information such as size and shape is essential for material synthesis as they are the determining factors for the optical, mechanical, and chemical properties and related applications. In this paper, we report a computational imaging platform to characterize nanoparticle size and morphology under conventional optical microscopy. We established a machine learning model based on a series of images acquired by through-focus scanning optical microscopy (TSOM) on a conventional optical microscope. This model predicts the size of silver nanocubes with an estimation error below 5% on individual particles. At the ensemble level, the estimation error is 1.6% for the averaged size and 0.4 nm for the standard deviation. The method can also identify the tip morphology of silver nanowires from the mix of sharp-tip and blunt-tip samples at an accuracy of 82%. Furthermore, we demonstrated online monitoring for the evolution of the size distribution of nanoparticles during synthesis. This method can be potentially extended to more complicated nanomaterials such as anisotropic and dielectric nanoparticles.

Defect height estimation via model-less TSOM under optical resolution

We propose a new method of through-focus scanning optical microscopy (TSOM) without a reference database, i.e., a model-less TSOM method. Building a TSOM reference database is time-consuming or even impractical in some TSOM applications that involve complex structures, such as 3D NAND, or irregular shapes such as defects. The proposed model-less TSOM method was used to determine just the height of defect particles, for the first time as far as we are aware. Defect height is the only relevant dimension for the display panel application. Specifically, we analyzed 40 organic light-emitting diode (OLED) surface defects using a lab-developed motion-free TSOM tool consisting of a 50× objective lens (numerical aperture (NA) 0.55), a 532-nm light source, an imaging detector with a 7.5-µm pitch, and a deformable mirror. The tool is in-line and capable of achieving high throughput non-destructively, both relevant features for industrial applications. We investigated linear regression relations between newly defined TSOM parameters (TSOM height, TSOM area and TSOM volume) and the defect heights, which were first measured by atomic force microscopy (AFM). Following defect classification based on in-focus images, we successfully found that the AFM height has a linear correlation with 50% TSOM height (H50%) within ± 20.3 nm (1σ) error over the range of 140 to 950 nm. The one-sigma error, i.e., 20.3 nm, was approximately λ/26 or 1/43 of the depth of focus (DOF) of the applied microscope.

Motion-free TSOM using a deformable mirror

Through-focus scanning optical microscopy (TSOM) is a model-based optical metrology method that involves the scanning of a target through the focus of an optical microscope. Unlike a conventional optical microscope that directly extracts the diffraction-limited optical information from a single in-focus image, the TSOM method extracts nanometer scale sensitive information by matching the target TSOM data/image to reference TSOM data/images that are either experimentally or computationally collected. Therefore, the sensitivity and accuracy of the TSOM method strongly depends on the similarities between the conditions in which the target and reference TSOM images are taken or simulated, especially the lateral instability during through-focus scanning. As a remedy to the lateral instability, we proposed the application of adaptive optics to the through-focus scanning operation and initially developed a closed-loop system with a tip/tilt mirror and a Shack-Hartmann sensor, with which we were able to keep the plane position within peak-to-valley (PV) 33 nm. We then further developed a motion-free TSOM tool reducing the instability down to practically zero by the replacement of the tip/tilt mirror with a deformable mirror that performs through-focus scanning by deforming its mirror surface. The motion-free TSOM tool with a × 50 (NA 0.55) objective lens could provide a scanning range of up to ± 25 µm with a minimum step of 25 nm at a maximum update rate of 4 kHz. The tool was demonstrated to have a recognition accuracy of < 4 nm for critical dimension (CD) values in the range of 60 ∼ 120 nm with a reference TSOM image library generated by a Fourier modal method matching various observations conditions.

Detecting nanoscale contamination in semiconductor fabrication using through-focus scanning optical microscopy

This paper reports high-throughput, light-based, through-focus scanning optical microscopy (TSOM) for detecting industrially relevant sub-50 nm tall nanoscale contaminants. Measurement parameter optimization to maximize the TSOM signal using optical simulations made it possible to detect the nanoscale contaminants. Atomic force and scanning electron microscopies were used as reference methods for comparison.

Five-dimensional two-photon volumetric microscopy of in-vivo dynamic activities using liquid lens remote focusing

Multi-photon scanning microscopy provides a robust tool for optical sectioning, which can be used to capture fast biological events such as blood flow, mitochondrial activity, and neuronal action potentials. For many studies, it is important to visualize several different focal planes at a rate akin to the biological event frequency. Typically, a microscope is equipped with mechanical elements to move either the sample or the objective lens to capture volumetric information, but these strategies are limited due to their slow speeds or inertial artifacts. To overcome this problem, remote focusing methods have been developed to shift the focal plane axially without physical movement of the sample or the microscope. Among these methods is liquid lens technology, which adjusts the focus of the lens by changing the wettability of the liquid and hence its curvature. Liquid lenses are inexpensive active optical elements that have the potential for fast multi-photon volumetric imaging, hence a promising and accessible approach for the study of biological systems with complex dynamics. Although remote focusing using liquid lens technology can be used for volumetric point scanning multi-photon microscopy, optical aberrations and the effects of high energy laser pulses have been concerns in its implementation. In this paper, we characterize a liquid lens and validate its use in relevant biological applications. We measured optical aberrations that are caused by the liquid lens, and calculated its response time, defocus hysteresis, and thermal response to a pulsed laser. We applied this method of remote focusing for imaging and measurement of multiple in-vivo specimens, including mesenchymal stem cell dynamics, mouse tibialis anterior muscle mitochondrial electrical potential fluctuations, and mouse brain neural activity. Our system produces 5 dimensional (x,y,z,λ,t) data sets at the speed of 4.2 volumes per second over volumes as large as 160 x 160 x 35 µm³.

Multiple parametric nanoscale measurements with high sensitivity based on through-focus scanning optical microscopy

High-throughput through-focus scanning optical microscopy (TSOM) involves defocusing along the optical axis and capturing a series of defocus images and is useful in optical nanoscale measurement. However, TSOM is usually affected by its optical and mechanical noises. In this study, the issue of sensitivity and application in three-dimensional (3D) multiple parameter measurement of TSOM is investigated. First, a TSOM system with objective scanning and its relative simulation algorithm are proposed. Second, based upon the system and algorithm, an experiment on an isolated Au line is performed and the corresponding matching library is established. Comparing the experimental TSOM image and simulated TSOM images of the library, 3D multiple parameter results of the Au line are extracted. Third, the precision of the system is analysed through a fidelity test particular for through-focus images. According to this study, the system is robust to the optical and mechanical noises and hence could be useful in 3D multiple parametric measurement and high-volume nanomanufacturing.

Nondestructive shape process monitoring of three-dimensional high aspect ratio targets using through-focus scanning optical microscopy

Low-cost, high-throughput and nondestructive metrology of truly three-dimensional (3-D) targets for process control/monitoring is a critically needed enabling technology for high-volume manufacturing (HVM) of nano/micro technologies in multi-disciplinary areas. In particular, a survey of the typically used metrology tools indicates the lack of a tool that truly satisfies the HVM metrology needs of 3-D targets, such as high aspect ratio (HAR) targets. Using HAR targets here we demonstrate that through-focus scanning optical microscopy (TSOM) is a strong contender to fill the gap for 3-D shape metrology. Differential TSOM (D-TSOM) images are extremely sensitive to small and/or dissimilar types of 3-D shape variations. Based on this here we propose a TSOM method that involves creating a database of cross-sectional profiles of the HAR targets along with their respective D-TSOM signals. Using the database, we present a simple-to-use, low-cost, high-throughput and nondestructive process-monitoring method suitable for HVM of truly 3-D targets, which also does not require optical simulations, making its use straightforward and automatable. Even though HAR targets are used for this demonstration, the similar process can be applied to any truly 3-D targets with dimensions ranging from micro-scale to nano-scale. The TSOM method couples the advantage of analyzing truly isolated targets with the ability to simultaneously analyze many targets present in the large field-of-view of a conventional optical microscope.

Fidelity test for through-focus or volumetric type of optical imaging methods

Rapid increase in interest and applications of through-focus (TF) or volumetric type of optical imaging in biology and other areas has resulted in the development of several TF image collection methods. Achieving quantitative results from images requires standardization and optimization of image acquisition protocols. Several standardization protocols are available for conventional optical microscopy where a best-focus image is used, but to date, rigorous testing protocols do not exist for TF optical imaging. In this paper, we present a method to determine the fidelity of the TF optical data using the TF scanning optical microscopy images.