Generalized spot auto-focusing method with a high-definition auto-correlation function in transmission electron microscopy

The spot auto-focusing (AF) method with a unique high-definition auto-correlation function (HD-ACF) proposed in the previous paper is improved and is now applicable to general specimens at a wide range of magnifications. According to the definition where the AF is defocused to obtain the highest resolution, the proposed method achieves the sharpest HD-ACF profile in the AF spot image. The relationship where the sharpest HD-ACF profile gives the highest resolution is theoretically explained, and practical AF examples for different specimens and magnifications are experimentally demonstrated. Specimens include a yeast cell thin section at 10-k magnification, a standard grating replica used as a ruler at 50-k, a crystal lattice of graphitized carbon at 400-k and a 60°-tilted thin section (yeast cell) at 10-k. Different procedures are prepared to actively identify the defocus position that gives the sharpest HD-ACF profile. Every AF result demonstrates the highest-resolution image.

Spot auto-focusing and spot auto-stigmation methods with high-definition auto-correlation function in high-resolution TEM

As alternatives to the diffractogram-based method in high-resolution transmission electron microscopy, a spot auto-focusing (AF) method and a spot auto-stigmation (AS) method are presented with a unique high-definition auto-correlation function (HD-ACF). The HD-ACF clearly resolves the ACF central peak region in small amorphous-thin-film images, reflecting the phase contrast transfer function. At a 300-k magnification for a 120-kV transmission electron microscope, the smallest areas used are 64 × 64 pixels (~3 nm2) for the AF and 256 × 256 pixels for the AS. A useful advantage of these methods is that the AF function has an allowable accuracy even for a low s/n (~1.0) image. A reference database on the defocus dependency of the HD-ACF by the pre-acquisition of through-focus amorphous-thin-film images must be prepared to use these methods. This can be very beneficial because the specimens are not limited to approximations of weak phase objects but can be extended to objects outside such approximations.

Improvement in focusing accuracy of DNA sequencing microscope with multi-position laser differential confocal autofocus method

High focusing accuracy in microscopes could improve the imaging quality to reduce the error rate in DNA sequencing. We propose a new feedback method to improve the focusing condition to a very high accuracy. A reference laser reflected by the sample is detected by two or more sensors around the confocal point. After acquiring the signals from the out-of-focus positions, online data processing is implemented to provide feedbacks for real-time focus-plane locking on the sample surface. This method provides an accuracy better than 1/10 of the objective depth-of-focus. To balance optical aberrations, a specific optical feedback system should be designed, with athermal design considerations to adapt DNA sequencing work to temperature fluctuations.

Automatic system for electron tomography data collection in the ultra-high voltage electron microscope

In this study, we report an automatic system for collection of tilt series for electron tomography based on the ultra-HVEM in Osaka University. By remotely controlling the microscope and reading the observation image, the system can track the field of view and do focus in each tilt angle. The automatic tracking is carried out with an image matching technique based on normalized correlation coefficient. Auto focus is realized by the optimization of image sharpness. A toolkit that can expand the field of view with technique of image stitching is also developed. The system can automatically collect the tilt series with much smaller time consumption.

Autofocus on moving object in scanning electron microscope

The sharpness of the images coming from a Scanning Electron Microscope (SEM) is a very important property for many computer vision applications at micro- and nanoscale. It represents how much object details are distinctive in the images: the object may be perceived sharp or blurred. Image sharpness highly depends on the value of focal distance, or working distance in the case of the SEM. Autofocus is the technique allowing to automatically adjust the working distance to maximize the sharpness. Most of the existing algorithms allows working only with a static object which is enough for the tasks of visualization, manual microanalysis or microcharacterization. These applications work with a low frame rate, less than 1 Hz, that guarantees a low level of noise. However, static autofocus can not be used for samples performing continuous 3D motion, which is the case of robotic applications where it is required to carry out a continuous 3D position measurement, e.g., nano-assembly or nanomanipulation. Moreover, in addition to constantly keeping object in focus while it is moving, it is required to perform the operation at high frame rate. The approach offering both these possibilities is presented in this paper and is referred as dynamic autofocus. The presented solution is based on stochastic optimization techniques. It allows tracking the maximum of the sharpness of the images without sweep and without training. It works under uncertainty conditions: presence of noise in images, unknown maximal sharpness and unknown 3D motion of the specimen. The experiments, that were performed with noisy images at high frame rate (5 Hz), were conducted on a Carl Zeiss Auriga 60 FE-SEM. They prove the robustness of the algorithm with respect to the variation of optimization parameters, object speed and magnification. Moreover, it is invariant to the object structure and its variation in time.

Combining gradient ascent search and support vector machines for effective autofocus of a field emission-scanning electron microscope

Autofocus is an important issue in electron microscopy, particularly at high magnification. It consists in searching for sharp image of a specimen, that is corresponding to the peak of focus. The paper presents a machine learning solution to this issue. From seven focus measures, support vector machines fitting is used to compute the peak with an initial guess obtained from a gradient ascent search, that is search in the direction of higher gradient of focus. The solution is implemented on a Carl Zeiss Auriga FE-SEM with a three benchmark specimen and magnification ranging from x300 to x160 000. Based on regularized nonlinear least squares optimization, the solution overtakes the literature nonregularized search and Fibonacci search methods: accuracy improvement ranges from 1.25 to 8 times, fidelity improvement ranges from 1.6 to 28 times, and speed improvement ranges from 1.5 to 4 times. Moreover, the solution is practical by requiring only an off-line easy automatic train with cross-validation of the support vector machines.

Fast auto-acquisition tomography tilt series by using HD video camera in ultra-high voltage electron microscope

The ultra-high voltage electron microscope (UHVEM) H-3000 with the world highest acceleration voltage of 3 MV can observe remarkable three dimensional microstructures of microns-thick samples[1]. Acquiring a tilt series of electron tomography is laborious work and thus an automatic technique is highly desired. We proposed the Auto-Focus system using image Sharpness (AFS)[2,3] for UHVEM tomography tilt series acquisition. In the method, five images with different defocus values are firstly acquired and the image sharpness are calculated. The sharpness are then fitted to a quasi-Gaussian function to decide the best focus value[3]. Defocused images acquired by the slow scan CCD (SS-CCD) camera (Hitachi F486BK) are of high quality but one minute is taken for acquisition of five defocused images.In this study, we introduce a high-definition video camera (HD video camera; Hamamatsu Photonics K. K. C9721S) for fast acquisition of images[4]. It is an analog camera but the camera image is captured by a PC and the effective image resolution is 1280×1023 pixels. This resolution is lower than that of the SS-CCD camera of 4096×4096 pixels. However, the HD video camera captures one image for only 1/30 second. In exchange for the faster acquisition the S/N of images are low. To improve the S/N, 22 captured frames are integrated so that each image sharpness is enough to become lower fitting error. As countermeasure against low resolution, we selected a large defocus step, which is typically five times of the manual defocus step, to discriminate different defocused images.By using HD video camera for autofocus process, the time consumption for each autofocus procedure was reduced to about six seconds. It took one second for correction of an image position and the total correction time was seven seconds, which was shorter by one order than that using SS-CCD camera. When we used SS-CCD camera for final image capture, it took 30 seconds to record one tilt image. We can obtain a tilt series of 61 images within 30 minutes. Accuracy and repeatability were good enough to practical use (Figure 1). We successfully reduced the total acquisition time of a tomography tilt series in half than before.jmicro;63/suppl_1/i25/DFU066F1F1DFU066F1Fig. 1.Objective lens current change with a tilt angle during acquisition of tomography series (Sample: a rat hepatocyte, thickness: 2 m, magnification: 4k, acc. voltage: 2 MV). Tilt angle range is ±60 degree with 2 degree step angle. Two series were acquired in the same area. Both data were almost same and the deviation was smaller than the minimum step by manual, so auto-focus worked well. We also developed a computer-aided three dimensional (3D) visualization and analysis software for electron tomography "HawkC" which can sectionalize the 3D data semi-automatically[5,6]. If this auto-acquisition system is used with IMOD reconstruction software[7] and HawkC software, we will be able to do on-line UHVEM tomography. The system would help pathology examination in the future.This work was supported by the Ministry of Education, Culture, Sports, Science and Technology (MEXT), Japan, under a Grant-in-Aid for Scientific Research (Grant No. 23560024, 23560786), and SENTAN, Japan Science and Technology Agency, Japan.