Dynamic evaluation of autofocusing for automated microscopic analysis of blood smear and pap smear

An autofocusing method for dynamic surface-enhanced Raman spectroscopy detection realized by optimized hill-climbing algorithm with long time stable hotspots

In the manual dynamic surface-enhanced Raman spectroscopy (D-SERS) detection process, it is difficult to focus on sample drop due to the constantly changing hotspot and easy judgment method. In this paper, we proposed an automatic focusing method based on long time stable hotspot with aid of optimization of hill-climbing algorithm and achieved on a designed device. First, set up a high temperature accelerating evaporation process to obtain hotspot and then cool to a low temperature rapidly to maintain it. Then, the spectral intensity was used as a focus of feedback signal in optimized hill-climbing algorithm to drive the sample stage to move up and down to adjust the depth of the laser on the samples to realize automatic focusing. As a result, the hotspot can be maintained for 5 min, and the autofocusing result can be achieved within 9 s, while the sensitivity was improved with two orders of magnitude in D-SERS detection of crystal violet (CV) compared with manual focusing.

Medical image fusion using enhanced cross-visual cortex model based on artificial selection and impulse-coupled neural network

OBJECTIVE

The traditional ICM is widely used in applications, such as image edge detection and image segmentation. However, several model parameters must be set, which tend to lead to reduced accuracy and increased cost. As medical images have more complex edges, contours and details, more suitable combinatorial algorithms are needed to handle the pathological diagnosis of multiple cerebral infarcts and acute strokes, resulting in the findings being more applicable, as well as having good clinical value.

METHODS

To better solve the medical image fusion and diagnosis problems, this paper introduces the image fusion algorithm based on the combination of NSCT and improved ICM and proposes low-frequency, sub-band fusion rules and high-frequency sub-band fusion rules. The above method is applied to the fusion of CT/MRI images, subsequently, three other fusion algorithms, including NSCT-SF-PCNN, NSCT-SR-PCNN and Adaptive-PCNN are compared, and the simulation results of image fusion are analyzed and validated.

RESULTS

According to the experimental findings, the suggested algorithm performs better than other fusion algorithms in terms of five objective evaluation metrics or subjective evaluation. The NSCT transform and the improved ICM were combined, and the outcomes were evaluated against those of other fusion algorithms. The CT/MRI medical images of healthy brain tissue, numerous cerebral infarcts and acute strokes were combined using this technique.

CONCLUSION

Medical image fusion using Adaptive-PCNN produces satisfactory results, not only in relation to improved image clarity but also in terms of outstanding edge information, high contrast and brightness.

Recent technical advances in whole slide imaging instrumentation

Microscopic observation of biological specimen smears is the mainstay of diagnostic pathology, as defined by the Digital Pathology Association. Though automated systems for this are commercially available, their bulky size and high cost renders them unusable for remote areas. The research community is investing much effort towards building equivalent but portable, low-cost systems. An overview of such research is presented here, including a comparative analysis of recent reports. This paper also reviews recently reported systems for automated staining and smear formation, including microfluidic devices; and optical and computational automated microscopy systems including smartphone-based devices. Image pre-processing and analysis methods for automated diagnosis are also briefly discussed. It concludes with a set of foreseeable research directions that could lead to affordable, integrated and accurate whole slide imaging systems.

An Auto-Focus Method of Microscope for the Surface Structure of Transparent Materials under Transmission Illumination

This paper is concerned with auto-focus of microscopes for the surface structure of transparent materials under transmission illumination, where two distinct focus states appear in the focusing process and the focus position is located between the two states with the local minimum of sharpness. Please note that most existing results are derived for one focus state with the global maximum value of sharpness, they cannot provide a feasible solution to this particular problem. In this paper, an auto-focus method is developed for such a specific situation with two focus states. Firstly, a focus state recognition model, which is essentially an image classification model based on a deep convolution neural network, is established to identify the focus states of the microscopy system. Then, an endpoint search algorithm which is an evolutionary algorithm based on differential evolution is designed to obtain the positions of the two endpoints of the region where the real focus position is located, by updating the parameters according to the focus states. At last, a region search algorithm is devised to locate the focus position. The experimental results show that our method can achieve auto-focus rapidly and accurately for such a specific situation with two focus states.

Autofocusing technologies for whole slide imaging and automated microscopy

Whole slide imaging (WSI) has moved digital pathology closer to diagnostic practice in recent years. Due to the inherent tissue topography variability, accurate autofocusing remains a critical challenge for WSI and automated microscopy systems. The traditional focus map surveying method is limited in its ability to acquire a high degree of focus points while still maintaining high throughput. Real-time approaches decouple image acquisition from focusing, thus allowing for rapid scanning while maintaining continuous accurate focus. This work reviews the traditional focus map approach and discusses the choice of focus measure for focal plane determination. It also discusses various real-time autofocusing approaches including reflective-based triangulation, confocal pinhole detection, low-coherence interferometry, tilted sensor approach, independent dual sensor scanning, beam splitter array, phase detection, dual-LED illumination and deep-learning approaches. The technical concepts, merits and limitations of these methods are explained and compared to those of a traditional WSI system. This review may provide new insights for the development of high-throughput automated microscopy imaging systems that can be made broadly available and utilizable without loss of capacity.

Autofocus of whole slide imaging based on convolution and recurrent neural networks

During the process of whole slide imaging, it is necessary to focus thousands of fields of view to obtain a high-quality image. To make the focusing procedure efficient and effective, we propose a novel autofocus algorithm for whole slide imaging. It is based on convolution and recurrent neural networks to predict the out-of-focus distance and subsequently update the focus location of the camera lens in an iterative manner. More specifically, we train a convolution neural network to extract focus information in the form of a focus feature vector. In order to make the prediction more accurate, we apply a recurrent neural network to combine focus information from previous search iteration and current search iteration to form a feature aggregation vector. This vector contains more focus information than the previous one and is subsequently used to predict the out-of-focus distance. Our experiments indicate that our proposed autofocus algorithm is able to rapidly determine the optimal in-focus image. The code is available at https://github.com/hezhujun/autofocus-rnn.

Study of an image autofocus method based on power threshold function wavelet reconstruction and a quality evaluation algorithm

As a key component in optical microscopy imaging systems, autofocus technology has a significant effect on imaging quality. In this paper, an optical microscopy autofocus method that includes a wavelet denoising algorithm based on a power threshold function and a Brenner image quality evaluation algorithm is presented. Experimental results show that the power threshold function wavelet denoising algorithm, which can be adopted to obtain more realistic optical images, is superior to the traditional soft, hard, hyperbolic, and exponential threshold functions in terms of peak signal-to-noise ratio, signal-to-noise ratio, mean squared error, and histogram indicators; moreover, compared to the Roberts, sum modulus difference (SMD), and energy gradient functions, the Brenner image quality evaluation algorithm can be used to quickly and accurately lock onto the focal plane. By integrating and applying these two core algorithms in the autofocus image acquisition system of a microscope, the image sharpness and focusing quality are greatly improved, which benefits the further evaluation of images.

Region-based multifocus image fusion for the precise acquisition of Pap smear images

A multifocus image fusion method to obtain a single focused image from a sequence of microscopic high-magnification Papanicolau source (Pap smear) images is presented. These images, captured each in a different position of the microscope lens, frequently show partially focused cells or parts of cells, which makes them unpractical for the direct application of image analysis techniques. The proposed method obtains a focused image with a high preservation of original pixels information while achieving a negligible visibility of the fusion artifacts. The method starts by identifying the best-focused image of the sequence; then, it performs a mean-shift segmentation over this image; the focus level of the segmented regions is evaluated in all the images of the sequence, and best-focused regions are merged in a single combined image; finally, this image is processed with an adaptive artifact removal process. The combination of a region-oriented approach, instead of block-based approaches, and a minimum modification of the value of focused pixels in the original images achieve a highly contrasted image with no visible artifacts, which makes this method especially convenient for the medical imaging domain. The proposed method is compared with several state-of-the-art alternatives over a representative dataset. The experimental results show that our proposal obtains the best and more stable quality indicators.

FocusALL: Focal Stacking of Microscopic Images Using Modified Harris Corner Response Measure

Automated image analysis of microscopic images such as protein crystallization images and cellular images is one of the important research areas. If objects in a scene appear at different depths with respect to the camera's focal point, objects outside the depth of field usually appear blurred. Therefore, scientists capture a collection of images with different depths of field. Focal stacking is a technique of creating a single focused image from a stack of images collected with different depths of field. In this paper, we introduce a novel focal stacking technique, FocusALL, which is based on our modified Harris Corner Response Measure. We also propose enhanced FocusALL for application on images collected under high resolution and varying illumination. FocusALL resolves problems related to the assumption that focus regions have high contrast and high intensity. Especially, FocusALL generates sharper boundaries around protein crystal regions and good in focus images for high resolution images in reasonable time. FocusALL outperforms other methods on protein crystallization images and performs comparably well on other datasets such as retinal epithelial images and simulated datasets.

Compact multi-band fluorescent microscope with an electrically tunable lens for autofocusing

Autofocusing is a routine technique in redressing focus drift that occurs in time-lapse microscopic image acquisition. To date, most automatic microscopes are designed on the distance detection scheme to fulfill the autofocusing operation, which may suffer from the low contrast of the reflected signal due to the refractive index mismatch at the water/glass interface. To achieve high autofocusing speed with minimal motion artifacts, we developed a compact multi-band fluorescent microscope with an electrically tunable lens (ETL) device for autofocusing. A modified searching algorithm based on equidistant scanning and curve fitting is proposed, which no longer requires a single-peak focus curve and then efficiently restrains the impact of external disturbance. This technique enables us to achieve an autofocusing time of down to 170 ms and the reproductivity of over 97%. The imaging head of the microscope has dimensions of 12 cm × 12 cm × 6 cm. This portable instrument can easily fit inside standard incubators for real-time imaging of living specimens.

An automated imaging system for radiation biodosimetry

We describe here an automated imaging system developed at the Center for High Throughput Minimally Invasive Radiation Biodosimetry. The imaging system is built around a fast, sensitive sCMOS camera and rapid switchable LED light source. It features complete automation of all the steps of the imaging process and contains built-in feedback loops to ensure proper operation. The imaging system is intended as a back end to the RABiT-a robotic platform for radiation biodosimetry. It is intended to automate image acquisition and analysis for four biodosimetry assays for which we have developed automated protocols: The Cytokinesis Blocked Micronucleus assay, the γ-H2AX assay, the Dicentric assay (using PNA or FISH probes) and the RABiT-BAND assay.

Influence of the phase effect on gradient-based and statistics-based focus measures in bright field microscopy

Autofocusing is essential to high throughput microscopy and live cell imaging and requires reliable focus measures. Phase objects such as separated single Chinese hamster ovary cells are almost invisible at the optical focus position in bright field microscopy images. Because of the phase effect, defocused images of phase objects have more contrast. In this paper, we show that widely used focus measures exhibit an untypical behaviour for such images. In the case of homogeneous cells, that is, when most cells tend to lie in the same focal plane, both gradient-based and statistics-based focus measures tend to have a local minimum instead of a global maximum at the optical focus position. On the other hand, if images show inhomogeneous cells, gradient-based focus measures tend to yield typical focus curves, whereas statistics-based focus measures deliver curves similar to the case of homogeneous cells. These results were interpreted using the equation describing the phase effect and patch-wise analysis of the focus curves. Bioprocess engineering experts are also influenced by the phase effect. Forty-four focus positions selected by them led to the conclusion that they prefer to look at defocused images instead of those at the optical focus.

A novel autofocusing method using the angle of Hilbert space for microscopy

Autofocusing technology is indispensable for routine use of microscopes on a large scale in biological field. The autofocusing method using the angle of Hilbert space is brought forward to measure whether the image is focused or not. The angle of Hillbert space can be used to evaluate accurately the similarity degree of two images. The experiment results show that the autofocusing method can decrease the computational cost and get accuracy for real-time biological and biomedical images with noise robustness. The focus curves are smooth and possess the unimodality, the monotonicity and the symmetry. Compared with other classic and optimum focus method, the Hilbert method demonstrates its robustness to noise and can improve the focus speed. The experiments showed that the proposed method can increase the overall performance of an autofocus system and has strong applicability in various autofocusing algorithms.

Automation-assisted cervical cancer screening in manual liquid-based cytology with hematoxylin and eosin staining

Current automation-assisted technologies for screening cervical cancer mainly rely on automated liquid-based cytology slides with proprietary stain. This is not a cost-efficient approach to be utilized in developing countries. In this article, we propose the first automation-assisted system to screen cervical cancer in manual liquid-based cytology (MLBC) slides with hematoxylin and eosin (H&E) stain, which is inexpensive and more applicable in developing countries. This system consists of three main modules: image acquisition, cell segmentation, and cell classification. First, an autofocusing scheme is proposed to find the global maximum of the focus curve by iteratively comparing image qualities of specific locations. On the autofocused images, the multiway graph cut (GC) is performed globally on the a* channel enhanced image to obtain cytoplasm segmentation. The nuclei, especially abnormal nuclei, are robustly segmented by using GC adaptively and locally. Two concave-based approaches are integrated to split the touching nuclei. To classify the segmented cells, features are selected and preprocessed to improve the sensitivity, and contextual and cytoplasm information are introduced to improve the specificity. Experiments on 26 consecutive image stacks demonstrated that the dynamic autofocusing accuracy was 2.06 μm. On 21 cervical cell images with nonideal imaging condition and pathology, our segmentation method achieved a 93% accuracy for cytoplasm, and a 87.3% F-measure for nuclei, both outperformed state of the art works in terms of accuracy. Additional clinical trials showed that both the sensitivity (88.1%) and the specificity (100%) of our system are satisfyingly high. These results proved the feasibility of automation-assisted cervical cancer screening in MLBC slides with H&E stain, which is highly desirable in community health centers and small hospitals.

An interacting multiple model filter-based autofocus strategy for confocal time-lapse microscopy

Gene expression and other cellular processes are stochastic, thus their study requires observing multiple events in multiple cells. Therefore, confocal microscopy cell imaging has recently gained much interest. In time-lapse imaging, adjustments are needed at short intervals to compensate for focus drift. There are several automated methods for this purpose. In general, before acquiring higher resolution images, software-based autofocus algorithms require a set of low-resolution images along the z-axis to determine the plane for which a predefined focusing function is maximized. These algorithms require 10-100 z-slices each time, and there is no fixed number or upper limit of required z-slices that ensures optimal focusing. The higher is this number, the stronger is photo bleaching, hampering the feasibility of long-time series measurements. We propose a new focusing strategy in time-lapse imaging. The algorithm relies on the nature and predictability of the focus drift. We first show that the focus drift curve is predictable within a small error bound in standard experimental setups. We, then, exploit the interacting multiple model filter algorithm to predict the drift at time, t, based on the measurement at time t-1. This allows a drastic reduction of the number of required z-slices for focus drift correction, largely overcoming the problem of photo bleaching. In addition, we propose a new set of functions for focusing in time-lapse imaging, derived from preexisting ones. We demonstrate the method's efficiency in time-lapse imaging of Escherichia coli cells expressing MS2d-GFP tagged RNA molecules.

Automated focusing in bright-field microscopy for tuberculosis detection

Automated microscopy to detect Mycobacterium tuberculosis in sputum smear slides would enable laboratories in countries with a high tuberculosis burden to cope efficiently with large numbers of smears. Focusing is a core component of automated microscopy, and successful autofocusing depends on selection of an appropriate focus algorithm for a specific task. We examined autofocusing algorithms for bright-field microscopy of Ziehl-Neelsen stained sputum smears. Six focus measures, defined in the spatial domain, were examined with respect to accuracy, execution time, range, full width at half maximum of the peak and the presence of local maxima. Curve fitting around an estimate of the focal plane was found to produce good results and is therefore an acceptable strategy to reduce the number of images captured for focusing and the processing time. Vollath's F₄ measure performed best for full z-stacks, with a mean difference of 0.27 μm between manually and automatically determined focal positions, whereas it is jointly ranked best with the Brenner gradient for curve fitting.

Automated training data generation for microscopy focus classification

Image focus quality is of utmost importance in digital microscopes because the pathologist cannot accurately characterize the tissue state without focused images. We propose to train a classifier to measure the focus quality of microscopy scans based on an extensive set of image features. However, classifiers rely heavily on the quality and quantity of the training data, and collecting annotated data is tedious and expensive. We therefore propose a new method to automatically generate large amounts of training data using image stacks. Our experiments demonstrate that a classifier trained with the image stacks performs comparably with one trained with manually annotated data. The classifier is able to accurately detect out-of-focus regions, provide focus quality feedback to the user, and identify potential problems of the microscopy design.

Multispot live-image autofocusing for high-throughput microscopy of fluorescently stained bacteria

Screening by automated high-throughput microscopy has become a valuable research tool. An essential component of such systems is the autonomous acquisition of focused images. Here we describe the implementation of a high-precision autofocus routine for imaging of fluorescently stained bacteria on a commercially available microscope. We integrated various concepts and strategies that together substantially enhance the performance of autonomous image acquisition. These are (i) nested focusing in bright-field and fluorescence illumination, (ii) autofocusing by continuous life-image acquisition during movement in z-direction rather than at distinct z-positions, (iii) assessment of the quality and topology of a field of view (FOV) by multispot focus measurements, and (iv) acquisition of z-stacks and application of an extended depth of field algorithm to compensate for FOV unevenness. The freely provided program and documented source code allow ready adaptation of the here presented approach to various platforms and scientific questions.

Halton sampling for autofocus

Reliable autofocusing is a critical part of any automated microscopy system: by precisely positioning the sample in the focal plane, the acquired images are sharp and can be accurately segmented and quantified. The three main components of an autofocus algorithm are a contrast function, an optimization algorithm and a sampling strategy. The latter has not been given much attention in the literature. It is however a very important part of the autofocusing algorithm, especially in high content and high throughput image-based screening. It deals with the problem of sampling the focus surface as sparsely as possible to reduce bleaching and computation time while with sufficient detail as to permit a faithful interpolation. We propose a new strategy that has higher performance compared to the classical square grid or the hexagonal lattice, which is based on the concept of low discrepancy point sets and in particular on the Halton point set. We tested the new algorithm on nine different focus surfaces, each under 24 different combinations of Signal-to-Noise ratio (SNR) and sampling rate, obtaining that in 88% of the tested conditions, Halton sampling outperforms its counterparts.