A pixel-based autofocusing technique for digital histologic and cytologic slides

Machine learning-based gene alteration prediction model for primary lung cancer using cytologic images

BACKGROUND

Understanding the gene alteration status of primary lung cancers is important for determining treatment strategies, but gene testing is both time-consuming and costly, limiting its application in clinical practice. Here, potential therapeutic targets were selected by predicting gene alterations in cytologic specimens before conventional gene testing.

METHODS

This was a retrospective study to develop a cytologic image-based gene alteration prediction model for primary lung cancer. Photomicroscopic images of cytology samples were collected and image patches were generated for analyses. Cancer-positive (n = 106) and cancer-negative (n = 32) samples were used to develop a neural network model for selecting cancer-positive images. Cancer-positive cases were randomly assigned to training (n = 77) and validation (n = 26) data sets. Another neural network model was developed to classify cancer images of the training data set into 4 groups: anaplastic lymphoma kinase (ALK)-fusion, epidermal growth factor receptor (EGFR), or Kirsten rat sarcoma viral oncogene homologue (KRAS) mutated groups, and other (None group), and images of the validation data set were classified. A decision algorithm to predict gene alteration for cases with 3 probability ranks was developed.

RESULTS

The accuracy and precision for selecting cancer-positive patches were 0.945 and 0.991, respectively. Predictive accuracy for the EGFR and KRAS groups in the validation data set was ~0.95, whereas that for the ALK and None groups was ~0.75 and ~ 0.80, respectively. Gene status was correctly predicted in the probability rank A cases. The model extracted characteristic conventional cytologic findings in images and a novel specific feature was discovered for the EGFR group.

CONCLUSIONS

A gene alteration prediction model for lung cancers by machine learning based on cytologic images was successfully developed.

Accelerating wavefront-sensing-based autofocusing using pixel reduction in spatial and frequency domains

The wavefront-sensing-based autofocus method can precisely determine the focal plane only with few captured images; however, the required phase retrieval, numerical wavefront propagation, and in-focus determination are often time consuming, inevitably limiting its high-speed applications. To accelerate its processing speed, the pixel-reduced wavefront-sensing-based autofocus (PRWSA) method is proposed: with field of interest selection as pixel reduction in the spatial domain and image compression as pixel reduction in the frequency domain, the wavefront with fewer pixels can be used for autofocusing, significantly decreasing the processing time. With simulations, pixel reduction criteria in both the spatial and frequency domains are first determined and tested; next certificated by experiments, the PRWSA method is proved to be well implemented for different specimens. Considering it can precisely locate the focal plane with simple setup, and accelerate the processing speed, this PRWSA method can be a potential tool for high-speed autofocusing.

An automated system for whole microscopic image acquisition and analysis

The field of anatomic pathology has experienced major changes over the last decade. Virtual microscopy (VM) systems have allowed experts in pathology and other biomedical areas to work in a safer and more collaborative way. VMs are automated systems capable of digitizing microscopic samples that were traditionally examined one by one. The possibility of having digital copies reduces the risk of damaging original samples, and also makes it easier to distribute copies among other pathologists. This article describes the development of an automated high-resolution whole slide imaging (WSI) system tailored to the needs and problems encountered in digital imaging for pathology, from hardware control to the full digitization of samples. The system has been built with an additional digital monochromatic camera together with the color camera by default and LED transmitted illumination (RGB). Monochrome cameras are the preferred method of acquisition for fluorescence microscopy. The system is able to digitize correctly and form large high resolution microscope images for both brightfield and fluorescence. The quality of the digital images has been quantified using three metrics based on sharpness, contrast and focus. It has been proved on 150 tissue samples of brain autopsies, prostate biopsies and lung cytologies, at five magnifications: 2.5×, 10×, 20×, 40×, and 63×. The article is focused on the hardware set-up and the acquisition software, although results of the implemented image processing techniques included in the software and applied to the different tissue samples are also presented.

Image division technique in pre-acquisition analysis of information content for automated microscopy

This article presents a method that allows for reliable automated image acquisition of specimens with high information content in light microscopy with emphasis on fluorescence microscopy applications. Automated microscopy typically relies on autofocusing used for the analysis of information content behaviour along the z-axis within each field of view. However, in the case of a field of view containing more objects that do not lie precisely in one z-plane, traditional autofocusing methods fail due to their principle of operation. We avoid this issue by reducing the original problem to a set of simple and performable tasks: we divide the field of view into a small number of tiles and process each of them individually. The obtained results enable discovering z-planes with rich information content that remain hidden during global analysis of the whole field of view. Our approach therefore outperforms other acquisition methods including the manual one. A large part of the contribution is oriented towards practical application.

Information content analysis in automated microscopy imaging using an adaptive autofocus algorithm for multimodal functions

We present a new algorithm to analyse information content in images acquired using automated fluorescence microscopy. The algorithm belongs to the group of autofocusing methods, but differs from its predecessors in that it can handle thick specimens and operate also in confocal mode. It measures the information content in images using a 'content function', which is essentially the same concept as a focus function. Unlike previously presented algorithms, this algorithm tries to find all significant axial positions in cases where the content function applied to real data is not unimodal, which is often the case. This requirement precludes using algorithms that rely on unimodality. Moreover, choosing a content function requires careful consideration, because some functions suppress local maxima. First, we test 19 content functions and evaluate their ability to show local maxima clearly. The results show that only six content functions succeed. To save time, the acquisition procedure needs to vary the step size adaptively, because a wide range of possible axial positions has to be passed so as not to miss a local maximum. The algorithm therefore has to assess the steepness of the content function online so that it can decide to use a bigger or smaller step size to acquire the next image. Therefore, the algorithm needs to know about typical behaviour of content functions. We show that for normalized variance, one of the most promising content functions, this knowledge can be obtained after normalizing with respect to the theoretical maximum of this function, and using hierarchical clustering. The resulting algorithm is more reliable and efficient than a simple procedure with constant steps.

Automated training data generation for microscopy focus classification

Image focus quality is of utmost importance in digital microscopes because the pathologist cannot accurately characterize the tissue state without focused images. We propose to train a classifier to measure the focus quality of microscopy scans based on an extensive set of image features. However, classifiers rely heavily on the quality and quantity of the training data, and collecting annotated data is tedious and expensive. We therefore propose a new method to automatically generate large amounts of training data using image stacks. Our experiments demonstrate that a classifier trained with the image stacks performs comparably with one trained with manually annotated data. The classifier is able to accurately detect out-of-focus regions, provide focus quality feedback to the user, and identify potential problems of the microscopy design.

A feasibility study of virtual slides in surgical pathology in China

China's huge territorial expanse and its imbalance of regional economic development have resulted in an uneven distribution of experienced pathologists. Developing telepathology for consultation is of special relevance to China. We developed a newly designed telepathology workstation, which includes a small file size of each slide, permitting easy transmission, storage, and manipulation, and a feedback function, and also evaluated its feasibility in surgical pathology in China. Four hundred cases covering a broad spectrum of surgical pathology problems were investigated in a blinded fashion by the 2 pathologists using this virtual microscope system. These cases were then randomized and re-reviewed a second time with light microscope. Diagnoses and time spent for each diagnosis were recorded for both methods. The diagnostic accuracies achieved by viewing glass slides and virtual images were 97.25% (389 of 400) and 95.5% (382 of 400) for pathologist A and 96.25% (385 of 400) and 94.75% (379 of 400) for pathologist B, respectively. There was no significant diagnostic discrepancy between the 2 methods for the 2 pathologists. The average times for viewing a virtual slide were 3.41 and 5.24 minutes for pathologists A and B, respectively, whereas the average times for viewing a glass slide were 1.16 and 3.35 minutes for pathologists A and B. There was a statistical difference between the time costs of the 2 methods. However, the slight time increase using virtual slides is less than that using dynamic telepathology and traditional consultation, and is acceptable to the pathologists. These results showed that this newly designed virtual microscope system have an acceptable diagnostic accuracy that is of practical value and may be suitable for application in China.

Eye-movement study and human performance using telepathology virtual slides. Implications for medical education and differences with experience

A core skill in diagnostic pathology is light microscopy. Remarkably little is known about human factors that affect the proficiency of pathologists as light microscopists. The cognitive skills of pathologists have received relatively little attention in comparison with the large literature on human performance studies in radiology. One reason for this lack of formal visual search studies in pathology has been the physical restrictions imposed by the close positioning of a microscope operator's head to the microscope's eyepieces. This blocks access to the operator's eyes and precludes assessment of the microscopist's eye movements. Virtual slide microscopy now removes this barrier and opens the door for studies on human factors and visual search strategies in light microscopy. The aim of this study was to assess eye movements of medical students, pathology residents, and practicing pathologists examining virtual slides on a digital display monitor. Whole histopathology glass slide digital images, so-called virtual slides, of 20 consecutive breast core biopsy cases were used in a retrospective study. These high-quality virtual slides were produced with an array-microscope equipped DMetrix DX-40 ultrarapid virtual slide processor (DMetrix, Tucson, Ariz). Using an eye-tracking device, we demonstrated for the first time that when a virtual slide reader initially looks at a virtual slide his or her eyes are very quickly attracted to regions of interest (ROIs) within the slide and that these ROIs are likely to contain diagnostic information. In a matter of seconds, critical decisions are made on the selection of ROIs for further examination at higher magnification. We recorded: (1) the time virtual slide readers spent fixating on self-selected locations on the video monitor; (2) the characteristics of the ways the eyes jumped between fixation locations; and (3) x and y coordinates for each virtual slide marking the sites the virtual slide readers manually selected for zooming to higher ROI magnifications. We correlated the locations of the visually selected fixation locations and the manually selected ROIs. Viewing profiles were identified for each group. Fully trained pathologists spent significantly less time (mean, 4.471 seconds) scanning virtual slides when compared to pathology residents (mean, 7.148 seconds) or medical students (mean, 11.861 seconds), but had relatively prolonged saccadic eye movements (P < .0001). Saccadic eye movements are defined as eye movements between fixation locations. On the other hand, the pathologists spent significantly more time than trainees dwelling on the 3 locations they subsequently chose for zooming. Unlike either the medical students or the residents, the pathologists frequently choose areas for viewing at higher magnification outside of areas of foveal (central) vision. Eye movement studies of scanning pathways (scan paths) may be useful for developing eye movement profiles for individuals and for understanding the difference in performances between novices and experts. They may also be useful for developing new visual search strategies for rendering diagnoses on telepathology virtual slides.

The development and validation of the Virtual Tissue Matrix, a software application that facilitates the review of tissue microarrays on line

Background

The Tissue Microarray (TMA) facilitates high-throughput analysis of hundreds of tissue specimens simultaneously. However, bottlenecks in the storage and manipulation of the data generated from TMA reviews have become apparent. A number of software applications have been developed to assist in image and data management; however no solution currently facilitates the easy online review, scoring and subsequent storage of images and data associated with TMA experimentation.

Results

This paper describes the design, development and validation of the Virtual Tissue Matrix (VTM). Through an intuitive HTML driven user interface, the VTM provides digital/virtual slide based images of each TMA core and a means to record observations on each TMA spot. Data generated from a TMA review is stored in an associated relational database, which facilitates the use of flexible scoring forms. The system allows multiple users to record their interpretation of each TMA spot for any parameters assessed. Images generated for the VTM were captured using a standard background lighting intensity and corrective algorithms were applied to each image to eliminate any background lighting hue inconsistencies or vignetting.

Validation of the VTM involved examination of inter-and intra-observer variability between microscope and digital TMA reviews. Six bladder TMAs were immunohistochemically stained for E-Cadherin, β-Catenin and PhosphoMet and were assessed by two reviewers for the amount of core and tumour present, the amount and intensity of membrane, cytoplasmic and nuclear staining.

Conclusion

Results show that digital VTM images are representative of the original tissue viewed with a microscope. There were equivalent levels of inter-and intra-observer agreement for five out of the eight parameters assessed. Results also suggest that digital reviews may correct potential problems experienced when reviewing TMAs using a microscope, for example, removal of background lighting variance and tint, and potential disorientation of the reviewer, which may have resulted in the discrepancies evident in the remaining three parameters.

User attitudes in analyzing digital slides in a quality control test bed: a preliminary study

The pathologist examines suitably stained glass slides through a bright field microscope in order to render histopathological or cytological diagnosis by looking at tissues and cells. Glass slides serve as a permanent record of the patient disease. Over the course of a patient's treatment slides may need to be reviewed at other institutions before treatment can commence. Due to their fragile nature a transportable permanent digital facsimile of the glass slide would be ideal. A digital slide is a set of digital images representing the whole slide normally used by the pathologist, or a significant part of it; it is usually made by a large amount of images, up to thousands, which makes its management difficult. The present paper provides a description of the requirements needed to reproduce glass slides and of the available technological equipment, then the features of the two systems we implemented on different hardware are described, together with those of the digital slide viewer. The viewer was evaluated in two experimental test phases, during which user behaviour and diagnostic reports were measured. Digital slides used in the two experiments were acquired with either system. Possible applications of digital slides are then discussed, including undergraduate and professional education, quality control, and image analysis on full samples as well as on tissue microarrays.