Distributed computing in image analysis using open source frameworks and application to image sharpness assessment of histological whole slide images

Automated staining analysis in digital cytopathology and applications

The progress of digital pathology in recent years has been an opportunity for the development of automated image analysis algorithms for quantitative measurements and computer aided diagnosis. With those new methods comes the need for high staining quality and reproducibility, as image analysis tools are typically more sensible to slight stain variations than trained pathologists. This article presents a method for the automated analysis of cytology slides stains specifically adapted to the challenges encountered in digital cytopathology. In particular, the variety of cell types in cytology slides, the 3D distribution of the cellular material, the presence of superposed cells and the need for independent analysis of sub-cellular compartments are addressed. The proposed method is applied to the quantification of staining variations for quality control, resulting from changes in the staining protocol such as reagent immersion time or a reagent change. Another demonstrated application is the selection of staining protocol parameters that maximize the visible details in nucleus. Finally the analysis pipeline is also used to compare different stain normalization algorithms on digital cytology slides. Code available at: https://gitlab.com/vitadx/articles/automated_staining_analysis.

Investigation of cell mechanics using single-beam acoustic tweezers as a versatile tool for the diagnosis and treatment of highly invasive breast cancer cell lines: an in vitro study

Advancements in diagnostic systems for metastatic cancer over the last few decades have played a significant role in providing patients with effective treatment by evaluating the characteristics of cancer cells. Despite the progress made in cancer prognosis, we still rely on the visual analysis of tissues or cells from histopathologists, where the subjectivity of traditional manual interpretation persists. This paper presents the development of a dual diagnosis and treatment tool using an in vitro acoustic tweezers platform with a 50 MHz ultrasonic transducer for label-free trapping and bursting of human breast cancer cells. For cancer cell detection and classification, the mechanical properties of a single cancer cell were quantified by single-beam acoustic tweezers (SBAT), a noncontact assessment tool using a focused acoustic beam. Cell-mimicking phantoms and agarose hydrogel spheres (AHSs) served to standardize the biomechanical characteristics of the cells. Based on the analytical comparison of deformability levels between the cells and the AHSs, the mechanical properties of the cells could be indirectly measured by interpolating the Young's moduli of the AHSs. As a result, the calculated Young's moduli, i.e., 1.527 kPa for MDA-MB-231 (highly invasive breast cancer cells), 2.650 kPa for MCF-7 (weakly invasive breast cancer cells), and 2.772 kPa for SKBR-3 (weakly invasive breast cancer cells), indicate that highly invasive cancer cells exhibited a lower Young's moduli than weakly invasive cells, which indicates a higher deformability of highly invasive cancer cells, leading to a higher metastasis rate. Single-cell treatment may also be carried out by bursting a highly invasive cell with high-intensity, focused ultrasound.

Classification of Breast Cancer Cells Using the Integration of High-Frequency Single-Beam Acoustic Tweezers and Convolutional Neural Networks

Single-beam acoustic tweezers (SBAT) is a widely used trapping technique to manipulate microscopic particles or cells. Recently, the characterization of a single cancer cell using high-frequency (>30 MHz) SBAT has been reported to determine its invasiveness and metastatic potential. Investigation of cell elasticity and invasiveness is based on the deformability of cells under SBAT’s radiation forces, and in general, more physically deformed cells exhibit higher levels of invasiveness and therefore higher metastatic potential. However, previous imaging analysis to determine substantial differences in cell deformation, where the SBAT is turned ON or OFF, relies on the subjective observation that may vary and requires follow-up evaluations from experts. In this study, we propose an automatic and reliable cancer cell classification method based on SBAT and a convolutional neural network (CNN), which provides objective and accurate quantitative measurement results. We used a custom-designed 50 MHz SBAT transducer to obtain a series of images of deformed human breast cancer cells. CNN-based classification methods with data augmentation applied to collected images determined and validated the metastatic potential of cancer cells. As a result, with the selected optimizers, precision, and recall of the model were found to be greater than 0.95, which highly validates the classification performance of our integrated method. CNN-guided cancer cell deformation analysis using SBAT may be a promising alternative to current histological image analysis, and this pretrained model will significantly reduce the evaluation time for a larger population of cells.

Whole-Slide Image Focus Quality: Automatic Assessment and Impact on AI Cancer Detection

Background

Digital pathology enables remote access or consults and powerful image analysis algorithms. However, the slide digitization process can create artifacts such as out-of-focus (OOF). OOF is often only detected on careful review, potentially causing rescanning, and workflow delays. Although scan time operator screening for whole-slide OOF is feasible, manual screening for OOF affecting only parts of a slide is impractical.

Methods

We developed a convolutional neural network (ConvFocus) to exhaustively localize and quantify the severity of OOF regions on digitized slides. ConvFocus was developed using our refined semi-synthetic OOF data generation process and evaluated using seven slides spanning three different tissue and three different stain types, each of which were digitized using two different whole-slide scanner models ConvFocus's predictions were compared with pathologist-annotated focus quality grades across 514 distinct regions representing 37,700 35 μm × 35 μm image patches, and 21 digitized "z-stack" WSIs that contain known OOF patterns.

Results

When compared to pathologist-graded focus quality, ConvFocus achieved Spearman rank coefficients of 0.81 and 0.94 on two scanners and reproduced the expected OOF patterns from z-stack scanning. We also evaluated the impact of OOF on the accuracy of a state-of-the-art metastatic breast cancer detector and saw a consistent decrease in performance with increasing OOF.

Conclusions

Comprehensive whole-slide OOF categorization could enable rescans before pathologist review, potentially reducing the impact of digitization focus issues on the clinical workflow. We show that the algorithm trained on our semi-synthetic OOF data generalizes well to real OOF regions across tissue types, stains, and scanners. Finally, quantitative OOF maps can flag regions that might otherwise be misclassified by image analysis algorithms, preventing OOF-induced errors.

Automated and Reproducible Detection of Vascular Endothelial Growth Factor (VEGF) in Renal Tissue Sections

Background

Manual analysis of tissue sections, such as for pathological diagnosis, requires an analyst with substantial knowledge and experience. Reproducible image analysis of biological samples is steadily gaining scientific importance. The aim of the present study was to employ image analysis followed by machine learning to identify vascular endothelial growth factor (VEGF) in kidney tissue that had been subjected to hypoxia.

Methods

Light microscopy images of renal tissue sections stained for VEGF were analyzed. Subsequently, machine learning classified the cells as VEGF⁺ and VEGF⁻ cells.

Results

VEGF was detected and cells were counted with high sensitivity and specificity.

Conclusion

With great clinical, diagnostic, and research potential, automatic image analysis offers a new quantitative capability, thereby adding numerical information to a mostly qualitative diagnostic approach.

DeepFocus: Detection of out-of-focus regions in whole slide digital images using deep learning

The development of whole slide scanners has revolutionized the field of digital pathology. Unfortunately, whole slide scanners often produce images with out-of-focus/blurry areas that limit the amount of tissue available for a pathologist to make accurate diagnosis/prognosis. Moreover, these artifacts hamper the performance of computerized image analysis systems. These areas are typically identified by visual inspection, which leads to a subjective evaluation causing high intra- and inter-observer variability. Moreover, this process is both tedious, and time-consuming. The aim of this study is to develop a deep learning based software called, DeepFocus, which can automatically detect and segment blurry areas in digital whole slide images to address these problems. DeepFocus is built on TensorFlow, an open source library that exploits data flow graphs for efficient numerical computation. DeepFocus was trained by using 16 different H&E and IHC-stained slides that were systematically scanned on nine different focal planes, generating 216,000 samples with varying amounts of blurriness. When trained and tested on two independent datasets, DeepFocus resulted in an average accuracy of 93.2% (± 9.6%), which is a 23.8% improvement over an existing method. DeepFocus has the potential to be integrated with whole slide scanners to automatically re-scan problematic areas, hence improving the overall image quality for pathologists and image analysis algorithms.

Cellular heterogeneity contributes to subtype-specific expression of ZEB1 in human glioblastoma

The transcription factor ZEB1 has gained attention in tumor biology of epithelial cancers because of its function in epithelial-mesenchymal transition, DNA repair, stem cell biology and tumor-induced immunosuppression, but its role in gliomas with respect to invasion and prognostic value is controversial. We characterized ZEB1 expression at single cell level in 266 primary brain tumors and present a comprehensive dataset of high grade gliomas with Ki67, p53, IDH1, and EGFR immunohistochemistry, as well as EGFR FISH. ZEB1 protein expression in glioma stem cell lines was compared to their parental tumors with respect to gene expression subtypes based on RNA-seq transcriptomic profiles. ZEB1 is widely expressed in glial tumors, but in a highly variable fraction of cells. In glioblastoma, ZEB1 labeling index is higher in tumors with EGFR amplification or IDH1 mutation. Co-labeling studies showed that tumor cells and reactive astroglia, but not immune cells contribute to the ZEB1 positive population. In contrast, glioma cell lines constitutively express ZEB1 irrespective of gene expression subtype. In conclusion, our data indicate that immune infiltration likely contributes to differential labelling of ZEB1 and confounds interpretation of bulk ZEB1 expression data.

Objective detection of apoptosis in rat renal tissue sections using light microscopy and free image analysis software with subsequent machine learning: Detection of apoptosis in renal tissue

OBJECTIVE

The current study proposes an automated machine learning approach for the quantification of cells in cell death pathways according to DNA fragmentation.

METHODS

A total of 17 images of kidney histological slide samples from male Wistar rats were used. The slides were photographed using an Axio Zeiss Vert.A1 microscope with a 40x objective lens coupled with an Axio Cam MRC Zeiss camera and Zen 2012 software. The images were analyzed using CellProfiler (version 2.1.1) and CellProfiler Analyst open-source software.

RESULTS

Out of the 10,378 objects, 4970 (47,9%) were identified as TUNEL positive, and 5408 (52,1%) were identified as TUNEL negative. On average, the sensitivity and specificity values of the machine learning approach were 0.80 and 0.77, respectively.

CONCLUSION

Image cytometry provides a quantitative analytical alternative to the more traditional qualitative methods more commonly used in studies.

Parallel Versus Distributed Data Access for Gigapixel-Resolution Histology Images: Challenges and Opportunities

Recent advances in digital pathology technology have led to significant improvements in terms of both the quality and resolution of the resulting images, which now often exceed several gigabytes each. Today, several leading institutions across the country utilize whole-slide imaging (WSI) as part of their routine workflow. WSIs have utility in a wide range of diagnostic and investigative pathology applications. The fact that these images are both large in size (about 30 GB when uncompressed) and are generated in nonstandard proprietary formats has limited wider adoption of these technologies and makes the task of accessing, processing, and analyzing them in high-throughput fashion extremely challenging. The common approach for such data analytic applications is to preprocess the large whole-slide images into smaller size files and store them in a generic format. However, this approach limits the advantages that might be realized if different scalability levels and data unit sizes could be dynamically changed based on the specifications of the task at hand and the architectural limits of the infrastructure (e.g., node memory size). Such strategies also introduce extra processing time to the workflow. To address these challenges, we present, in this paper, novel scalable access methods for parallel file systems and distributed file/object storage systems. Experimental results gathered during the course of our studies show that these methods provide opportunities not realizable using traditional approaches. We demonstrate tangible, scalability, and high-throughput advantages using a Lustre parallel file system and AWS S3 distributed storage system.

Towards better digital pathology workflows: programming libraries for high-speed sharpness assessment of Whole Slide Images

BACKGROUND

Since microscopic slides can now be automatically digitized and integrated in the clinical workflow, quality assessment of Whole Slide Images (WSI) has become a crucial issue. We present a no-reference quality assessment method that has been thoroughly tested since 2010 and is under implementation in multiple sites, both public university-hospitals and private entities. It is part of the FlexMIm R&D project which aims to improve the global workflow of digital pathology. For these uses, we have developed two programming libraries, in Java and Python, which can be integrated in various types of WSI acquisition systems, viewers and image analysis tools.

METHODS

Development and testing have been carried out on a MacBook Pro i7 and on a bi-Xeon 2.7GHz server. Libraries implementing the blur assessment method have been developed in Java, Python, PHP5 and MySQL5. For web applications, JavaScript, Ajax, JSON and Sockets were also used, as well as the Google Maps API. Aperio SVS files were converted into the Google Maps format using VIPS and Openslide libraries.

RESULTS

We designed the Java library as a Service Provider Interface (SPI), extendable by third parties. Analysis is computed in real-time (3 billion pixels per minute). Tests were made on 5000 single images, 200 NDPI WSI, 100 Aperio SVS WSI converted to the Google Maps format.

CONCLUSIONS

Applications based on our method and libraries can be used upstream, as calibration and quality control tool for the WSI acquisition systems, or as tools to reacquire tiles while the WSI is being scanned. They can also be used downstream to reacquire the complete slides that are below the quality threshold for surgical pathology analysis. WSI may also be displayed in a smarter way by sending and displaying the regions of highest quality before other regions. Such quality assessment scores could be integrated as WSI's metadata shared in clinical, research or teaching contexts, for a more efficient medical informatics workflow.

An automated blur detection method for histological whole slide imaging

Whole slide scanners are novel devices that enable high-resolution imaging of an entire histological slide. Furthermore, the imaging is achieved in only a few minutes, which enables image rendering of large-scale studies involving multiple immunohistochemistry biomarkers. Although whole slide imaging has improved considerably, locally poor focusing causes blurred regions of the image. These artifacts may strongly affect the quality of subsequent analyses, making a slide review process mandatory. This tedious and time-consuming task requires the scanner operator to carefully assess the virtual slide and to manually select new focus points. We propose a statistical learning method that provides early image quality feedback and automatically identifies regions of the image that require additional focus points.

Sonographic and pathologic image analysis of pure mucinous carcinoma of the breast

The aims of this study were to elucidate sonographic and histologic features of pure mucinous carcinoma (P-MC) of the breast using quantitative analysis and to evaluate the relationship between quantitative analysis and visual qualitative assessment. Eleven P-MCs (nine patients) were evaluated qualitatively and quantitatively. Three experts assessed these sonographic images using the Breast Imaging Reporting and Data System (BI-RADS) lexicon. For assessment of internal echoes and posterior echoes, quantitative measures were determined using ImageJ software. Histologic thin sections were stained for classification into separate parts of the tumor (stroma, mucin and cancer cells) and were digitized. Internal echoes were isoechoic in 7 of 11 (63.6%) tumors and hypoechoic in 4 of 11 (36.4%); all P-MCs were "enhanced" in qualitative evaluation. As internal echoes increased, the proportion of stroma increased and that of mucin decreased. The high level of internal echoes is correlated with reflection and back-scattering, which are caused mainly by the interface between mucin and stroma.

The history of pathology informatics: A global perspective

Pathology informatics has evolved to varying levels around the world. The history of pathology informatics in different countries is a tale with many dimensions. At first glance, it is the familiar story of individuals solving problems that arise in their clinical practice to enhance efficiency, better manage (e.g., digitize) laboratory information, as well as exploit emerging information technologies. Under the surface, however, lie powerful resource, regulatory, and societal forces that helped shape our discipline into what it is today. In this monograph, for the first time in the history of our discipline, we collectively perform a global review of the field of pathology informatics. In doing so, we illustrate how general far-reaching trends such as the advent of computers, the Internet and digital imaging have affected pathology informatics in the world at large. Major drivers in the field included the need for pathologists to comply with national standards for health information technology and telepathology applications to meet the scarcity of pathology services and trained people in certain countries. Following trials by a multitude of investigators, not all of them successful, it is apparent that innovation alone did not assure the success of many informatics tools and solutions. Common, ongoing barriers to the widespread adoption of informatics devices include poor information technology infrastructure in undeveloped areas, the cost of technology, and regulatory issues. This review offers a deeper understanding of how pathology informatics historically developed and provides insights into what the promising future might hold.

CognitionMaster: an object-based image analysis framework

Background

Automated image analysis methods are becoming more and more important to extract and quantify image features in microscopy-based biomedical studies and several commercial or open-source tools are available. However, most of the approaches rely on pixel-wise operations, a concept that has limitations when high-level object features and relationships between objects are studied and if user-interactivity on the object-level is desired.

Results

In this paper we present an open-source software that facilitates the analysis of content features and object relationships by using objects as basic processing unit instead of individual pixels. Our approach enables also users without programming knowledge to compose “analysis pipelines“ that exploit the object-level approach. We demonstrate the design and use of example pipelines for the immunohistochemistry-based cell proliferation quantification in breast cancer and two-photon fluorescence microscopy data about bone-osteoclast interaction, which underline the advantages of the object-based concept.

Conclusions

We introduce an open source software system that offers object-based image analysis. The object-based concept allows for a straight-forward development of object-related interactive or fully automated image analysis solutions. The presented software may therefore serve as a basis for various applications in the field of digital image analysis.

[Whole slide imaging technology: from digitization to online applications]

As e-health becomes essential to modern care, whole slide images (virtual slides) are now an important clinical, teaching and research tool in pathology. Virtual microscopy consists of digitizing a glass slide by acquiring hundreds of tiles of regions of interest at different zoom levels and assembling them into a structured file. This gigapixel image can then be remotely viewed over a terminal, exactly the way pathologists use a microscope. In this article, we will first describe the key elements of this technology, from the acquisition, using a scanner or a motorized microscope, to the broadcasting of virtual slides through a local or distant viewer over an intranet or Internet connection. As virtual slides are now commonly used in virtual classrooms, clinical data and research databases, we will highlight the main issues regarding its uses in modern pathology. Emphasis will be made on quality assurance policies, standardization and scaling.

Introduction of virtual microscopy in routine surgical pathology--a hypothesis and personal view from Europe

The technology of whole image acquisition from histological glass slides (Virtual slides, (VS)) and its associated software such as image storage, viewers, and virtual microscopy (VM), has matured in the recent years. There is an ongoing discussion whether to introduce VM into routine diagnostic surgical pathology (tissue-based diagnosis) or not, and if these are to be introduced how best to do this. The discussion also centres around how to substantially define the mandatory standards and working conditions related to introducing VM. This article briefly describes some hypotheses alongside our perspective and that of several of our European colleagues who have experienced VS and VM either in research or routine praxis. After consideration of the different opinions and published data the following statements can be derived: 1. Experiences from static and remote telepathology as well as from daily routine diagnoses, confirm that VM is a diagnostic tool that can be handled with the same diagnostic accuracy as conventional microscopy; at least no statistically significant differences (p > 0.05) exist. 2. VM possesses several practical advantages in comparison to conventional microscopy; such as digital image storage and retrieval and contemporary display of multiple images (acquired from different stains, and/or different cases). 3. VM enables fast and efficient feedback between the pathologist and the laboratory in terms of ordered additional stains, automated access to the latest research for references, and fast consultation with outstanding telepathology experts. 4. Industry has already invested “big money” into this technology which certainly will be of influence in its future development. The main constraints against VM include the questionable reimbursement of the initial investment, the missing direct and short term financial benefit, and the loss of potential biological identity between the patient and the examined tissue. This article tries to analyze and evaluate the factors that influence the implementation of VM into routine tissue-based diagnosis, for example in combination with predictive diagnosis. It focuses on describing the advantages of modern and innovative electronically based communication technology.