RegWSI: Whole slide image registration using combined deep feature- and intensity-based methods: Winner of the ACROBAT 2023 challenge

BACKGROUND AND OBJECTIVE

The automatic registration of differently stained whole slide images (WSIs) is crucial for improving diagnosis and prognosis by fusing complementary information emerging from different visible structures. It is also useful to quickly transfer annotations between consecutive or restained slides, thus significantly reducing the annotation time and associated costs. Nevertheless, the slide preparation is different for each stain and the tissue undergoes complex and large deformations. Therefore, a robust, efficient, and accurate registration method is highly desired by the scientific community and hospitals specializing in digital pathology.

METHODS

We propose a two-step hybrid method consisting of (i) deep learning- and feature-based initial alignment algorithm, and (ii) intensity-based nonrigid registration using the instance optimization. The proposed method does not require any fine-tuning to a particular dataset and can be used directly for any desired tissue type and stain. The registration time is low, allowing one to perform efficient registration even for large datasets. The method was proposed for the ACROBAT 2023 challenge organized during the MICCAI 2023 conference and scored 1st place. The method is released as open-source software.

RESULTS

The proposed method is evaluated using three open datasets: (i) Automatic Nonrigid Histological Image Registration Dataset (ANHIR), (ii) Automatic Registration of Breast Cancer Tissue Dataset (ACROBAT), and (iii) Hybrid Restained and Consecutive Histological Serial Sections Dataset (HyReCo). The target registration error (TRE) is used as the evaluation metric. We compare the proposed algorithm to other state-of-the-art solutions, showing considerable improvement. Additionally, we perform several ablation studies concerning the resolution used for registration and the initial alignment robustness and stability. The method achieves the most accurate results for the ACROBAT dataset, the cell-level registration accuracy for the restained slides from the HyReCo dataset, and is among the best methods evaluated on the ANHIR dataset.

CONCLUSIONS

The article presents an automatic and robust registration method that outperforms other state-of-the-art solutions. The method does not require any fine-tuning to a particular dataset and can be used out-of-the-box for numerous types of microscopic images. The method is incorporated into the DeeperHistReg framework, allowing others to directly use it to register, transform, and save the WSIs at any desired pyramid level (resolution up to 220k x 220k). We provide free access to the software. The results are fully and easily reproducible. The proposed method is a significant contribution to improving the WSI registration quality, thus advancing the field of digital pathology.

On-cloud decision-support system for non-small cell lung cancer histology characterization from thorax computed tomography scans

Non-Small Cell Lung Cancer (NSCLC) accounts for about 85% of all lung cancers. Developing non-invasive techniques for NSCLC histology characterization may not only help clinicians to make targeted therapeutic treatments but also prevent subjects from undergoing lung biopsy, which is challenging and could lead to clinical implications. The motivation behind the study presented here is to develop an advanced on-cloud decision-support system, named LUCY, for non-small cell LUng Cancer histologY characterization directly from thorax Computed Tomography (CT) scans. This aim was pursued by selecting thorax CT scans of 182 LUng ADenocarcinoma (LUAD) and 186 LUng Squamous Cell carcinoma (LUSC) subjects from four openly accessible data collections (NSCLC-Radiomics, NSCLC-Radiogenomics, NSCLC-Radiomics-Genomics and TCGA-LUAD), in addition to the implementation and comparison of two end-to-end neural networks (the core layer of whom is a convolutional long short-term memory layer), the performance evaluation on test dataset (NSCLC-Radiomics-Genomics) from a subject-level perspective in relation to NSCLC histological subtype location and grade, and the dynamic visual interpretation of the achieved results by producing and analyzing one heatmap video for each scan. LUCY reached test Area Under the receiver operating characteristic Curve (AUC) values above 77% in all NSCLC histological subtype location and grade groups, and a best AUC value of 97% on the entire dataset reserved for testing, proving high generalizability to heterogeneous data and robustness. Thus, LUCY is a clinically-useful decision-support system able to timely, non-invasively and reliably provide visually-understandable predictions on LUAD and LUSC subjects in relation to clinically-relevant information.

Modelling digital health data: The ExaMode ontology for computational pathology

Computational pathology can significantly benefit from ontologies to standardize the employed nomenclature and help with knowledge extraction processes for high-quality annotated image datasets. The end goal is to reach a shared model for digital pathology to overcome data variability and integration problems. Indeed, data annotation in such a specific domain is still an unsolved challenge and datasets cannot be steadily reused in diverse contexts due to heterogeneity issues of the adopted labels, multilingualism, and different clinical practices.

Material and methods

This paper presents the ExaMode ontology, modeling the histopathology process by considering 3 key cancer diseases (colon, cervical, and lung tumors) and celiac disease. The ExaMode ontology has been designed bottom-up in an iterative fashion with continuous feedback and validation from pathologists and clinicians. The ontology is organized into 5 semantic areas that defines an ontological template to model any disease of interest in histopathology.

Results

The ExaMode ontology is currently being used as a common semantic layer in: (i) an entity linking tool for the automatic annotation of medical records; (ii) a web-based collaborative annotation tool for histopathology text reports; and (iii) a software platform for building holistic solutions integrating multimodal histopathology data.

Discussion

The ontology ExaMode is a key means to store data in a graph database according to the RDF data model. The creation of an RDF dataset can help develop more accurate algorithms for image analysis, especially in the field of digital pathology. This approach allows for seamless data integration and a unified query access point, from which we can extract relevant clinical insights about the considered diseases using SPARQL queries.

Data-driven color augmentation for H&E stained images in computational pathology

Computational pathology targets the automatic analysis of Whole Slide Images (WSI). WSIs are high-resolution digitized histopathology images, stained with chemical reagents to highlight specific tissue structures and scanned via whole slide scanners. The application of different parameters during WSI acquisition may lead to stain color heterogeneity, especially considering samples collected from several medical centers. Dealing with stain color heterogeneity often limits the robustness of methods developed to analyze WSIs, in particular Convolutional Neural Networks (CNN), the state-of-the-art algorithm for most computational pathology tasks. Stain color heterogeneity is still an unsolved problem, although several methods have been developed to alleviate it, such as Hue-Saturation-Contrast (HSC) color augmentation and stain augmentation methods. The goal of this paper is to present Data-Driven Color Augmentation (DDCA), a method to improve the efficiency of color augmentation methods by increasing the reliability of the samples used for training computational pathology models. During CNN training, a database including over 2 million H&E color variations collected from private and public datasets is used as a reference to discard augmented data with color distributions that do not correspond to realistic data. DDCA is applied to HSC color augmentation, stain augmentation and H&E-adversarial networks in colon and prostate cancer classification tasks. DDCA is then compared with 11 state-of-the-art baseline methods to handle color heterogeneity, showing that it can substantially improve classification performance on unseen data including heterogeneous color variations.

Unleashing the potential of digital pathology data by training computer-aided diagnosis models without human annotations

The digitalization of clinical workflows and the increasing performance of deep learning algorithms are paving the way towards new methods for tackling cancer diagnosis. However, the availability of medical specialists to annotate digitized images and free-text diagnostic reports does not scale with the need for large datasets required to train robust computer-aided diagnosis methods that can target the high variability of clinical cases and data produced. This work proposes and evaluates an approach to eliminate the need for manual annotations to train computer-aided diagnosis tools in digital pathology. The approach includes two components, to automatically extract semantically meaningful concepts from diagnostic reports and use them as weak labels to train convolutional neural networks (CNNs) for histopathology diagnosis. The approach is trained (through 10-fold cross-validation) on 3’769 clinical images and reports, provided by two hospitals and tested on over 11’000 images from private and publicly available datasets. The CNN, trained with automatically generated labels, is compared with the same architecture trained with manual labels. Results show that combining text analysis and end-to-end deep neural networks allows building computer-aided diagnosis tools that reach solid performance (micro-accuracy = 0.908 at image-level) based only on existing clinical data without the need for manual annotations.

Multi_Scale_Tools: A Python Library to Exploit Multi-Scale Whole Slide Images

Algorithms proposed in computational pathology can allow to automatically analyze digitized tissue samples of histopathological images to help diagnosing diseases. Tissue samples are scanned at a high-resolution and usually saved as images with several magnification levels, namely whole slide images (WSIs). Convolutional neural networks (CNNs) represent the state-of-the-art computer vision methods targeting the analysis of histopathology images, aiming for detection, classification and segmentation. However, the development of CNNs that work with multi-scale images such as WSIs is still an open challenge. The image characteristics and the CNN properties impose architecture designs that are not trivial. Therefore, single scale CNN architectures are still often used. This paper presents Multi_Scale_Tools, a library aiming to facilitate exploiting the multi-scale structure of WSIs. Multi_Scale_Tools currently include four components: a pre-processing component, a scale detector, a multi-scale CNN for classification and a multi-scale CNN for segmentation of the images. The pre-processing component includes methods to extract patches at several magnification levels. The scale detector allows to identify the magnification level of images that do not contain this information, such as images from the scientific literature. The multi-scale CNNs are trained combining features and predictions that originate from different magnification levels. The components are developed using private datasets, including colon and breast cancer tissue samples. They are tested on private and public external data sources, such as The Cancer Genome Atlas (TCGA). The results of the library demonstrate its effectiveness and applicability. The scale detector accurately predicts multiple levels of image magnification and generalizes well to independent external data. The multi-scale CNNs outperform the single-magnification CNN for both classification and segmentation tasks. The code is developed in Python and it will be made publicly available upon publication. It aims to be easy to use and easy to be improved with additional functions.

Combining weakly and strongly supervised learning improves strong supervision in Gleason pattern classification

BACKGROUND

One challenge to train deep convolutional neural network (CNNs) models with whole slide images (WSIs) is providing the required large number of costly, manually annotated image regions. Strategies to alleviate the scarcity of annotated data include: using transfer learning, data augmentation and training the models with less expensive image-level annotations (weakly-supervised learning). However, it is not clear how to combine the use of transfer learning in a CNN model when different data sources are available for training or how to leverage from the combination of large amounts of weakly annotated images with a set of local region annotations. This paper aims to evaluate CNN training strategies based on transfer learning to leverage the combination of weak and strong annotations in heterogeneous data sources. The trade-off between classification performance and annotation effort is explored by evaluating a CNN that learns from strong labels (region annotations) and is later fine-tuned on a dataset with less expensive weak (image-level) labels.

RESULTS

As expected, the model performance on strongly annotated data steadily increases as the percentage of strong annotations that are used increases, reaching a performance comparable to pathologists ([Formula: see text]). Nevertheless, the performance sharply decreases when applied for the WSI classification scenario with [Formula: see text]. Moreover, it only provides a lower performance regardless of the number of annotations used. The model performance increases when fine-tuning the model for the task of Gleason scoring with the weak WSI labels [Formula: see text].

CONCLUSION

Combining weak and strong supervision improves strong supervision in classification of Gleason patterns using tissue microarrays (TMA) and WSI regions. Our results contribute very good strategies for training CNN models combining few annotated data and heterogeneous data sources. The performance increases in the controlled TMA scenario with the number of annotations used to train the model. Nevertheless, the performance is hindered when the trained TMA model is applied directly to the more challenging WSI classification problem. This demonstrates that a good pre-trained model for prostate cancer TMA image classification may lead to the best downstream model if fine-tuned on the WSI target dataset. We have made available the source code repository for reproducing the experiments in the paper: https://github.com/ilmaro8/Digital_Pathology_Transfer_Learning.

Iron excess in rice: from phenotypic changes to functional genomics of WRKY transcription factors

Iron (Fe) is an essential microelement for all living organisms playing important roles in several metabolic reactions. Rice (Oryza sativa L.) is commonly cultivated in paddy fields, where Fe goes through a reduction reaction from Fe³⁺ to Fe²⁺. Since Fe²⁺ is more soluble, it can reach toxic levels inside plant cells, constituting an important target for studies. Here we aimed to verify morphological changes of different rice genotypes focusing on deciphering the underlying molecular network induced upon Fe excess treatments with special emphasis on the role of four WRKY transcription factors. The transcriptional response peak of these WRKY transcription factors in rice seedlings occurs at 4 days of exposition to iron excess. OsWRKY55-like, OsWRKY46, OsWRKY64, and OsWRKY113 are up-regulated in BR IRGA 409, an iron-sensitive genotype, while in cultivars Nipponbare (moderately resistant) and EPAGRI 108 (resistant) the expression profiles of these transcription factors show similar behaviors. Here is also shown that some cis-regulatory elements known to be involved in other different stress responses can be linked to conditions of iron excess. Overall, here we support the role of WRKY transcription factors in iron stress tolerance with other important steps toward finding why some rice genotypes are more tolerant than others.

Identification of reference genes for RT-qPCR analysis in peach genotypes with contrasting chilling requirements

Selecting and validating reference genes are the first steps in studying gene expression by reverse transcriptase-quantitative polymerase chain reaction (RT-qPCR). The present study aimed to evaluate the stability of five reference genes for the purpose of normalization when studying gene expression in various cultivars of Prunus persica with different chilling requirements. Flower bud tissues of nine peach genotypes from Embrapa's peach breeding program with different chilling requirements were used, and five candidate reference genes based on the RT-qPCR that were useful for studying the relative quantitative gene expression and stability were evaluated using geNorm, NormFinder, and bestKeeper software packages. The results indicated that among the genes tested, the most stable genes to be used as reference genes are Act and UBQ10. This study is the first survey of the stability of reference genes in peaches under chilling stress and provides guidelines for more accurate RT-qPCR results.

Thermal behaviour modelling of tapered optical fibres for scanning near-field microscopy

A simple model allowing the calculation of the thermal field inside a metal-coated fibre tip is presented. The approach has been based on previous temperature measurements which operated in steady state and periodic rate. The modelhas been inspired from the general theory of heat transfer inside fins, after having divided the taper into a set of layers. The advantage of the method is the possibility to consider any taper shapes. Moreover, any kind of coating thickness and external heat transfer distributions can be considered. As a mean of comparison with some previous works, results obtained for simple configurations are presented. Then, a study of the main governing parameters provides the basic thermal behaviour analysis of optical tips and a comparison with experience is given in order to confirm the validity of our approach.

Seasonal mood patterns in eating disorders

There has been an increasing body of research literature suggesting a seasonal pattern of mood fluctuations and eating behavior in bulimic patients. Fornari et al. [5] reported worsening of bulimic symptoms during winter. There is a logical connection between Seasonal Affective Disorder (SAD) and bulimia nervosa as both show increased appetite and carbohydrate craving and probably share a common neurobiologic abnormality such as serotonergic dysfunction. The aim of this study was to determine the prevalence of SAD in a sample of 259 consecutively evaluated outpatients admitted to an eating disorders clinic (254 women and 5 men). Eating disorder diagnosis was established on the basis of DSM-III-R criteria, and a modified version of the Seasonal Pattern Assessment Questionnaire was used to determine seasonality among patients. The sample was comprised of the following: 53.7% bulimics, 27.4% anorexics, 15.1% were classified as having an eating disorder not otherwise specified, and 3.9% had a diagnosis other than an eating disorder. The results indicated that 27.0% of the eating disorder patients met criteria for SAD. Of this group, 86 (71.4%) were bulimic, 35 (18.6%) were anorexic, and 20 (10.0%) were nonspecified. Details and additional findings are discussed.

Supracrestal circular collagen fiber network around osseointegrated nonsubmerged titanium implants

Eight non-submerged titanium implant screws were placed in the first upper molar edentulous sites of monkeys and subsequently kept loaded with single crown prosthesis 1 month following implantation. The animals were killed after a further 14 months and specimens including implant and adjacent teeth were processed for light and electron microscopy. Histological pictures of all samples showed the neck and most of the screw body to be surrounded by new bone. The soft tissue surrounding the implant post included pocket epithelium and supra-crestal connective tissue displaying collagen fiber bundles comparable to gingival ligaments. These peri-implant collagen fiber bundles arose from the neighboring alveolar crest, root cementum of adjacent teeth or, superficially, from the epithelium and followed a circular array around the implant neck.