A deep-learning workflow to predict upper tract urothelial carcinoma protein-based subtypes from H&E slides supporting the prioritization of patients for molecular testing

Upper tract urothelial carcinoma (UTUC) is a rare and aggressive, yet understudied, urothelial carcinoma (UC). The more frequent UC of the bladder comprises several molecular subtypes, associated with different targeted therapies and overlapping with protein-based subtypes. However, if and how these findings extend to UTUC remains unclear. Artificial intelligence-based approaches could help elucidate UTUC's biology and extend access to targeted treatments to a wider patient audience. Here, UTUC protein-based subtypes were identified, and a deep-learning (DL) workflow was developed to predict them directly from routine histopathological H&E slides. Protein-based subtypes in a retrospective cohort of 163 invasive tumors were assigned by hierarchical clustering of the immunohistochemical expression of three luminal (FOXA1, GATA3, and CK20) and three basal (CD44, CK5, and CK14) markers. Cluster analysis identified distinctive luminal (N = 80) and basal (N = 42) subtypes. The luminal subtype mostly included pushing, papillary tumors, whereas the basal subtype diffusely infiltrating, non-papillary tumors. DL model building relied on a transfer-learning approach by fine-tuning a pre-trained ResNet50. Classification performance was measured via three-fold repeated cross-validation. A mean area under the receiver operating characteristic curve of 0.83 (95% CI: 0.67-0.99), 0.8 (95% CI: 0.62-0.99), and 0.81 (95% CI: 0.65-0.96) was reached in the three repetitions. High-confidence DL-based predicted subtypes showed significant associations (p < 0.001) with morphological features, i.e. tumor type, histological subtypes, and infiltration type. Furthermore, a significant association was found with programmed cell death ligand 1 (PD-L1) combined positive score (p < 0.001) and FGFR3 mutational status (p = 0.002), with high-confidence basal predictions containing a higher proportion of PD-L1 positive samples and high-confidence luminal predictions a higher proportion of FGFR3-mutated samples. Testing of the DL model on an independent cohort highlighted the importance to accommodate histological subtypes. Taken together, our DL workflow can predict protein-based UTUC subtypes, associated with the presence of targetable alterations, directly from H&E slides.

Proposal for a Novel Histological Scoring System as a Potential Grading Approach for Muscle-invasive Urothelial Bladder Cancer Correlating with Disease Aggressiveness and Patient Outcomes

BACKGROUND

Grading of muscle-invasive bladder cancer (MIBC) according to the current World Health Organization (WHO) criteria is controversial due to its limited prognostic value. All MIBC cases except a tiny minority are of high grade.

OBJECTIVE

To develop a prognostic histological scoring system for MIBC integrating histomorphological phenotype, stromal tumor-infiltrating lymphocytes (sTILs), tumor budding, and growth and spreading patterns.

DESIGN, SETTING, AND PARTICIPANTS

Tissue specimens and clinical data of 484 patients receiving cystectomy and lymphadenectomy with curative intent with or without adjuvant chemotherapy. Histomorphological phenotypes, sTILs, tumor budding, and growth and spreading patterns were evaluated and categorized into four grade groups (GGs). GGs were correlated with molecular subtypes, immune infiltration, immune checkpoint expression, extracellular matrix (ECM) remodeling, and epithelial-mesenchymal transition (EMT) activity.

OUTCOME MEASUREMENTS AND STATISTICAL ANALYSIS

GGs were associated with overall (OS), disease-specific (DSS), and progression-free (PFS) survival in univariable and multivariable analyses. Association with biological features was analyzed with descriptive statistics.

RESULTS AND LIMITATIONS

Integration of two histomorphological tumor groups, three sTILs groups, three tumor budding groups, and four growth/spread patterns yielded four novel GGs that had high significance in the univariable survival analysis (OS, DSS, and PFS). GGs were confirmed as independent prognostic predictors with the greatest effect in the multivariable Cox regression analysis. Correlation with molecular data showed a gradual transition from basal to luminal subtypes from GG1 to GG4; a gradual decrease in survival, immune infiltration, and immune checkpoint activity; and a gradual increase in ECM remodeling and EMT activity.

CONCLUSIONS

We propose a novel, prognostically relevant, and biologically based scoring system for MIBC in cystectomies applicable to routine pathological sections.

PATIENT SUMMARY

We developed a novel approach to assess the aggressiveness of advanced bladder cancer, which allows improved risk stratification compared with the method currently proposed by the World Health Organization.

Built to Last? Reproducibility and Reusability of Deep Learning Algorithms in Computational Pathology

Recent progress in computational pathology has been driven by deep learning. While code and data availability are essential to reproduce findings from preceding publications, ensuring a deep learning model's reusability is more challenging. For that, the codebase should be well-documented and easy to integrate into existing workflows and models should be robust toward noise and generalizable toward data from different sources. Strikingly, only a few computational pathology algorithms have been reused by other researchers so far, let alone employed in a clinical setting. To assess the current state of reproducibility and reusability of computational pathology algorithms, we evaluated peer-reviewed articles available in PubMed, published between January 2019 and March 2021, in 5 use cases: stain normalization; tissue type segmentation; evaluation of cell-level features; genetic alteration prediction; and inference of grading, staging, and prognostic information. We compiled criteria for data and code availability and statistical result analysis and assessed them in 160 publications. We found that only one-quarter (41 of 160 publications) made code publicly available. Among these 41 studies, three-quarters (30 of 41) analyzed their results statistically, half of them (20 of 41) released their trained model weights, and approximately a third (16 of 41) used an independent cohort for evaluation. Our review is intended for both pathologists interested in deep learning and researchers applying algorithms to computational pathology challenges. We provide a detailed overview of publications with published code in the field, list reusable data handling tools, and provide criteria for reproducibility and reusability.

Sporadic Colorectal Tubular Adenomas Thrive in Symbiosis With Underlying Nondysplastic Branching Crypts

BACKGROUND/AIM

Nondysplastic crypt branching (NDCB), mostly asymmetric branching (NDCAB), was previously found beneath the dysplastic epithelium of colorectal tubular adenomas (TA) in Swedish patients. This study examined the frequency of NDCB and NDCAB beneath the dysplastic epithelium of TA, in German patients.

PATIENTS AND METHODS

From a collection of 305 TA, 121 TA fulfilled the prerequisites for inclusion. All NDCB were registered.

RESULTS

Of 673 NDBCs, 572 (85%) NDCABs and 101 (15%) NDCSs, were found beneath the neoplastic tissue in the 121 TA. When the frequency of NDCB was challenged against the TA size, a linear correlation was found in the 121 TA (p<0.05, p=0.020172). Most NDCB were NDCAB (p<0.05, p=0.00001). The frequency of NDCB correlated with increasing TA size, implying that the higher frequency of both NDCB, dysplastic crypt branching, and their dysplastic offspring crypts were the most probable sources of TA enlargement. The frequency of NDCB underneath TA was not influenced by increasing age, sex or TA localization.

CONCLUSION

Similar findings as those reported here were previously found in TA in Swedish patients. The similarity between these two populations, located in disparate geographical areas and subjected to dissimilar microenvironmental conditions suggests that NDBC in TA might be a ubiquitous unreported phenomenon. According to the literature, normal colon cells often harbor somatic mutations. Consequently, NDCB underneath TA may be mutated nondysplastic branching crypts upon which the dysplastic epithelium in TA eventually develops.

Transformer-based biomarker prediction from colorectal cancer histology: A large-scale multicentric study

Deep learning (DL) can accelerate the prediction of prognostic biomarkers from routine pathology slides in colorectal cancer (CRC). However, current approaches rely on convolutional neural networks (CNNs) and have mostly been validated on small patient cohorts. Here, we develop a new transformer-based pipeline for end-to-end biomarker prediction from pathology slides by combining a pre-trained transformer encoder with a transformer network for patch aggregation. Our transformer-based approach substantially improves the performance, generalizability, data efficiency, and interpretability as compared with current state-of-the-art algorithms. After training and evaluating on a large multicenter cohort of over 13,000 patients from 16 colorectal cancer cohorts, we achieve a sensitivity of 0.99 with a negative predictive value of over 0.99 for prediction of microsatellite instability (MSI) on surgical resection specimens. We demonstrate that resection specimen-only training reaches clinical-grade performance on endoscopic biopsy tissue, solving a long-standing diagnostic problem.

The Number of Colon Crypts in Digital Mucosal Samples: A New Independent Parameter for Diagnosing Ulcerative Colitis

Background/Aim

It has been demonstrated that most routine biopsies from the colon and rectum display cross-cut crypts (CCC). The aim was to assess the number of CCC in microscopic isometric digital samples (0.500 mm2) from routine colon biopsies.

Patients and Methods

Colon biopsies from 224 patients were investigated: 99 in patients with ulcerative colitis (UC), 31 UC in remission (UCR), 28 infectious colitis (IC), 7 resolved IC (RIC), 19 diverticular sigmoiditis (DS), and 40 normal colon mucosa (NCM).

Results

A total of 8,024 CCC were registered: 2,860 (35.6%) in UC, 1,319 UCR (16.4%), 849 (10.6%) in IC, 340 (4.2%) in RIC, 795 (9.9%) in DS, and 1,861 (23.2%) in NCM. The CCC frequencies in UC and IC were significantly lower (p<0.05) than those in UCR, RIC, DS, and NCM.

Conclusion

By the simple algorithm of counting CCC in standardized isometric microscopic digital circles measuring 0.500 mm2, it was possible to differentiate between UC (long-lasting inflammation) and IC (short-lasting inflammation) on the one hand, and UCR, RIC, DS (persistent inflammation), and NCM, on the other. The counting of CCC in the algorithm by five pathologists working in three disparate European Countries, was found to be reproducible.

[Artificial Intelligence for computer-aided leukemia diagnostics]

The manual examination of blood and bone marrow specimens for leukemia patients is time-consuming and limited by intra- and inter-observer variance. The development of AI algorithms for leukemia diagnostics requires high-quality sample digitization and reliable annotation of large datasets. Deep learning-based algorithms using these datasets attain human-level performance for some well-defined, clinically relevant questions such as the blast character of cells. Methods such as multiple - instance - learning allow predicting diagnoses from a collection of leukocytes, but are more data-intensive. Using "explainable AI" methods can make the prediction process more transparent and allow users to verify the algorithm's predictions. Stability and robustness analyses are necessary for routine application of these algorithms, and regulatory institutions are developing standards for this purpose. Integrated diagnostics, which link different diagnostic modalities, offer the promise of even greater accuracy but require more extensive and diverse datasets.

Tumor-Stroma Ratio in Colorectal Cancer-Comparison between Human Estimation and Automated Assessment

The tumor-stroma ratio (TSR) has been repeatedly shown to be a prognostic factor for survival prediction of different cancer types. However, an objective and reliable determination of the tumor-stroma ratio remains challenging. We present an easily adaptable deep learning model for accurately segmenting tumor regions in hematoxylin and eosin (H&E)-stained whole slide images (WSIs) of colon cancer patients into five distinct classes (tumor, stroma, necrosis, mucus, and background). The tumor-stroma ratio can be determined in the presence of necrotic or mucinous areas. We employ a few-shot model, eventually aiming for the easy adaptability of our approach to related segmentation tasks or other primaries, and compare the results to a well-established state-of-the art approach (U-Net). Both models achieve similar results with an overall accuracy of 86.5% and 86.7%, respectively, indicating that the adaptability does not lead to a significant decrease in accuracy. Moreover, we comprehensively compare with TSR estimates of human observers and examine in detail discrepancies and inter-rater reliability. Adding a second survey for segmentation quality on top of a first survey for TSR estimation, we found that TSR estimations of human observers are not as reliable a ground truth as previously thought.

Explainable AI identifies diagnostic cells of genetic AML subtypes

Explainable AI is deemed essential for clinical applications as it allows rationalizing model predictions, helping to build trust between clinicians and automated decision support tools. We developed an inherently explainable AI model for the classification of acute myeloid leukemia subtypes from blood smears and found that high-attention cells identified by the model coincide with those labeled as diagnostically relevant by human experts. Based on over 80,000 single white blood cell images from digitized blood smears of 129 patients diagnosed with one of four WHO-defined genetic AML subtypes and 60 healthy controls, we trained SCEMILA, a single-cell based explainable multiple instance learning algorithm. SCEMILA could perfectly discriminate between AML patients and healthy controls and detected the APL subtype with an F1 score of 0.86±0.05 (mean±s.d., 5-fold cross-validation). Analyzing a novel multi-attention module, we confirmed that our algorithm focused with high concordance on the same AML-specific cells as human experts do. Applied to classify single cells, it is able to highlight subtype specific cells and deconvolve the composition of a patient's blood smear without the need of single-cell annotation of the training data. Our large AML genetic subtype dataset is publicly available, and an interactive online tool facilitates the exploration of data and predictions. SCEMILA enables a comparison of algorithmic and expert decision criteria and can present a detailed analysis of individual patient data, paving the way to deploy AI in the routine diagnostics for identifying hematopoietic neoplasms.

More than just sound: Harnessing metadata to improve neural network classifiers for medical auscultation

Label-efficient algorithms are of central importance for machine learning applications in many medical fields, where obtaining expert annotations is often expensive and time-consuming. Soni et al. show how contrastive learning can help build classifiers for one of the oldest and most revered methods of clinical medicine: auscultation of heart and lung sounds.

Machine Learning for Health: Algorithm Auditing & Quality Control

Developers proposing new machine learning for health (ML4H) tools often pledge to match or even surpass the performance of existing tools, yet the reality is usually more complicated. Reliable deployment of ML4H to the real world is challenging as examples from diabetic retinopathy or Covid-19 screening show. We envision an integrated framework of algorithm auditing and quality control that provides a path towards the effective and reliable application of ML systems in healthcare. In this editorial, we give a summary of ongoing work towards that vision and announce a call for participation to the special issue  Machine Learning for Health: Algorithm Auditing & Quality Control in this journal to advance the practice of ML4H auditing.

Tens of images can suffice to train neural networks for malignant leukocyte detection

Convolutional neural networks (CNNs) excel as powerful tools for biomedical image classification. It is commonly assumed that training CNNs requires large amounts of annotated data. This is a bottleneck in many medical applications where annotation relies on expert knowledge. Here, we analyze the binary classification performance of a CNN on two independent cytomorphology datasets as a function of training set size. Specifically, we train a sequential model to discriminate non-malignant leukocytes from blast cells, whose appearance in the peripheral blood is a hallmark of leukemia. We systematically vary training set size, finding that tens of training images suffice for a binary classification with an ROC-AUC over 90%. Saliency maps and layer-wise relevance propagation visualizations suggest that the network learns to increasingly focus on nuclear structures of leukocytes as the number of training images is increased. A low dimensional tSNE representation reveals that while the two classes are separated already for a few training images, the distinction between the classes becomes clearer when more training images are used. To evaluate the performance in a multi-class problem, we annotated single-cell images from a acute lymphoblastic leukemia dataset into six different hematopoietic classes. Multi-class prediction suggests that also here few single-cell images suffice if differences between morphological classes are large enough. The incorporation of deep learning algorithms into clinical practice has the potential to reduce variability and cost, democratize usage of expertise, and allow for early detection of disease onset and relapse. Our approach evaluates the performance of a deep learning based cytology classifier with respect to size and complexity of the training data and the classification task.

The temperature dependence of the helical twist of DNA

DNA is the carrier of all cellular genetic information and increasingly used in nanotechnology. Quantitative understanding and optimization of its functions requires precise experimental characterization and accurate modeling of DNA properties. A defining feature of DNA is its helicity. DNA unwinds with increasing temperature, even for temperatures well below the melting temperature. However, accurate quantitation of DNA unwinding under external forces and a microscopic understanding of the corresponding structural changes are currently lacking. Here we combine single-molecule magnetic tweezers measurements with atomistic molecular dynamics and coarse-grained simulations to obtain a comprehensive view of the temperature dependence of DNA twist. Experimentally, we find that DNA twist changes by ΔTw(T) = (-11.0 ± 1.2)°/(°C·kbp), independent of applied force, in the range of forces where torque-induced melting is negligible. Our atomistic simulations predict ΔTw(T) = (-11.1 ± 0.3)°/(°C·kbp), in quantitative agreement with experiments, and suggest that the untwisting of DNA with temperature is predominantly due to changes in DNA structure for defined backbone substates, while the effects of changes in substate populations are minor. Coarse-grained simulations using the oxDNA framework yield a value of ΔTw(T) = (-6.4 ± 0.2)°/(°C·kbp) in semi-quantitative agreement with experiments.

Coarse-grained modelling of supercoiled RNA

We study the behaviour of double-stranded RNA under twist and tension using oxRNA, a recently developed coarse-grained model of RNA. Introducing explicit salt-dependence into the model allows us to directly compare our results to data from recent single-molecule experiments. The model reproduces extension curves as a function of twist and stretching force, including the buckling transition and the behaviour of plectoneme structures. For negative supercoiling, we predict denaturation bubble formation in plectoneme end-loops, suggesting preferential plectoneme localisation in weak base sequences. OxRNA exhibits a positive twist-stretch coupling constant, in agreement with recent experimental observations.

Long-range correlations in the mechanics of small DNA circles under topological stress revealed by multi-scale simulation

It is well established that gene regulation can be achieved through activator and repressor proteins that bind to DNA and switch particular genes on or off, and that complex metabolic networks determine the levels of transcription of a given gene at a given time. Using three complementary computational techniques to study the sequence-dependence of DNA denaturation within DNA minicircles, we have observed that whenever the ends of the DNA are constrained, information can be transferred over long distances directly by the transmission of mechanical stress through the DNA itself, without any requirement for external signalling factors. Our models combine atomistic molecular dynamics (MD) with coarse-grained simulations and statistical mechanical calculations to span three distinct spatial resolutions and timescale regimes. While they give a consensus view of the non-locality of sequence-dependent denaturation in highly bent and supercoiled DNA loops, each also reveals a unique aspect of long-range informational transfer that occurs as a result of restraining the DNA within the closed loop of the minicircles.

Coarse-graining DNA for simulations of DNA nanotechnology

To simulate long time and length scale processes involving DNA it is necessary to use a coarse-grained description. Here we provide an overview of different approaches to such coarse-graining, focussing on those at the nucleotide level that allow the self-assembly processes associated with DNA nanotechnology to be studied. OxDNA, our recently-developed coarse-grained DNA model, is particularly suited to this task, and has opened up this field to systematic study by simulations. We illustrate some of the range of DNA nanotechnology systems to which the model is being applied, as well as the insights it can provide into fundamental biophysical properties of DNA.

DNA cruciform arms nucleate through a correlated but asynchronous cooperative mechanism

Inverted repeat (IR) sequences in DNA can form noncanonical cruciform structures to relieve torsional stress. We use Monte Carlo simulations of a recently developed coarse-grained model of DNA to demonstrate that the nucleation of a cruciform can proceed through a cooperative mechanism. First, a twist-induced denaturation bubble must diffuse so that its midpoint is near the center of symmetry of the IR sequence. Second, bubble fluctuations must be large enough to allow one of the arms to form a small number of hairpin bonds. Once the first arm is partially formed, the second arm can rapidly grow to a similar size. Because bubbles can twist back on themselves, they need considerably fewer bases to resolve torsional stress than the final cruciform state does. The initially stabilized cruciform therefore continues to grow, which typically proceeds synchronously, reminiscent of the S-type mechanism of cruciform formation. By using umbrella sampling techniques, we calculate, for different temperatures and superhelical densities, the free energy as a function of the number of bonds in each cruciform arm along the correlated but asynchronous nucleation pathways we observed in direct simulations.