From whole-slide image to biomarker prediction: end-to-end weakly supervised deep learning in computational pathology

Deep learning algorithm on H&E whole slide images to characterize TP53 alterations frequency and spatial distribution in breast cancer

The tumor suppressor TP53 is frequently mutated in hormone receptor-negative, HER2-positive breast cancer (BC), contributing to tumor aggressiveness. Traditional ancillary methods like immunohistochemistry (IHC) to assess TP53 functionality face pre- and post-analytical challenges. This proof-of-concept study employed a deep learning (DL) algorithm to predict TP53 mutational status from H&E-stained whole slide images (WSIs) of BC tissue. Using a pre-trained convolutional neural network, the model identified tumor areas and predicted TP53 mutations with a Dice coefficient score of 0.82. Predictions were validated through IHC and next-generation sequencing (NGS), confirming TP53 aberrant expression in 92 % of the tumor area, closely matching IHC findings (90 %). The DL model exhibited high accuracy in tissue quantification and TP53 status prediction, outperforming traditional methods in terms of precision and efficiency. DL-based approaches offer significant promise for enhancing biomarker testing and precision oncology by reducing intra- and inter-observer variability, but further validation is required to optimize their integration into real-world clinical workflows. This study underscores the potential of DL algorithms to predict key genetic alterations, such as TP53 mutations, in BC. DL-based histopathological analysis represents a valuable tool for improving patient management and tailoring treatment approaches based on molecular biomarker status.

In-context learning enables multimodal large language models to classify cancer pathology images

Medical image classification requires labeled, task-specific datasets which are used to train deep learning networks de novo, or to fine-tune foundation models. However, this process is computationally and technically demanding. In language processing, in-context learning provides an alternative, where models learn from within prompts, bypassing the need for parameter updates. Yet, in-context learning remains underexplored in medical image analysis. Here, we systematically evaluate the model Generative Pretrained Transformer 4 with Vision capabilities (GPT-4V) on cancer image processing with in-context learning on three cancer histopathology tasks of high importance: Classification of tissue subtypes in colorectal cancer, colon polyp subtyping and breast tumor detection in lymph node sections. Our results show that in-context learning is sufficient to match or even outperform specialized neural networks trained for particular tasks, while only requiring a minimal number of samples. In summary, this study demonstrates that large vision language models trained on non-domain specific data can be applied out-of-the box to solve medical image-processing tasks in histopathology. This democratizes access of generalist AI models to medical experts without technical background especially for areas where annotated data is scarce.