Weakly supervised learning on unannotated hematoxylin and eosin stained slides predicts BRAF mutation in thyroid cancer with high accuracy

Computational pathology: A survey review and the way forward

Computational Pathology (CPath) is an interdisciplinary science that augments developments of computational approaches to analyze and model medical histopathology images. The main objective for CPath is to develop infrastructure and workflows of digital diagnostics as an assistive CAD system for clinical pathology, facilitating transformational changes in the diagnosis and treatment of cancer that are mainly address by CPath tools. With evergrowing developments in deep learning and computer vision algorithms, and the ease of the data flow from digital pathology, currently CPath is witnessing a paradigm shift. Despite the sheer volume of engineering and scientific works being introduced for cancer image analysis, there is still a considerable gap of adopting and integrating these algorithms in clinical practice. This raises a significant question regarding the direction and trends that are undertaken in CPath. In this article we provide a comprehensive review of more than 800 papers to address the challenges faced in problem design all-the-way to the application and implementation viewpoints. We have catalogued each paper into a model-card by examining the key works and challenges faced to layout the current landscape in CPath. We hope this helps the community to locate relevant works and facilitate understanding of the field's future directions. In a nutshell, we oversee the CPath developments in cycle of stages which are required to be cohesively linked together to address the challenges associated with such multidisciplinary science. We overview this cycle from different perspectives of data-centric, model-centric, and application-centric problems. We finally sketch remaining challenges and provide directions for future technical developments and clinical integration of CPath. For updated information on this survey review paper and accessing to the original model cards repository, please refer to GitHub. Updated version of this draft can also be found from arXiv.

A digital pathology model for predicting radioiodine-avid metastases on initial post-therapeutic 131I scan in patients with papillary thyroid cancer

Accurate postoperative assessment is critical for optimizing 131I therapy in patients with papillary thyroid cancer (PTC). This study aimed to develop a pathology model utilizing postoperative digital pathology slides to predict lymph node and/or distant metastases on post-therapeutic 131I scan after initial 131I treatment in PTC patients. A retrospective analysis was conducted on 229 PTC patients who underwent total or near-total thyroidectomy and subsequent 131I treatment after levothyroxine (LT4) withdrawal between January 2022 and August 2023. The pathology model was developed through two stages: patch-level prediction and WSI-level prediction. The clinical model was constructed using statistically significant variables identified from univariate and multivariate logistic regression analysis. Of the 229 patients, 19.6% (45/229) exhibited 131I-avid metastatic foci in post-therapeutic 131I scan. Multifactorial analysis identified stimulated thyroglobulin (sTg) as the sole independent risk factor. The AUC of the pathology model in the training and test cohorts were 0.976 (95% CI 0.948-1.000) and 0.805 (95% CI 0.660-0.951), respectively, which were significantly higher than the clinical model (AUC 0.652 and 0.548, Pall < 0.05). This model has the potential to serve as a valuable tool for clinicians in tailoring treatment strategies, thereby optimizing therapeutic outcomes for PTC patients.

From Bench-to-Bedside: How Artificial Intelligence is Changing Thyroid Nodule Diagnostics, a Systematic Review

CONTEXT

Use of artificial intelligence (AI) to predict clinical outcomes in thyroid nodule diagnostics has grown exponentially over the past decade. The greatest challenge is in understanding the best model to apply to one's own patient population, and how to operationalize such a model in practice.

EVIDENCE ACQUISITION

A literature search of PubMed and IEEE Xplore was conducted for English-language publications between January 1, 2015 and January 1, 2023, studying diagnostic tests on suspected thyroid nodules that used AI. We excluded articles without prospective or external validation, nonprimary literature, duplicates, focused on nonnodular thyroid conditions, not using AI, and those incidentally using AI in support of an experimental diagnostic outside standard clinical practice. Quality was graded by Oxford level of evidence.

EVIDENCE SYNTHESIS

A total of 61 studies were identified; all performed external validation, 16 studies were prospective, and 33 compared a model to physician prediction of ground truth. Statistical validation was reported in 50 papers. A diagnostic pipeline was abstracted, yielding 5 high-level outcomes: (1) nodule localization, (2) ultrasound (US) risk score, (3) molecular status, (4) malignancy, and (5) long-term prognosis. Seven prospective studies validated a single commercial AI; strengths included automating nodule feature assessment from US and assisting the physician in predicting malignancy risk, while weaknesses included automated margin prediction and interobserver variability.

CONCLUSION

Models predominantly used US images to predict malignancy. Of 4 Food and Drug Administration-approved products, only S-Detect was extensively validated. Implementing an AI model locally requires data sanitization and revalidation to ensure appropriate clinical performance.

MAMILNet: advancing precision oncology with multi-scale attentional multi-instance learning for whole slide image analysis

BACKGROUND

Whole Slide Image (WSI) analysis, driven by deep learning algorithms, has the potential to revolutionize tumor detection, classification, and treatment response prediction. However, challenges persist, such as limited model generalizability across various cancer types, the labor-intensive nature of patch-level annotation, and the necessity of integrating multi-magnification information to attain a comprehensive understanding of pathological patterns.

METHODS

In response to these challenges, we introduce MAMILNet, an innovative multi-scale attentional multi-instance learning framework for WSI analysis. The incorporation of attention mechanisms into MAMILNet contributes to its exceptional generalizability across diverse cancer types and prediction tasks. This model considers whole slides as "bags" and individual patches as "instances." By adopting this approach, MAMILNet effectively eliminates the requirement for intricate patch-level labeling, significantly reducing the manual workload for pathologists. To enhance prediction accuracy, the model employs a multi-scale "consultation" strategy, facilitating the aggregation of test outcomes from various magnifications.

RESULTS

Our assessment of MAMILNet encompasses 1171 cases encompassing a wide range of cancer types, showcasing its effectiveness in predicting complex tasks. Remarkably, MAMILNet achieved impressive results in distinct domains: for breast cancer tumor detection, the Area Under the Curve (AUC) was 0.8872, with an Accuracy of 0.8760. In the realm of lung cancer typing diagnosis, it achieved an AUC of 0.9551 and an Accuracy of 0.9095. Furthermore, in predicting drug therapy responses for ovarian cancer, MAMILNet achieved an AUC of 0.7358 and an Accuracy of 0.7341.

CONCLUSION

The outcomes of this study underscore the potential of MAMILNet in driving the advancement of precision medicine and individualized treatment planning within the field of oncology. By effectively addressing challenges related to model generalization, annotation workload, and multi-magnification integration, MAMILNet shows promise in enhancing healthcare outcomes for cancer patients. The framework's success in accurately detecting breast tumors, diagnosing lung cancer types, and predicting ovarian cancer therapy responses highlights its significant contribution to the field and paves the way for improved patient care.

A systematic pan-cancer study on deep learning-based prediction of multi-omic biomarkers from routine pathology images

BACKGROUND

The objective of this comprehensive pan-cancer study is to evaluate the potential of deep learning (DL) for molecular profiling of multi-omic biomarkers directly from hematoxylin and eosin (H&E)-stained whole slide images.

METHODS

A total of 12,093 DL models predicting 4031 multi-omic biomarkers across 32 cancer types were trained and validated. The study included a broad range of genetic, transcriptomic, and proteomic biomarkers, as well as established prognostic markers, molecular subtypes, and clinical outcomes.

RESULTS

Here we show that 50% of the models achieve an area under the curve (AUC) of 0.644 or higher. The observed AUC for 25% of the models is at least 0.719 and exceeds 0.834 for the top 5%. Molecular profiling with image-based histomorphological features is generally considered feasible for most of the investigated biomarkers and across different cancer types. The performance appears to be independent of tumor purity, sample size, and class ratio (prevalence), suggesting a degree of inherent predictability in histomorphology.

CONCLUSIONS

The results demonstrate that DL holds promise to predict a wide range of biomarkers across the omics spectrum using only H&E-stained histological slides of solid tumors. This paves the way for accelerating diagnosis and developing more precise treatments for cancer patients.

Artificial Intelligence Detected the Relationship Between Nuclear Morphological Features and Molecular Abnormalities of Papillary Thyroid Carcinoma

Papillary thyroid carcinoma (PTC) is the most common type of thyroid carcinoma and has characteristic nuclear features. Genetic abnormalities of PTC affect recent molecular target therapeutic strategy towards RET-altered cases, and they affect clinical prognosis and progression. However, there has been insufficient objective analysis of the correlation between genetic abnormalities and nuclear features. Using our newly developed methods, we studied the correlation between nuclear morphology and molecular abnormalities of PTC with the aim of predicting genetic abnormalities of PTC. We studied 72 cases of PTC and performed genetic analysis to detect BRAF p.V600E mutation and RET fusions. Nuclear features of PTC, such as nuclear grooves, pseudo-nuclear inclusions, and glassy nuclei, were also automatically detected by deep learning models. After analyzing the correlation between genetic abnormalities and nuclear features of PTC, logistic regression models could be used to predict gene abnormalities. Nuclear features were accurately detected with over 0.90 of AUCs in every class. The ratio of glassy nuclei to nuclear groove and the ratio of pseudo-nuclear inclusion to glassy nuclei were significantly higher in cases that were positive for RET fusions (p = 0.027, p = 0.043, respectively) than in cases that were negative for RET fusions. RET fusions were significantly predicted by glassy nuclei/nuclear grooves, pseudo-nuclear inclusions/glassy nuclei, and age (p = 0.023). Our deep learning models could accurately detect nuclear features. Genetic abnormalities had a correlation with nuclear features of PTC. Furthermore, our artificial intelligence model could significantly predict RET fusions of classic PTC.

Applications of machine and deep learning to thyroid cytology and histopathology: a review

This review synthesises past research into how machine and deep learning can improve the cyto- and histopathology processing pipelines for thyroid cancer diagnosis. The current gold-standard preoperative technique of fine-needle aspiration cytology has high interobserver variability, often returns indeterminate samples and cannot reliably identify some pathologies; histopathology analysis addresses these issues to an extent, but it requires surgical resection of the suspicious lesions so cannot influence preoperative decisions. Motivated by these issues, as well as by the chronic shortage of trained pathologists, much research has been conducted into how artificial intelligence could improve current pipelines and reduce the pressure on clinicians. Many past studies have indicated the significant potential of automated image analysis in classifying thyroid lesions, particularly for those of papillary thyroid carcinoma, but these have generally been retrospective, so questions remain about both the practical efficacy of these automated tools and the realities of integrating them into clinical workflows. Furthermore, the nature of thyroid lesion classification is significantly more nuanced in practice than many current studies have addressed, and this, along with the heterogeneous nature of processing pipelines in different laboratories, means that no solution has proven itself robust enough for clinical adoption. There are, therefore, multiple avenues for future research: examine the practical implementation of these algorithms as pathologist decision-support systems; improve interpretability, which is necessary for developing trust with clinicians and regulators; and investigate multiclassification on diverse multicentre datasets, aiming for methods that demonstrate high performance in a process- and equipment-agnostic manner.

Lymph Node Metastases in Papillary Thyroid Carcinoma can be Predicted by a Convolutional Neural Network: a Multi-Institution Study

OBJECTIVES

The presence of nodal metastases in patients with papillary thyroid carcinoma (PTC) has both staging and treatment implications. However, lymph nodes are often not removed during thyroidectomy. Prior work has demonstrated the capability of artificial intelligence (AI) to predict the presence of nodal metastases in PTC based on the primary tumor histopathology alone. This study aimed to replicate these results with multi-institutional data.

METHODS

Cases of conventional PTC were identified from the records of 2 large academic institutions. Only patients with complete pathology data, including at least 3 sampled lymph nodes, were included in the study. Tumors were designated "positive" if they had at least 5 positive lymph node metastases. First, algorithms were trained separately on each institution's data and tested independently on the other institution's data. Then, the data sets were combined and new algorithms were developed and tested. The primary tumors were randomized into 2 groups, one to train the algorithm and another to test it. A low level of supervision was used to train the algorithm. Board-certified pathologists annotated the slides. HALO-AI convolutional neural network and image software was used to perform training and testing. Receiver operator characteristic curves and the Youden J statistic were used for primary analysis.

RESULTS

There were 420 cases used in analyses, 45% of which were negative. The best performing single institution algorithm had an area under the curve (AUC) of 0.64 with a sensitivity and specificity of 65% and 61% respectively, when tested on the other institution's data. The best performing combined institution algorithm had an AUC of 0.84 with a sensitivity and specificity of 68% and 91% respectively.

CONCLUSION

A convolutional neural network can produce an accurate and robust algorithm that is capable of predicting nodal metastases from primary PTC histopathology alone even in the setting of multi-institutional data.

Deep Learning-Based Differential Diagnosis of Follicular Thyroid Tumors Using Histopathological Images

Deep learning systems (DLSs) have been developed for the histopathological assessment of various types of tumors, but none are suitable for differential diagnosis between follicular thyroid carcinoma (FTC) and follicular adenoma (FA). Furthermore, whether DLSs can identify the malignant characteristics of thyroid tumors based only on random views of tumor tissue histology has not been evaluated. In this study, we developed DLSs able to differentiate between FTC and FA based on 3 types of convolutional neural network architecture: EfficientNet, VGG16, and ResNet50. The performance of all 3 DLSs was excellent (area under the receiver operating characteristic curve = 0.91 ± 0.04; F1 score = 0.82 ± 0.06). Visual explanations using gradient-weighted class activation mapping suggested that the diagnosis of both FTC and FA was largely dependent on nuclear features. The DLSs were then trained with FTC images and linked information (presence or absence of recurrence within 10 years, vascular invasion, and wide capsular invasion). The ability of the DLSs to diagnose these characteristics was then determined. The results showed that, based on the random views of histology, the DLSs could predict the risk of FTC recurrence, vascular invasion, and wide capsular invasion with a certain level of accuracy (area under the receiver operating characteristic curve = 0.67 ± 0.13, 0.62 ± 0.11, and 0.65 ± 0.09, respectively). Further improvement of our DLSs could lead to the establishment of automated differential diagnosis systems requiring only biopsy specimens.

Deep learning in computational dermatopathology of melanoma: A technical systematic literature review

Deep learning (DL) has become one of the major approaches in computational dermatopathology, evidenced by a significant increase in this topic in the current literature. We aim to provide a structured and comprehensive overview of peer-reviewed publications on DL applied to dermatopathology focused on melanoma. In comparison to well-published DL methods on non-medical images (e.g., classification on ImageNet), this field of application comprises a specific set of challenges, such as staining artifacts, large gigapixel images, and various magnification levels. Thus, we are particularly interested in the pathology-specific technical state-of-the-art. We also aim to summarize the best performances achieved thus far with respect to accuracy, along with an overview of self-reported limitations. Accordingly, we conducted a systematic literature review of peer-reviewed journal and conference articles published between 2012 and 2022 in the databases ACM Digital Library, Embase, IEEE Xplore, PubMed, and Scopus, expanded by forward and backward searches to identify 495 potentially eligible studies. After screening for relevance and quality, a total of 54 studies were included. We qualitatively summarized and analyzed these studies from technical, problem-oriented, and task-oriented perspectives. Our findings suggest that the technical aspects of DL for histopathology in melanoma can be further improved. The DL methodology was adopted later in this field, and still lacks the wider adoption of DL methods already shown to be effective for other applications. We also discuss upcoming trends toward ImageNet-based feature extraction and larger models. While DL has achieved human-competitive accuracy in routine pathological tasks, its performance on advanced tasks is still inferior to wet-lab testing (for example). Finally, we discuss the challenges impeding the translation of DL methods to clinical practice and provide insight into future research directions.

Computational pathology in cancer diagnosis, prognosis, and prediction - present day and prospects

Computational pathology refers to applying deep learning techniques and algorithms to analyse and interpret histopathology images. Advances in artificial intelligence (AI) have led to an explosion in innovation in computational pathology, ranging from the prospect of automation of routine diagnostic tasks to the discovery of new prognostic and predictive biomarkers from tissue morphology. Despite the promising potential of computational pathology, its integration in clinical settings has been limited by a range of obstacles including operational, technical, regulatory, ethical, financial, and cultural challenges. Here, we focus on the pathologists' perspective of computational pathology: we map its current translational research landscape, evaluate its clinical utility, and address the more common challenges slowing clinical adoption and implementation. We conclude by describing contemporary approaches to drive forward these techniques. © 2023 The Authors. The Journal of Pathology published by John Wiley & Sons Ltd on behalf of The Pathological Society of Great Britain and Ireland.

Unleashing the potential of AI for pathology: challenges and recommendations

Computational pathology is currently witnessing a surge in the development of AI techniques, offering promise for achieving breakthroughs and significantly impacting the practices of pathology and oncology. These AI methods bring with them the potential to revolutionize diagnostic pipelines as well as treatment planning and overall patient care. Numerous peer-reviewed studies reporting remarkable performance across diverse tasks serve as a testimony to the potential of AI in the field. However, widespread adoption of these methods in clinical and pre-clinical settings still remains a challenge. In this review article, we present a detailed analysis of the major obstacles encountered during the development of effective models and their deployment in practice. We aim to provide readers with an overview of the latest developments, assist them with insights into identifying some specific challenges that may require resolution, and suggest recommendations and potential future research directions. © 2023 The Authors. The Journal of Pathology published by John Wiley & Sons Ltd on behalf of The Pathological Society of Great Britain and Ireland.

Artificial Intelligence to Predict the BRAF V595E Mutation in Canine Urinary Bladder Urothelial Carcinomas

In dogs, the BRAF mutation (V595E) is common in bladder and prostate cancer and represents a specific diagnostic marker. Recent advantages in artificial intelligence (AI) offer new opportunities in the field of tumour marker detection. While AI histology studies have been conducted in humans to detect BRAF mutation in cancer, comparable studies in animals are lacking. In this study, we used commercially available AI histology software to predict BRAF mutation in whole slide images (WSI) of bladder urothelial carcinomas (UC) stained with haematoxylin and eosin (HE), based on a training (n = 81) and a validation set (n = 96). Among 96 WSI, 57 showed identical PCR and AI-based BRAF predictions, resulting in a sensitivity of 58% and a specificity of 63%. The sensitivity increased substantially to 89% when excluding small or poor-quality tissue sections. Test reliability depended on tumour differentiation (p < 0.01), presence of inflammation (p < 0.01), slide quality (p < 0.02) and sample size (p < 0.02). Based on a small subset of cases with available adjacent non-neoplastic urothelium, AI was able to distinguish malignant from benign epithelium. This is the first study to demonstrate the use of AI histology to predict BRAF mutation status in canine UC. Despite certain limitations, the results highlight the potential of AI in predicting molecular alterations in routine tissue sections.

Laser tweezer Raman spectroscopy combined with deep neural networks for identification of liver cancer cells

Rapid identification of cancer cells is crucial for clinical treatment guidance. Laser tweezer Raman spectroscopy (LTRS) that provides biochemical characteristics of cells can be used to identify cell phenotypes through classification models in a non-invasive and label-free manner. However, traditional classification methods require extensive reference databases and clinical experience, which is challenging when sampling at inaccessible locations. Here, we describe a classification method combing LTRS with deep neural network (DNN) for differential and discriminative analysis of multiple liver cancer (LC) cells. By using LTRS, we obtained high-quality single-cell Raman spectra of normal hepatocytes (HL-7702) and liver cancer cell lines (SMMC-7721, Hep3B, HepG2, SK-Hep1 and Huh7). The tentative assignment of Raman peaks indicated that arginine content was elevated and phenylalanine, glutathione and glutamate content was decreased in liver cancer cells. Subsequently, we randomly selected 300 spectra from each cell line for DNN model analysis, achieving a mean accuracy of 99.2%, a mean sensitivity of 99.2% and a mean specificity of 99.8% for the identification and classification of multiple LC cells and hepatocyte cells. These results demonstrate the combination of LTRS and DNN is a promising method for rapid and accurate cancer cell identification at single cell level.

Recent application of artificial intelligence on histopathologic image-based prediction of gene mutation in solid cancers

PURPOSE

Evaluation of genetic mutations in cancers is important because distinct mutational profiles help determine individualized drug therapy. However, molecular analyses are not routinely performed in all cancers because they are expensive, time-consuming and not universally available. Artificial intelligence (AI) has shown the potential to determine a wide range of genetic mutations on histologic image analysis. Here, we assessed the status of mutation prediction AI models on histologic images by a systematic review.

METHODS

A literature search using the MEDLINE, Embase and Cochrane databases was conducted in August 2021. The articles were shortlisted by titles and abstracts. After a full-text review, publication trends, study characteristic analysis and comparison of performance metrics were performed.

RESULTS

Twenty-four studies were found mostly from developed countries, and their number is increasing. The major targets were gastrointestinal, genitourinary, gynecological, lung and head and neck cancers. Most studies used the Cancer Genome Atlas, with a few using an in-house dataset. The area under the curve of some of the cancer driver gene mutations in particular organs was satisfactory, such as 0.92 of BRAF in thyroid cancers and 0.79 of EGFR in lung cancers, whereas the average of all gene mutations was 0.64, which is still suboptimal.

CONCLUSION

AI has the potential to predict gene mutations on histologic images with appropriate caution. Further validation with larger datasets is still required before AI models can be used in clinical practice to predict gene mutations.

A robust and lightweight deep attention multiple instance learning algorithm for predicting genetic alterations

Self-attention mechanism-based algorithms are attractive in digital pathology due to their interpretability, but suffer from computation complexity. This paper presents a novel, lightweight Attention-based Multiple Instance Mutation Learning (AMIML) model to allow small-scale attention operations for predicting gene mutations. Compared to the standard self-attention model, AMIML reduces the number of model parameters by approximately 70%. Using data for 24 clinically relevant genes from four cancer cohorts in TCGA studies (UCEC, BRCA, GBM, and KIRC), we compare AMIML with a standard self-attention model, five other deep learning models, and four traditional machine learning models. The results show that AMIML has excellent robustness and outperforms all the baseline algorithms in the vast majority of the tested genes. Conversely, the performance of the reference deep learning and machine learning models vary across different genes, and produce suboptimal prediction for certain genes. Furthermore, with the flexible and interpretable attention-based pooling mechanism, AMIML can further zero in and detect predictive image patches.

Applications of Deep Learning in Endocrine Neoplasms

Machine learning methods have been growing in prominence across all areas of medicine. In pathology, recent advances in deep learning (DL) have enabled computational analysis of histological samples, aiding in diagnosis and characterization in multiple disease areas. In cancer, and particularly endocrine cancer, DL approaches have been shown to be useful in tasks ranging from tumor grading to gene expression prediction. This review summarizes the current state of DL research in endocrine cancer histopathology with an emphasis on experimental design, significant findings, and key limitations.

Intelligent oncology: The convergence of artificial intelligence and oncology

With increasingly explored ideologies and technologies for potential applications of artificial intelligence (AI) in oncology, we here describe a holistic and structured concept termed intelligent oncology. Intelligent oncology is defined as a cross-disciplinary specialty which integrates oncology, radiology, pathology, molecular biology, multi-omics and computer sciences, aiming to promote cancer prevention, screening, early diagnosis and precision treatment. The development of intelligent oncology has been facilitated by fast AI technology development such as natural language processing, machine/deep learning, computer vision, and robotic process automation. While the concept and applications of intelligent oncology is still in its infancy, and there are still many hurdles and challenges, we are optimistic that it will play a pivotal role for the future of basic, translational and clinical oncology.

Annotation-Free Deep Learning-Based Prediction of Thyroid Molecular Cancer Biomarker BRAF (V600E) from Cytological Slides

Thyroid cancer is the most common endocrine cancer. Papillary thyroid cancer (PTC) is the most prevalent form of malignancy among all thyroid cancers arising from follicular cells. Fine needle aspiration cytology (FNAC) is a non-invasive method regarded as the most cost-effective and accurate diagnostic method of choice in diagnosing PTC. Identification of BRAF (V600E) mutation in thyroid neoplasia may be beneficial because it is specific for malignancy, implies a worse prognosis, and is the target for selective BRAF inhibitors. To the authors' best knowledge, this is the first automated precision oncology framework effectively predict BRAF (V600E) immunostaining result in thyroidectomy specimen directly from Papanicolaou-stained thyroid fine-needle aspiration cytology and ThinPrep cytological slides, which is helpful for novel targeted therapies and prognosis prediction. The proposed deep learning (DL) framework is evaluated on a dataset of 118 whole slide images. The results show that the proposed DL-based technique achieves an accuracy of 87%, a precision of 94%, a sensitivity of 91%, a specificity of 71% and a mean of sensitivity and specificity at 81% and outperformed three state-of-the-art deep learning approaches. This study demonstrates the feasibility of DL-based prediction of critical molecular features in cytological slides, which not only aid in accurate diagnosis but also provide useful information in guiding clinical decision-making in patients with thyroid cancer. With the accumulation of data and the continuous advancement of technology, the performance of DL systems is expected to be improved in the near future. Therefore, we expect that DL can provide a cost-effective and time-effective alternative tool for patients in the era of precision oncology.

Radio-pathomic approaches in pediatric neuro-oncology: Opportunities and challenges

With medical software platforms moving to cloud environments with scalable storage and computing, the translation of predictive artificial intelligence (AI) models to aid in clinical decision-making and facilitate personalized medicine for cancer patients is becoming a reality. Medical imaging, namely radiologic and histologic images, has immense analytical potential in neuro-oncology, and models utilizing integrated radiomic and pathomic data may yield a synergistic effect and provide a new modality for precision medicine. At the same time, the ability to harness multi-modal data is met with challenges in aggregating data across medical departments and institutions, as well as significant complexity in modeling the phenotypic and genotypic heterogeneity of pediatric brain tumors. In this paper, we review recent pathomic and integrated pathomic, radiomic, and genomic studies with clinical applications. We discuss current challenges limiting translational research on pediatric brain tumors and outline technical and analytical solutions. Overall, we propose that to empower the potential residing in radio-pathomics, systemic changes in cross-discipline data management and end-to-end software platforms to handle multi-modal data sets are needed, in addition to embracing modern AI-powered approaches. These changes can improve the performance of predictive models, and ultimately the ability to advance brain cancer treatments and patient outcomes through the development of such models.

Trends in AI-powered Classification of Thyroid Neoplasms Based on Histopathology Images - a Systematic Review

Background

Assessment of thyroid nodules histopathology using AI is crucial for an accurate diagnosis. This systematic review analyzes recent works employing deep learning approaches for classifying thyroid nodules based on histopathology images, evaluating their performance, and identifying limitations.

Methods

Eligibility criteria focused on peer-reviewed English papers published in the last 5 years, applying deep learning to categorize thyroid histopathology images. The PubMed database was searched using relevant keyword combinations.

Results

Out of 103 articles, 11 studies met inclusion criteria. They used convolutional neural networks to classify thyroid neoplasm. Most studies aimed for basic tumor subtyping; however, 3 studies targeted the prediction of tumor-associated mutation. Accuracy ranged from 77% to 100%, with most over 90%.

Discussion

The findings from our analysis reveal the effectiveness of deep learning in identifying discriminative morphological patterns from histopathology images, thus enhancing the accuracy of thyroid nodule histopathological classification. Key limitations were small sample sizes, subjective annotation, and limited dataset diversity. Further research with larger diverse datasets, model optimization, and real-world validation is essential to translate these tools into clinical practice.

Efficient quality control of whole slide pathology images with human-in-the-loop training

Histopathology whole slide images (WSIs) are being widely used to develop deep learning-based diagnostic solutions, especially for precision oncology. Most of these diagnostic softwares are vulnerable to biases and impurities in the training and test data which can lead to inaccurate diagnoses. For instance, WSIs contain multiple types of tissue regions, at least some of which might not be relevant to the diagnosis. We introduce HistoROI, a robust yet lightweight deep learning-based classifier to segregate WSI into 6 broad tissue regions-epithelium, stroma, lymphocytes, adipose, artifacts, and miscellaneous. HistoROI is trained using a novel human in-the-loop and active learning paradigm that ensures variations in training data for labeling efficient generalization. HistoROI consistently performs well across multiple organs, despite being trained on only a single dataset, demonstrating strong generalization. Further, we have examined the utility of HistoROI in improving the performance of downstream deep learning-based tasks using the CAMELYON breast cancer lymph node and TCGA lung cancer datasets. For the former dataset, the area under the receiver operating characteristic curve (AUC) for metastasis versus normal tissue of a neural network trained using weakly supervised learning increased from 0.88 to 0.92 by filtering the data using HistoROI. Similarly, the AUC increased from 0.88 to 0.93 for the classification between adenocarcinoma and squamous cell carcinoma on the lung cancer dataset. We also found that the performance of the HistoROI improves upon HistoQC for artifact detection on a test dataset of 93 annotated WSIs. The limitations of the proposed model are analyzed, and potential extensions are also discussed.

Deep Learning-Based Prediction of Molecular Tumor Biomarkers from H&E: A Practical Review

Molecular and genomic properties are critical in selecting cancer treatments to target individual tumors, particularly for immunotherapy. However, the methods to assess such properties are expensive, time-consuming, and often not routinely performed. Applying machine learning to H&E images can provide a more cost-effective screening method. Dozens of studies over the last few years have demonstrated that a variety of molecular biomarkers can be predicted from H&E alone using the advancements of deep learning: molecular alterations, genomic subtypes, protein biomarkers, and even the presence of viruses. This article reviews the diverse applications across cancer types and the methodology to train and validate these models on whole slide images. From bottom-up to pathologist-driven to hybrid approaches, the leading trends include a variety of weakly supervised deep learning-based approaches, as well as mechanisms for training strongly supervised models in select situations. While results of these algorithms look promising, some challenges still persist, including small training sets, rigorous validation, and model explainability. Biomarker prediction models may yield a screening method to determine when to run molecular tests or an alternative when molecular tests are not possible. They also create new opportunities in quantifying intratumoral heterogeneity and predicting patient outcomes.

Label-Free Virtual HER2 Immunohistochemical Staining of Breast Tissue using Deep Learning

The immunohistochemical (IHC) staining of the human epidermal growth factor receptor 2 (HER2) biomarker is widely practiced in breast tissue analysis, preclinical studies, and diagnostic decisions, guiding cancer treatment and investigation of pathogenesis. HER2 staining demands laborious tissue treatment and chemical processing performed by a histotechnologist, which typically takes one day to prepare in a laboratory, increasing analysis time and associated costs. Here, we describe a deep learning-based virtual HER2 IHC staining method using a conditional generative adversarial network that is trained to rapidly transform autofluorescence microscopic images of unlabeled/label-free breast tissue sections into bright-field equivalent microscopic images, matching the standard HER2 IHC staining that is chemically performed on the same tissue sections. The efficacy of this virtual HER2 staining framework was demonstrated by quantitative analysis, in which three board-certified breast pathologists blindly graded the HER2 scores of virtually stained and immunohistochemically stained HER2 whole slide images (WSIs) to reveal that the HER2 scores determined by inspecting virtual IHC images are as accurate as their immunohistochemically stained counterparts. A second quantitative blinded study performed by the same diagnosticians further revealed that the virtually stained HER2 images exhibit a comparable staining quality in the level of nuclear detail, membrane clearness, and absence of staining artifacts with respect to their immunohistochemically stained counterparts. This virtual HER2 staining framework bypasses the costly, laborious, and time-consuming IHC staining procedures in laboratory and can be extended to other types of biomarkers to accelerate the IHC tissue staining used in life sciences and biomedical workflow.

Artificial intelligence to identify genetic alterations in conventional histopathology

Precision oncology relies on the identification of targetable molecular alterations in tumor tissues. In many tumor types, a limited set of molecular tests is currently part of standard diagnostic workflows. However, universal testing for all targetable alterations, especially rare ones, is limited by the cost and availability of molecular assays. From 2017 to 2021, multiple studies have shown that artificial intelligence (AI) methods can predict the probability of specific genetic alterations directly from conventional hematoxylin and eosin (H&E) tissue slides. Although these methods are currently less accurate than gold-standard testing (e.g. immunohistochemistry, polymerase chain reaction or next-generation sequencing), they could be used as pre-screening tools to reduce the workload of genetic analyses. In this systematic literature review, we summarize the state of the art in predicting molecular alterations from H&E using AI. We found that AI methods perform reasonably well across multiple tumor types, although few algorithms have been broadly validated. In addition, we found that genetic alterations in FGFR, IDH, PIK3CA, BRAF, TP53 and DNA repair pathways are predictable from H&E in multiple tumor types, while many other genetic alterations have rarely been investigated or were only poorly predictable. Finally, we discuss the next steps for the implementation of AI-based surrogate tests in diagnostic workflows. This article is protected by copyright. All rights reserved.