Recent Applications of Artificial Intelligence from Histopathologic Image-Based Prediction of Microsatellite Instability in Solid Cancers: A Systematic Review

Accuracy of machine learning in diagnosing microsatellite instability in gastric cancer: A systematic review and meta-analysis

BACKGROUND

Significant challenges persist in the early identification of microsatellite instability (MSI) within current clinical practice. In recent years, with the growing utilization of machine learning (ML) in the diagnosis and management of gastric cancer (GC), numerous researchers have explored the effectiveness of ML methodologies in detecting MSI. Nevertheless, the predictive value of these approaches still lacks comprehensive evidence. Accordingly, this study was carried out to consolidate the accuracy of ML in the prompt detection of MSI in GC.

METHODS

PubMed, the Cochrane Library, the Web of Science, and Embase were retrieved up to March 20, 2024. The risk of bias in the encompassed studies was evaluated utilizing a risk assessment tool for predictive models. Models were then subjected to subgroup analysis based on the modeling variables.

RESULTS

A total of 12 studies, encompassing 11,912 patients with GC, satisfied the predefined inclusion criteria. ML models established in these studies were primarily based on pathological images, clinical features, and radiomics. The results suggested that in the validation sets, the pathological image-based models had a synthesized c-index of 0.86 [95 % CI (0.83-0.89)], with sensitivity and specificity being 0.86 [95 % CI (0.76-0.92)] and 0.83 [95 % CI (0.78-0.87)], respectively; radiomics feature-based models achieved respective values of 0.87 [95 % CI (0.81-0.92)], 0.77 [95 % CI (0.70-0.83)] and 0.81 [95 % CI (0.74-0.87)]; radiomics feature-based models + clinical feature-based models achieved respective values of 0.87 [95 % CI (0.81-0.93)], 0.78 [95 % CI (0.70-0.84)] and 0.79 [95 % CI (0.69-0.86)].

CONCLUSIONS

ML has demonstrated optimal performance in detecting MSI in GC and could serve as a prospective early adjunctive detection tool for MSI in GC. Future research should contemplate minimally invasive or non-invasive, readily collectible, and efficient predictors to augment the predictive accuracy of ML.

Diagnostic utility of transfer learning by using convolutional neural network for cytological diagnosis of malignant effusions

INTRODUCTION

Cytological analysis of effusion specimens provides critical information regarding the diagnosis and staging of malignancies, thus guiding their treatment and subsequent monitoring. Keeping in view the challenges encountered in the morphological interpretation, we explored convolutional neural networks (CNNs) as an important tool for the cytological diagnosis of malignant effusions.

MATERIALS AND METHODS

A retrospective review of patients at our institute, over 3.5 years yielded a dataset of 342 effusion samples and 518 images with known diagnoses. Cytological examination and cell block preparation were performed to establish correlation with the gold standard, histopathology. We developed a deep learning model using PyTorch, fine-tuned it on a labelled dataset, and evaluated its diagnostic performance using test samples.

RESULTS

The model exhibited encouraging results in the distinction of benign and malignant effusions with area under curve (AUC) of 0.8674, F-measure or F1 score which denotes the harmonic mean of precision and recall, to be 0.8678 thus, demonstrating optimal accuracy of our CNN model.

CONCLUSION

The study highlights the promising potential of transfer learning in enhancing the clinical pathology laboratory efficiency when dealing with malignant effusions.

Applications of artificial intelligence in digital pathology for gastric cancer

Gastric cancer is one of the most common cancers and is one of the leading causes of cancer-related deaths in worldwide. Early diagnosis and treatment are essential for a positive outcome. The integration of artificial intelligence in the pathology field is increasingly widespread, including histopathological images analysis. In recent years, the application of digital pathology technology emerged as a potential solution to enhance the understanding and management of gastric cancer. Through sophisticated image analysis algorithms, artificial intelligence technologies facilitate the accuracy and sensitivity of gastric cancer diagnosis and treatment and personalized therapeutic strategies. This review aims to evaluate the current landscape and future potential of artificial intelligence in transforming gastric cancer pathology, so as to provide ideas for future research.

Immune subtyping of melanoma whole slide images using multiple instance learning

Determining early-stage prognostic markers and stratifying patients for effective treatment are two key challenges for improving outcomes for melanoma patients. Previous studies have used tumour transcriptome data to stratify patients into immune subgroups, which were associated with differential melanoma specific survival and potential predictive biomarkers. However, acquiring transcriptome data is a time-consuming and costly process. Moreover, it is not routinely used in the current clinical workflow. Here, we attempt to overcome this by developing deep learning models to classify gigapixel haematoxylin and eosin (H&E) stained pathology slides, which are well established in clinical workflows, into these immune subgroups. We systematically assess six different multiple instance learning (MIL) frameworks, using five different image resolutions and three different feature extraction methods. We show that pathology-specific self-supervised models using 10x resolution patches generate superior representations for the classification of immune subtypes. In addition, in a primary melanoma dataset, we achieve a mean area under the receiver operating characteristic curve (AUC) of 0.80 for classifying histopathology images into 'high' or 'low immune' subgroups and a mean AUC of 0.82 in an independent TCGA melanoma dataset. Furthermore, we show that these models are able to stratify patients into 'high' and 'low immune' subgroups with significantly different melanoma specific survival outcomes (log rank test, P< 0.005). We anticipate that MIL methods will allow us to find new biomarkers of high importance, act as a tool for clinicians to infer the immune landscape of tumours and stratify patients, without needing to carry out additional expensive genetic tests.

Artificial-Intelligence-Assisted Detection of Metastatic Colorectal Cancer Cells in Ascitic Fluid

Ascites cytology is a cost-effective test for metastatic colorectal cancer (CRC) in the abdominal cavity. However, metastatic carcinoma of the peritoneum is difficult to diagnose based on biopsy findings, and ascitic aspiration cytology has a low sensitivity and specificity and a high inter-observer variability. The aim of the present study was to apply artificial intelligence (AI) to classify benign and malignant cells in ascites cytology patch images of metastatic CRC using a deep convolutional neural network. Datasets were collected from The OPEN AI Dataset Project, a nationwide cytology dataset for AI research. The numbers of patch images used for training, validation, and testing were 56,560, 7068, and 6534, respectively. We evaluated 1041 patch images of benign and metastatic CRC in the ascitic fluid to compare the performance of pathologists and an AI algorithm, and to examine whether the diagnostic accuracy of pathologists improved with the assistance of AI. This AI method showed an accuracy, a sensitivity, and a specificity of 93.74%, 87.76%, and 99.75%, respectively, for the differential diagnosis of malignant and benign ascites. The diagnostic accuracy and sensitivity of the pathologist with the assistance of the proposed AI method increased from 86.8% to 90.5% and from 73.3% to 79.3%, respectively. The proposed deep learning method may assist pathologists with different levels of experience in diagnosing metastatic CRC cells of ascites.

Diagnostic Challenges during Inflammation and Cancer: Current Biomarkers and Future Perspectives in Navigating through the Minefield of Reactive versus Dysplastic and Cancerous Lesions in the Digestive System

In the setting of pronounced inflammation, changes in the epithelium may overlap with neoplasia, often rendering it impossible to establish a diagnosis with certainty in daily clinical practice. Here, we discuss the underlying molecular mechanisms driving tissue response during persistent inflammatory signaling along with the potential association with cancer in the gastrointestinal tract, pancreas, extrahepatic bile ducts, and liver. We highlight the histopathological challenges encountered in the diagnosis of chronic inflammation in routine practice and pinpoint tissue-based biomarkers that could complement morphology to differentiate reactive from dysplastic or cancerous lesions. We refer to the advantages and limitations of existing biomarkers employing immunohistochemistry and point to promising new markers, including the generation of novel antibodies targeting mutant proteins, miRNAs, and array assays. Advancements in experimental models, including mouse and 3D models, have improved our understanding of tissue response. The integration of digital pathology along with artificial intelligence may also complement routine visual inspections. Navigating through tissue responses in various chronic inflammatory contexts will help us develop novel and reliable biomarkers that will improve diagnostic decisions and ultimately patient treatment.

Mismatch Repair Deficiency as a Predictive and Prognostic Biomarker in Endometrial Cancer: A Review on Immunohistochemistry Staining Patterns and Clinical Implications

Among the four endometrial cancer (EC) TCGA molecular groups, the MSI/hypermutated group represents an important percentage of tumors (30%), including different histotypes, and generally confers an intermediate prognosis for affected women, also providing new immunotherapeutic strategies. Immunohistochemistry for MMR proteins (MLH1, MSH2, MSH6 and PMS2) has become the optimal diagnostic MSI surrogate worldwide. This review aims to provide state-of-the-art knowledge on MMR deficiency/MSI in EC and to clarify the pathological assessment, interpretation pitfalls and reporting of MMR status.

Building Automation Pipeline for Diagnostic Classification of Sporadic Odontogenic Keratocysts and Non-Keratocysts Using Whole-Slide Images

The microscopic diagnostic differentiation of odontogenic cysts from other cysts is intricate and may cause perplexity for both clinicians and pathologists. Of particular interest is the odontogenic keratocyst (OKC), a developmental cyst with unique histopathological and clinical characteristics. Nevertheless, what distinguishes this cyst is its aggressive nature and high tendency for recurrence. Clinicians encounter challenges in dealing with this frequently encountered jaw lesion, as there is no consensus on surgical treatment. Therefore, the accurate and early diagnosis of such cysts will benefit clinicians in terms of treatment management and spare subjects from the mental agony of suffering from aggressive OKCs, which impact their quality of life. The objective of this research is to develop an automated OKC diagnostic system that can function as a decision support tool for pathologists, whether they are working locally or remotely. This system will provide them with additional data and insights to enhance their decision-making abilities. This research aims to provide an automation pipeline to classify whole-slide images of OKCs and non-keratocysts (non-KCs: dentigerous and radicular cysts). OKC diagnosis and prognosis using the histopathological analysis of tissues using whole-slide images (WSIs) with a deep-learning approach is an emerging research area. WSIs have the unique advantage of magnifying tissues with high resolution without losing information. The contribution of this research is a novel, deep-learning-based, and efficient algorithm that reduces the trainable parameters and, in turn, the memory footprint. This is achieved using principal component analysis (PCA) and the ReliefF feature selection algorithm (ReliefF) in a convolutional neural network (CNN) named P-C-ReliefF. The proposed model reduces the trainable parameters compared to standard CNN, achieving 97% classification accuracy.

Concordance between microsatellite instability testing and immunohistochemistry for mismatch repair proteins and efficient screening of mismatch repair deficient gastric cancer

Microsatellite instability (MSI) testing, an established technique that has gained prominence in recent years for its predictive potential regarding the efficacy of immune checkpoint inhibitors, is used to evaluate DNA mismatch repair (MMR) deficiency (dMMR). As with other methods, the immunohistochemistry (IHC) of MMR proteins is also widely adopted. Although both techniques have been validated, their concordance rate remains unknown, particularly regarding non-colorectal cancer. Therefore, the aim of the present study was to explore and elucidate their concordance in the context of gastric cancer (GC). A total of 489 surgically resected primary GC tissues were analyzed to compare the results yielded by the MSI test and those from IHC. Of 488 GC cases, 56 (11.5%) exhibited a loss of MMR proteins, whereas 52 (10.7%) were classified as high-frequency MSI (MSI-H). The concordance rate between these two categories was 99.2%. The microsatellite markers BAT26 and MONO27 demonstrated 100% sensitivity and 99.5% specificity in detecting dMMR GC. In addition, histopathological analysis revealed that MSI-H was more prevalent in GCs exhibiting coexisting Tub2 and Por1 subtypes. However, four discordant cases were observed. All four cases were microsatellite-stable cases but exhibited loss of MLH1 protein expression with hypermethylation of the MLH1 promoter. The results of the present study highlight that while there is a strong concordance between MSI and IHC testing results for determining dMMR status, IHC testing may offer superior efficacy in detecting dMMR.

A pan-cancer analysis of STAT3 expression and genetic alterations in human tumors

Combined cancer immunotherapy and targeted therapy have proven to be effective against various cancers and therefore have recently become the focus of cancer research. Signal transducer and activator of transcription 3 (STAT3) is a member of the STAT protein family of transcription factors. Several studies have shown that STAT3 can affect the prognosis of cancer patients by regulating immune microenvironment (IME). Therefore, STAT3 may have high research value for the development of combined immunotherapy/targeted therapy approaches for the treatment of cancer patients. We found differences in STAT3 expression between tumor and normal tissues. Kaplan-Meier survival and Cox regression analyses showed that high expression of STAT3 is associated with poor prognosis in low-grade glioma (LGG) patients. The results of the analysis of the area under the curve of the receiver operating characteristic curve further suggested that the expression of STAT3 is an effective way to evaluate the prognosis of patients with glioma. The results of the IME analysis revealed that the immune and matrix scores of LGGs were positively correlated with the expression of STAT3 (P < 0.05). The results of immune cell infiltration analysis showed that STAT3 was positively correlated with resting dendritic cells, eosinophils, neutrophils, M0 macrophages, M1 macrophages, CD4 memory resting T cells, and CD8 T cells in LGG patients, but negatively correlated with activated mast cells and M2 macrophages (P < 0.05). Our gene set enrichment analysis identified 384 enriched pathways. According to the enrichment scores, the top ten most significantly upregulated pathways were related to immune response. The top ten most significantly downregulated pathways were related to cell signal transduction and the regulation of cell survival, proliferation, and metabolism. Genetic alteration analysis showed that missense mutations in STAT3 account for the majority of mutations, and STAT3 mutations mostly occur in the Src homology domain. In conclusion overexpression of STAT3 can promote the development and growth of tumors by regulating IME, which is significantly related to the poor prognosis of cancer patients. Therefore, targeted inhibition of STAT3 expression may have high research value for the development of combined immunotherapy/targeted therapy approaches for the treatment of cancer patients.

ChampKit: A framework for rapid evaluation of deep neural networks for patch-based histopathology classification

BACKGROUND AND OBJECTIVE

Histopathology is the gold standard for diagnosis of many cancers. Recent advances in computer vision, specifically deep learning, have facilitated the analysis of histopathology images for many tasks, including the detection of immune cells and microsatellite instability. However, it remains difficult to identify optimal models and training configurations for different histopathology classification tasks due to the abundance of available architectures and the lack of systematic evaluations. Our objective in this work is to present a software tool that addresses this need and enables robust, systematic evaluation of neural network models for patch classification in histology in a light-weight, easy-to-use package for both algorithm developers and biomedical researchers.

METHODS

Here we present ChampKit (Comprehensive Histopathology Assessment of Model Predictions toolKit): an extensible, fully reproducible evaluation toolkit that is a one-stop-shop to train and evaluate deep neural networks for patch classification. ChampKit curates a broad range of public datasets. It enables training and evaluation of models supported by timm directly from the command line, without the need for users to write any code. External models are enabled through a straightforward API and minimal coding. As a result, Champkit facilitates the evaluation of existing and new models and deep learning architectures on pathology datasets, making it more accessible to the broader scientific community. To demonstrate the utility of ChampKit, we establish baseline performance for a subset of possible models that could be employed with ChampKit, focusing on several popular deep learning models, namely ResNet18, ResNet50, and R26-ViT, a hybrid vision transformer. In addition, we compare each model trained either from random weight initialization or with transfer learning from ImageNet pretrained models. For ResNet18, we also consider transfer learning from a self-supervised pretrained model.

RESULTS

The main result of this paper is the ChampKit software. Using ChampKit, we were able to systemically evaluate multiple neural networks across six datasets. We observed mixed results when evaluating the benefits of pretraining versus random intialization, with no clear benefit except in the low data regime, where transfer learning was found to be beneficial. Surprisingly, we found that transfer learning from self-supervised weights rarely improved performance, which is counter to other areas of computer vision.

CONCLUSIONS

Choosing the right model for a given digital pathology dataset is nontrivial. ChampKit provides a valuable tool to fill this gap by enabling the evaluation of hundreds of existing (or user-defined) deep learning models across a variety of pathology tasks. Source code and data for the tool are freely accessible at https://github.com/SBU-BMI/champkit.

Deep Learning-Based Computational Cytopathologic Diagnosis of Metastatic Breast Carcinoma in Pleural Fluid

A Pleural effusion cytology is vital for treating metastatic breast cancer; however, concerns have arisen regarding the low accuracy and inter-observer variability in cytologic diagnosis. Although artificial intelligence-based image analysis has shown promise in cytopathology research, its application in diagnosing breast cancer in pleural fluid remains unexplored. To overcome these limitations, we evaluate the diagnostic accuracy of an artificial intelligence-based model using a large collection of cytopathological slides, to detect the malignant pleural effusion cytology associated with breast cancer. This study includes a total of 569 cytological slides of malignant pleural effusion of metastatic breast cancer from various institutions. We extracted 34,221 augmented image patches from whole-slide images and trained and validated a deep convolutional neural network model (DCNN) (Inception-ResNet-V2) with the images. Using this model, we classified 845 randomly selected patches, which were reviewed by three pathologists to compare their accuracy. The DCNN model outperforms the pathologists by demonstrating higher accuracy, sensitivity, and specificity compared to the pathologists (81.1% vs. 68.7%, 95.0% vs. 72.5%, and 98.6% vs. 88.9%, respectively). The pathologists reviewed the discordant cases of DCNN. After re-examination, the average accuracy, sensitivity, and specificity of the pathologists improved to 87.9, 80.2, and 95.7%, respectively. This study shows that DCNN can accurately diagnose malignant pleural effusion cytology in breast cancer and has the potential to support pathologists.

Introducing JMIR Bioinformatics and Biotechnology: A Platform for Interdisciplinary Collaboration and Cutting-Edge Research

JMIR Bioinformatics and Biotechnology supports interdisciplinary research and welcomes contributions that push the boundaries of bioinformatics, genomics, artificial intelligence, and pathology informatics.

Recent application of artificial intelligence on histopathologic image-based prediction of gene mutation in solid cancers

PURPOSE

Evaluation of genetic mutations in cancers is important because distinct mutational profiles help determine individualized drug therapy. However, molecular analyses are not routinely performed in all cancers because they are expensive, time-consuming and not universally available. Artificial intelligence (AI) has shown the potential to determine a wide range of genetic mutations on histologic image analysis. Here, we assessed the status of mutation prediction AI models on histologic images by a systematic review.

METHODS

A literature search using the MEDLINE, Embase and Cochrane databases was conducted in August 2021. The articles were shortlisted by titles and abstracts. After a full-text review, publication trends, study characteristic analysis and comparison of performance metrics were performed.

RESULTS

Twenty-four studies were found mostly from developed countries, and their number is increasing. The major targets were gastrointestinal, genitourinary, gynecological, lung and head and neck cancers. Most studies used the Cancer Genome Atlas, with a few using an in-house dataset. The area under the curve of some of the cancer driver gene mutations in particular organs was satisfactory, such as 0.92 of BRAF in thyroid cancers and 0.79 of EGFR in lung cancers, whereas the average of all gene mutations was 0.64, which is still suboptimal.

CONCLUSION

AI has the potential to predict gene mutations on histologic images with appropriate caution. Further validation with larger datasets is still required before AI models can be used in clinical practice to predict gene mutations.

Deep Learning-Based Classification and Targeted Gene Alteration Prediction from Pleural Effusion Cell Block Whole-Slide Images

Cytopathological examination is one of the main examinations for pleural effusion, and especially for many patients with advanced cancer, pleural effusion is the only accessible specimen for establishing a pathological diagnosis. The lack of cytopathologists and the high cost of gene detection present opportunities for the application of deep learning. In this retrospective analysis, data representing 1321 consecutive cases of pleural effusion were collected. We trained and evaluated our deep learning model based on several tasks, including the diagnosis of benign and malignant pleural effusion, the identification of the primary location of common metastatic cancer from pleural effusion, and the prediction of genetic alterations associated with targeted therapy. We achieved good results in identifying benign and malignant pleural effusions (0.932 AUC (area under the ROC curve)) and the primary location of common metastatic cancer (0.910 AUC). In addition, we analyzed ten genes related to targeted therapy in specimens and used them to train the model regarding four alteration statuses, which also yielded reasonable results (0.869 AUC for ALK fusion, 0.804 AUC for KRAS mutation, 0.644 AUC for EGFR mutation and 0.774 AUC for NONE alteration). Our research shows the feasibility and benefits of deep learning to assist in cytopathological diagnosis in clinical settings.

Deep Learning-Based Prediction of Molecular Tumor Biomarkers from H&E: A Practical Review

Molecular and genomic properties are critical in selecting cancer treatments to target individual tumors, particularly for immunotherapy. However, the methods to assess such properties are expensive, time-consuming, and often not routinely performed. Applying machine learning to H&E images can provide a more cost-effective screening method. Dozens of studies over the last few years have demonstrated that a variety of molecular biomarkers can be predicted from H&E alone using the advancements of deep learning: molecular alterations, genomic subtypes, protein biomarkers, and even the presence of viruses. This article reviews the diverse applications across cancer types and the methodology to train and validate these models on whole slide images. From bottom-up to pathologist-driven to hybrid approaches, the leading trends include a variety of weakly supervised deep learning-based approaches, as well as mechanisms for training strongly supervised models in select situations. While results of these algorithms look promising, some challenges still persist, including small training sets, rigorous validation, and model explainability. Biomarker prediction models may yield a screening method to determine when to run molecular tests or an alternative when molecular tests are not possible. They also create new opportunities in quantifying intratumoral heterogeneity and predicting patient outcomes.

Development of quality assurance program for digital pathology by the Korean Society of Pathologists

Background: Digital pathology (DP) using whole slide imaging is a recently emerging game changer technology that can fundamentally change the way of working in pathology. The Digital Pathology Study Group (DPSG) of the Korean Society of Pathologists (KSP) published a consensus report on the recommendations for pathologic practice using DP. Accordingly, the need for the development and implementation of a quality assurance program (QAP) for DP has been raised.Methods: To provide a standard baseline reference for internal and external QAP for DP, the members of the Committee of Quality Assurance of the KSP developed a checklist for the Redbook and a QAP trial for DP based on the prior DPSG consensus report. Four leading institutes participated in the QAP trial in the first year, and we gathered feedback from these institutes afterwards.Results: The newly developed checklists of QAP for DP contain 39 items (216 score): eight items for quality control of DP systems; three for DP personnel; nine for hardware and software requirements for DP systems; 15 for validation, operation, and management of DP systems; and four for data security and personal information protection. Most participants in the QAP trial replied that continuous education on unfamiliar terminology and more practical experience is demanding.Conclusions: The QAP for DP is essential for the safe implementation of DP in pathologic practice. Each laboratory should prepare an institutional QAP according to this checklist, and consecutive revision of the checklist with feedback from the QAP trial for DP needs to follow.

Recent Application of Artificial Intelligence in Non-Gynecological Cancer Cytopathology: A Systematic Review

Simple Summary

Artificial intelligence (AI) has attracted significant interest in the healthcare sector due to its promising results. Cytological examination is a critical step in the initial diagnosis of cancer. Here, we conducted a systematic review with quantitative analysis to understand the current status of AI applications in non-gynecological (non-GYN) cancer cytology. In our analysis, we found that most of the studies focused on classification and segmentation tasks. Overall, AI showed promising results for non-GYN cancer cytopathology analysis. However, the lack of well-annotated, large-scale datasets with Z-stacking and external cross-validation was the major limitation across all studies.

Abstract

State-of-the-art artificial intelligence (AI) has recently gained considerable interest in the healthcare sector and has provided solutions to problems through automated diagnosis. Cytological examination is a crucial step in the initial diagnosis of cancer, although it shows limited diagnostic efficacy. Recently, AI applications in the processing of cytopathological images have shown promising results despite the elementary level of the technology. Here, we performed a systematic review with a quantitative analysis of recent AI applications in non-gynecological (non-GYN) cancer cytology to understand the current technical status. We searched the major online databases, including MEDLINE, Cochrane Library, and EMBASE, for relevant English articles published from January 2010 to January 2021. The searched query terms were: “artificial intelligence”, “image processing”, “deep learning”, “cytopathology”, and “fine-needle aspiration cytology.” Out of 17,000 studies, only 26 studies (26 models) were included in the full-text review, whereas 13 studies were included for quantitative analysis. There were eight classes of AI models treated of according to target organs: thyroid (n = 11, 39%), urinary bladder (n = 6, 21%), lung (n = 4, 14%), breast (n = 2, 7%), pleural effusion (n = 2, 7%), ovary (n = 1, 4%), pancreas (n = 1, 4%), and prostate (n = 1, 4). Most of the studies focused on classification and segmentation tasks. Although most of the studies showed impressive results, the sizes of the training and validation datasets were limited. Overall, AI is also promising for non-GYN cancer cytopathology analysis, such as pathology or gynecological cytology. However, the lack of well-annotated, large-scale datasets with Z-stacking and external cross-validation was the major limitation found across all studies. Future studies with larger datasets with high-quality annotations and external validation are required.