A narrative review of digital pathology and artificial intelligence: focusing on lung cancer

Computational pathology in the identification of HER2-low breast cancer: Opportunities and challenges

For the past 2 decades, pathologists have been accustomed to reporting the HER2 status of breast cancer as either positive or negative, based on HER2 IHC. Today, however, there is a clinical imperative to employ a 3-tier approach to interpreting HER2 IHC that can also identify tumours categorised as HER2-low. Meeting this need for a finer degree of discrimination may be challenging, and in this article, we consider the potential for the integration of computational approaches to support pathologists in achieving accurate and reproducible HER2 IHC scoring as well as outlining some of the practicalities involved.

Computational pathology: A survey review and the way forward

Computational Pathology (CPath) is an interdisciplinary science that augments developments of computational approaches to analyze and model medical histopathology images. The main objective for CPath is to develop infrastructure and workflows of digital diagnostics as an assistive CAD system for clinical pathology, facilitating transformational changes in the diagnosis and treatment of cancer that are mainly address by CPath tools. With evergrowing developments in deep learning and computer vision algorithms, and the ease of the data flow from digital pathology, currently CPath is witnessing a paradigm shift. Despite the sheer volume of engineering and scientific works being introduced for cancer image analysis, there is still a considerable gap of adopting and integrating these algorithms in clinical practice. This raises a significant question regarding the direction and trends that are undertaken in CPath. In this article we provide a comprehensive review of more than 800 papers to address the challenges faced in problem design all-the-way to the application and implementation viewpoints. We have catalogued each paper into a model-card by examining the key works and challenges faced to layout the current landscape in CPath. We hope this helps the community to locate relevant works and facilitate understanding of the field's future directions. In a nutshell, we oversee the CPath developments in cycle of stages which are required to be cohesively linked together to address the challenges associated with such multidisciplinary science. We overview this cycle from different perspectives of data-centric, model-centric, and application-centric problems. We finally sketch remaining challenges and provide directions for future technical developments and clinical integration of CPath. For updated information on this survey review paper and accessing to the original model cards repository, please refer to GitHub. Updated version of this draft can also be found from arXiv.

Artificial Intelligence and Lung Pathology

This manuscript provides a comprehensive overview of the application of artificial intelligence (AI) in lung pathology, particularly in the diagnosis of lung cancer. It discusses various AI models designed to support pathologists and clinicians. AI models supporting pathologists are to standardize diagnosis, score PD-L1 status, supporting tumor cellularity count, and indicating explainability for pathologic judgements. Several models predict outcomes beyond pathologic diagnosis and predict clinical outcomes like patients' survival and molecular alterations. The manuscript emphasizes the potential of AI to enhance accuracy and efficiency in pathology, while also addressing the challenges and future directions for integrating AI into clinical practice.

Artificial intelligence in digital histopathology for predicting patient prognosis and treatment efficacy in breast cancer

INTRODUCTION

Histological images contain phenotypic information predictive of patient outcomes. Due to the heavy workload of pathologists, the time-consuming nature of quantitatively assessing histological features, and human eye limitations to recognize spatial patterns, manually extracting prognostic information in routine pathological workflows remains challenging. Digital pathology has facilitated the mining and quantification of these features utilizing whole-slide image (WSI) scanners and artificial intelligence (AI) algorithms. AI algorithms to identify image-based biomarkers from the tumor microenvironment (TME) have the potential to revolutionize the field of oncology, reducing delays between diagnosis and prognosis determination, allowing for rapid stratification of patients and prescription of optimal treatment regimes, thereby improving patient outcomes.

AREAS COVERED

In this review, the authors discuss how AI algorithms and digital pathology can predict breast cancer patient prognosis and treatment outcomes using image-based biomarkers, along with the challenges of adopting this technology in clinical settings.

EXPERT OPINION

The integration of AI and digital pathology presents significant potential for analyzing the TME and its diagnostic, prognostic, and predictive value in breast cancer patients. Widespread clinical adoption of AI faces ethical, regulatory, and technical challenges, although prospective trials may offer reassurance and promote uptake, ultimately improving patient outcomes by reducing diagnosis-to-prognosis delivery delays.

Understanding the errors made by artificial intelligence algorithms in histopathology in terms of patient impact

An increasing number of artificial intelligence (AI) tools are moving towards the clinical realm in histopathology and across medicine. The introduction of such tools will bring several benefits to diagnostic specialities, namely increased diagnostic accuracy and efficiency, however, as no AI tool is infallible, their use will inevitably introduce novel errors. These errors made by AI tools are, most fundamentally, misclassifications made by a computational algorithm. Understanding of how these translate into clinical impact on patients is often lacking, meaning true reporting of AI tool safety is incomplete. In this Perspective we consider AI diagnostic tools in histopathology, which are predominantly assessed in terms of technical performance metrics such as sensitivity, specificity and area under the receiver operating characteristic curve. Although these metrics are essential and allow tool comparison, they alone give an incomplete picture of how an AI tool's errors could impact a patient's diagnosis, management and prognosis. We instead suggest assessing and reporting AI tool errors from a pathological and clinical stance, demonstrating how this is done in studies on human pathologist errors, and giving examples where available from pathology and radiology. Although this seems a significant task, we discuss ways to move towards this approach in terms of study design, guidelines and regulation. This Perspective seeks to initiate broader consideration of the assessment of AI tool errors in histopathology and across diagnostic specialities, in an attempt to keep patient safety at the forefront of AI tool development and facilitate safe clinical deployment.

A new model using deep learning to predict recurrence after surgical resection of lung adenocarcinoma

This study aimed to develop a deep learning (DL) model for predicting the recurrence risk of lung adenocarcinoma (LUAD) based on its histopathological features. Clinicopathological data and whole slide images from 164 LUAD cases were collected and used to train DL models with an ImageNet pre-trained efficientnet-b2 architecture, densenet201, and resnet152. The models were trained to classify each image patch into high-risk or low-risk groups, and the case-level result was determined by multiple instance learning with final FC layer's features from a model from all patches. Analysis of the clinicopathological and genetic characteristics of the model-based risk group was performed. For predicting recurrence, the model had an area under the curve score of 0.763 with 0.750, 0.633 and 0.680 of sensitivity, specificity, and accuracy in the test set, respectively. High-risk cases for recurrence predicted by the model (HR group) were significantly associated with shorter recurrence-free survival and a higher stage (both, p < 0.001). The HR group was associated with specific histopathological features such as poorly differentiated components, complex glandular pattern components, tumor spread through air spaces, and a higher grade. In the HR group, pleural invasion, necrosis, and lymphatic invasion were more frequent, and the size of the invasion was larger (all, p < 0.001). Several genetic mutations, including TP53 (p = 0.007) mutations, were more frequently found in the HR group. The results of stages I-II were similar to those of the general cohort. DL-based model can predict the recurrence risk of LUAD and identify the presence of the TP53 gene mutation by analyzing histopathologic features.

Why do errors arise in artificial intelligence diagnostic tools in histopathology and how can we minimize them?

Artificial intelligence (AI)-based diagnostic tools can offer numerous benefits to the field of histopathology, including improved diagnostic accuracy, efficiency and productivity. As a result, such tools are likely to have an increasing role in routine practice. However, all AI tools are prone to errors, and these AI-associated errors have been identified as a major risk in the introduction of AI into healthcare. The errors made by AI tools are different, in terms of both cause and nature, to the errors made by human pathologists. As highlighted by the National Institute for Health and Care Excellence, it is imperative that practising pathologists understand the potential limitations of AI tools, including the errors made. Pathologists are in a unique position to be gatekeepers of AI tool use, maximizing patient benefit while minimizing harm. Furthermore, their pathological knowledge is essential to understanding when, and why, errors have occurred and so to developing safer future algorithms. This paper summarises the literature on errors made by AI diagnostic tools in histopathology. These include erroneous errors, data concerns (data bias, hidden stratification, data imbalances, distributional shift, and lack of generalisability), reinforcement of outdated practices, unsafe failure mode, automation bias, and insensitivity to impact. Methods to reduce errors in both tool design and clinical use are discussed, and the practical roles for pathologists in error minimisation are highlighted. This aims to inform and empower pathologists to move safely through this seismic change in practice and help ensure that novel AI tools are adopted safely.

Standardized Classification of Lung Adenocarcinoma Subtypes and Improvement of Grading Assessment Through Deep Learning

The histopathologic distinction of lung adenocarcinoma (LADC) subtypes is subject to high interobserver variability, which can compromise the optimal assessment of patient prognosis. Therefore, this study developed convolutional neural networks capable of distinguishing LADC subtypes and predicting disease-specific survival, according to the recently established LADC tumor grades. Consensus LADC histopathologic images were obtained from 17 expert pulmonary pathologists and one pathologist in training. Two deep learning models (AI-1 and AI-2) were trained to predict eight different LADC classes. Furthermore, the trained models were tested on an independent cohort of 133 patients. The models achieved high precision, recall, and F1 scores exceeding 0.90 for most of the LADC classes. Clear stratification of the three LADC grades was reached in predicting the disease-specific survival by the two models, with both Kaplan-Meier curves showing significance (P = 0.0017 and 0.0003). Moreover, both trained models showed high stability in the segmentation of each pair of predicted grades with low variation in the hazard ratio across 200 bootstrapped samples. These findings indicate that the trained convolutional neural networks improve the diagnostic accuracy of the pathologist and refine LADC grade assessment. Thus, the trained models are promising tools that may assist in the routine evaluation of LADC subtypes and grades in clinical practice.

Real-World Implementation of Digital Pathology: Results From an Intercontinental Survey

The past 70 years have been characterized by rapid advancements in computer technology, and the health care system has not been immune to this trend. However, anatomical pathology has remained largely an analog discipline. In recent years, this has been changing with the growing adoption of digital pathology, partly driven by the potential of computer-aided diagnosis. As part of an international collaboration, we conducted a comprehensive survey to gain a deeper understanding of the status of digital pathology implementation in Europe and Asia. A total of 127 anatomical pathology laboratories participated in the survey, including 75 from Europe and 52 from Asia, with 72 laboratories having established digital pathology workflow and 55 without digital pathology. Laboratories using digital pathology for diagnostic (n = 29) and nondiagnostic (n = 43) purposes were thoroughly questioned about their implementation strategies and institutional experiences, including details on equipment, storage, integration with laboratory information system, computer-aided diagnosis, and the costs of going digital. The impact of the digital pathology workflow was also evaluated, focusing on turnaround time, specimen traceability, quality control, and overall satisfaction. Laboratories without access to digital pathology were asked to provide insights into their perceptions of the technology, expectations, barriers to adoption, and potential facilitators. Our findings indicate that although digital pathology is still the future for many, it is already the present for some. This decade may be a time when anatomical pathology finally embraces digital revolution on a larger scale.

Comparison between vision transformers and convolutional neural networks to predict non-small lung cancer recurrence

Non-Small cell lung cancer (NSCLC) is one of the most dangerous cancers, with 85% of all new lung cancer diagnoses and a 30-55% of recurrence rate after surgery. Thus, an accurate prediction of recurrence risk in NSCLC patients during diagnosis could be essential to drive targeted therapies preventing either overtreatment or undertreatment of cancer patients. The radiomic analysis of CT images has already shown great potential in solving this task; specifically, Convolutional Neural Networks (CNNs) have already been proposed providing good performances. Recently, Vision Transformers (ViTs) have been introduced, reaching comparable and even better performances than traditional CNNs in image classification. The aim of the proposed paper was to compare the performances of different state-of-the-art deep learning algorithms to predict cancer recurrence in NSCLC patients. In this work, using a public database of 144 patients, we implemented a transfer learning approach, involving different Transformers architectures like pre-trained ViTs, pre-trained Pyramid Vision Transformers, and pre-trained Swin Transformers to predict the recurrence of NSCLC patients from CT images, comparing their performances with state-of-the-art CNNs. Although, the best performances in this study are reached via CNNs with AUC, Accuracy, Sensitivity, Specificity, and Precision equal to 0.91, 0.89, 0.85, 0.90, and 0.78, respectively, Transformer architectures reach comparable ones with AUC, Accuracy, Sensitivity, Specificity, and Precision equal to 0.90, 0.86, 0.81, 0.89, and 0.75, respectively. Based on our preliminary experimental results, it appears that Transformers architectures do not add improvements in terms of predictive performance to the addressed problem.

A Systematic Review of the Barriers to the Implementation of Artificial Intelligence in Healthcare

Artificial intelligence (AI) is expected to improve healthcare outcomes by facilitating early diagnosis, reducing the medical administrative burden, aiding drug development, personalising medical and oncological management, monitoring healthcare parameters on an individual basis, and allowing clinicians to spend more time with their patients. In the post-pandemic world where there is a drive for efficient delivery of healthcare and manage long waiting times for patients to access care, AI has an important role in supporting clinicians and healthcare systems to streamline the care pathways and provide timely and high-quality care for the patients. Despite AI technologies being used in healthcare for some decades, and all the theoretical potential of AI, the uptake in healthcare has been uneven and slower than anticipated and there remain a number of barriers, both overt and covert, which have limited its incorporation. This literature review highlighted barriers in six key areas: ethical, technological, liability and regulatory, workforce, social, and patient safety barriers. Defining and understanding the barriers preventing the acceptance and implementation of AI in the setting of healthcare will enable clinical staff and healthcare leaders to overcome the identified hurdles and incorporate AI technologies for the benefit of patients and clinical staff.

Integrating cytology into routine digital pathology workflow: a 5-year journey

Despite recent advances in digital imaging, the adoption of digital cytology is challenging due to technical limitations. This study describes our 5-year institutional experience with the implementation of digital cytology. The routine cytology workflow included conventional two-step screening by cytotechnologists, followed by sign out by pathologists. We introduced sign out of cytologic cases using a microscopic digital imaging platform operated by cytotechnologists, which allowed for remote review of slides by cytopathologists via video streaming. We also provided cytologic correlation to support the virtual slide-based sign out of histopathological specimens and for a weekly pathology-radiology conference. In addition, positive cytology cases were archived for integration into the laboratory information system and for prospective computational pathology studies. We also summarized lessons learned over the years and outlined our vision for future developments. This unique experience may serve as a role model for other institutions.

ChampKit: A framework for rapid evaluation of deep neural networks for patch-based histopathology classification

BACKGROUND AND OBJECTIVE

Histopathology is the gold standard for diagnosis of many cancers. Recent advances in computer vision, specifically deep learning, have facilitated the analysis of histopathology images for many tasks, including the detection of immune cells and microsatellite instability. However, it remains difficult to identify optimal models and training configurations for different histopathology classification tasks due to the abundance of available architectures and the lack of systematic evaluations. Our objective in this work is to present a software tool that addresses this need and enables robust, systematic evaluation of neural network models for patch classification in histology in a light-weight, easy-to-use package for both algorithm developers and biomedical researchers.

METHODS

Here we present ChampKit (Comprehensive Histopathology Assessment of Model Predictions toolKit): an extensible, fully reproducible evaluation toolkit that is a one-stop-shop to train and evaluate deep neural networks for patch classification. ChampKit curates a broad range of public datasets. It enables training and evaluation of models supported by timm directly from the command line, without the need for users to write any code. External models are enabled through a straightforward API and minimal coding. As a result, Champkit facilitates the evaluation of existing and new models and deep learning architectures on pathology datasets, making it more accessible to the broader scientific community. To demonstrate the utility of ChampKit, we establish baseline performance for a subset of possible models that could be employed with ChampKit, focusing on several popular deep learning models, namely ResNet18, ResNet50, and R26-ViT, a hybrid vision transformer. In addition, we compare each model trained either from random weight initialization or with transfer learning from ImageNet pretrained models. For ResNet18, we also consider transfer learning from a self-supervised pretrained model.

RESULTS

The main result of this paper is the ChampKit software. Using ChampKit, we were able to systemically evaluate multiple neural networks across six datasets. We observed mixed results when evaluating the benefits of pretraining versus random intialization, with no clear benefit except in the low data regime, where transfer learning was found to be beneficial. Surprisingly, we found that transfer learning from self-supervised weights rarely improved performance, which is counter to other areas of computer vision.

CONCLUSIONS

Choosing the right model for a given digital pathology dataset is nontrivial. ChampKit provides a valuable tool to fill this gap by enabling the evaluation of hundreds of existing (or user-defined) deep learning models across a variety of pathology tasks. Source code and data for the tool are freely accessible at https://github.com/SBU-BMI/champkit.

Clinical importance of the range of detectable variants between the Oncomine Dx target test and a conventional single-gene test for EGFR mutation

Although we have experienced some cases with discordant results between the Oncomine Dx target test (ODxTT) and conventional single gene tests for detecting EGFR alterations, the clinical efficacy of EGFR-TKIs in these discordant cases remains little known. We retrospectively reviewed consecutive patients with non-small-cell lung cancer whose FFPE samples were simultaneously submitted for the ODxTT, and a PNA-LNA PCR clamp test. We evaluated the clinical efficacy of EGFR-TKIs in patients with discordant results between the two tests, focusing on the common EGFR mutations. Among 444 successful results, 10 patients had discordant results for common EGFR mutations (9 Ex 19 deletion and 1 Ex 21 L858R mutation), and all of these were detected only by the PNA-LNA PCR clamp test. Among six discordant cases treated with EGFR-TKI, the mutations detected in 3 patients were not included in the list of detectable variants that are reportable by the ODxTT, while the mutations detected in the other 3 patients were included in the list. For all three discordant cases harboring the mutations not reportable by the ODxTT, good clinical responses were demonstrated. However, among the other three discordant cases harboring the mutations reportable by the ODxTT, only one patient had a clinical response with short duration. Among the discordant cases for common EGFR mutations between the ODxTT and the conventional single gene test, there are a certain number of suitable patients responsive to EGFR-TKIs, especially when the cause of the discordant results comes from the difference in the range of detectable variants that are reportable between the tests.

SMILE: Cost-sensitive multi-task learning for nuclear segmentation and classification with imbalanced annotations

High throughput nuclear segmentation and classification of whole slide images (WSIs) is crucial to biological analysis, clinical diagnosis and precision medicine. With the advances of CNN algorithms and the continuously growing datasets, considerable progress has been made in nuclear segmentation and classification. However, few works consider how to reasonably deal with nuclear heterogeneity in the following two aspects: imbalanced data distribution and diversified morphology characteristics. The minority classes might be dominated by the majority classes due to the imbalanced data distribution and the diversified morphology characteristics may lead to fragile segmentation results. In this study, a cost-Sensitive MultI-task LEarning (SMILE) framework is conducted to tackle the data heterogeneity problem. Based on the most popular multi-task learning backbone in nuclei segmentation and classification, we propose a multi-task correlation attention (MTCA) to perform feature interaction of multiple high relevant tasks to learn better feature representation. A cost-sensitive learning strategy is proposed to solve the imbalanced data distribution by increasing the penalization for the error classification of the minority classes. Furthermore, we propose a novel post-processing step based on the coarse-to-fine marker-controlled watershed scheme to alleviate fragile segmentation when nuclei are with large size and unclear contour. Extensive experiments show that the proposed method achieves state-of-the-art performances on CoNSeP and MoNuSAC 2020 datasets. The code is available at: https://github.com/panxipeng/nuclear_segandcls.

Overcoming the Interobserver Variability in Lung Adenocarcinoma Subtyping: A Clustering Approach to Establish a Ground Truth for Downstream Applications

CONTEXT.—

The accurate identification of different lung adenocarcinoma histologic subtypes is important for determining prognosis but can be challenging because of overlaps in the diagnostic features, leading to considerable interobserver variability.

OBJECTIVE.—

To provide an overview of the diagnostic agreement for lung adenocarcinoma subtypes among pathologists and to create a ground truth using the clustering approach for downstream computational applications.

DESIGN.—

Three sets of lung adenocarcinoma histologic images with different evaluation levels (small patches, areas with relatively uniform histology, and whole slide images) were reviewed by 17 international expert lung pathologists and 1 pathologist in training. Each image was classified into one or several lung adenocarcinoma subtypes.

RESULTS.—

Among the 4702 patches of the first set, 1742 (37%) had an overall consensus among all pathologists. The overall Fleiss κ score for the agreement of all subtypes was 0.58. Using cluster analysis, pathologists were hierarchically grouped into 2 clusters, with κ scores of 0.588 and 0.563 in clusters 1 and 2, respectively. Similar results were obtained for the second and third sets, with fair-to-moderate agreements. Patches from the first 2 sets that obtained the consensus of the 18 pathologists were retrieved to form consensus patches and were regarded as the ground truth of lung adenocarcinoma subtypes.

CONCLUSIONS.—

Our observations highlight discrepancies among experts when assessing lung adenocarcinoma subtypes. However, a subsequent number of consensus patches could be retrieved from each cluster, which can be used as ground truth for the downstream computational pathology applications, with minimal influence from interobserver variability.

Leveraging the Academic Artificial Intelligence Silecosystem to Advance the Community Oncology Enterprise

Over the last 75 years, artificial intelligence has evolved from a theoretical concept and novel paradigm describing the role that computers might play in our society to a tool with which we daily engage. In this review, we describe AI in terms of its constituent elements, the synthesis of which we refer to as the AI Silecosystem. Herein, we provide an historical perspective of the evolution of the AI Silecosystem, conceptualized and summarized as a Kuhnian paradigm. This manuscript focuses on the role that the AI Silecosystem plays in oncology and its emerging importance in the care of the community oncology patient. We observe that this important role arises out of a unique alliance between the academic oncology enterprise and community oncology practices. We provide evidence of this alliance by illustrating the practical establishment of the AI Silecosystem at the City of Hope Comprehensive Cancer Center and its team utilization by community oncology providers.

Artificial Intelligence-Based Opportunities in Liver Pathology-A Systematic Review

BACKGROUND

Artificial Intelligence (AI)-based Deep Neural Networks (DNNs) can handle a wide range of applications in image analysis, ranging from automated segmentation to diagnostic and prediction. As such, they have revolutionized healthcare, including in the liver pathology field.

OBJECTIVE

The present study aims to provide a systematic review of applications and performances provided by DNN algorithms in liver pathology throughout the Pubmed and Embase databases up to December 2022, for tumoral, metabolic and inflammatory fields.

RESULTS

42 articles were selected and fully reviewed. Each article was evaluated through the Quality Assessment of Diagnostic Accuracy Studies (QUADAS-2) tool, highlighting their risks of bias.

CONCLUSIONS

DNN-based models are well represented in the field of liver pathology, and their applications are diverse. Most studies, however, presented at least one domain with a high risk of bias according to the QUADAS-2 tool. Hence, DNN models in liver pathology present future opportunities and persistent limitations. To our knowledge, this review is the first one solely focused on DNN-based applications in liver pathology, and to evaluate their bias through the lens of the QUADAS2 tool.

Deep Learning-Based Modified YOLACT Algorithm on Magnetic Resonance Imaging Images for Screening Common and Difficult Samples of Breast Cancer

Computer-aided methods have been extensively applied for diagnosing breast lesions with magnetic resonance imaging (MRI), but fully-automatic diagnosis using deep learning is rarely documented. Deep-learning-technology-based artificial intelligence (AI) was used in this work to classify and diagnose breast cancer based on MRI images. Breast cancer MRI images from the Rider Breast MRI public dataset were converted into processable joint photographic expert group (JPG) format images. The location and shape of the lesion area were labeled using the Labelme software. A difficult-sample mining mechanism was introduced to improve the performance of the YOLACT algorithm model as a modified YOLACT algorithm model. Diagnostic efficacy was compared with the Mask R-CNN algorithm model. The deep learning framework was based on PyTorch version 1.0. Four thousand and four hundred labeled data with corresponding lesions were labeled as normal samples, and 1600 images with blurred lesion areas as difficult samples. The modified YOLACT algorithm model achieved higher accuracy and better classification performance than the YOLACT model. The detection accuracy of the modified YOLACT algorithm model with the difficult-sample-mining mechanism is improved by nearly 3% for common and difficult sample images. Compared with Mask R-CNN, it is still faster in running speed, and the difference in recognition accuracy is not obvious. The modified YOLACT algorithm had a classification accuracy of 98.5% for the common sample test set and 93.6% for difficult samples. We constructed a modified YOLACT algorithm model, which is superior to the YOLACT algorithm model in diagnosis and classification accuracy.

The Approach of Artificial Intelligence in Neuroendocrine Carcinomas of the Breast: A Next Step towards Precision Pathology?—A Case Report and Review of the Literature

Primary neuroendocrine tumors (NETs) of the breast are considered a rare and undervalued subtype of breast carcinoma that occur mainly in postmenopausal women and are graded as G1 or G2 NETs or an invasive neuroendocrine carcinoma (NEC) (small cell or large cell). To establish a final diagnosis of breast carcinoma with neuroendocrine differentiation, it is essential to perform an immunohistochemical profile of the tumor, using antibodies against synaptophysin or chromogranin, as well as the MIB-1 proliferation index, one of the most controversial markers in breast pathology regarding its methodology in current clinical practice. A standardization error between institutions and pathologists regarding the evaluation of the MIB-1 proliferation index is present. Another challenge refers to the counting process of MIB-1′s expressiveness, which is known as a time-consuming process. The involvement of AI (artificial intelligence) automated systems could be a solution for diagnosing early stages, as well. We present the case of a post-menopausal 79-year-old woman diagnosed with primary neuroendocrine carcinoma of the breast (NECB). The purpose of this paper is to expose the interpretation of MIB-1 expression in our patient’ s case of breast neuroendocrine carcinoma, assisted by artificial intelligence (AI) software (HALO—IndicaLabs), and to analyze the associations between MIB-1 and common histopathological parameters.

Determining HER2 Status by Artificial Intelligence: An Investigation of Primary, Metastatic, and HER2 Low Breast Tumors

The expression of human epidermal growth factor receptor 2 (HER2) protein or gene transcripts is critical for therapeutic decision making in breast cancer. We examined the performance of a digitalized and artificial intelligence (AI)-assisted workflow for HER2 status determination in accordance with the American Society of Clinical Oncology (ASCO)/College of Pathologists (CAP) guidelines. Our preliminary cohort consisted of 495 primary breast carcinomas, and our study cohort included 67 primary breast carcinomas and 30 metastatic deposits, which were evaluated for HER2 status by immunohistochemistry (IHC) and in situ hybridization (ISH). Three practicing breast pathologists independently assessed and scored slides, building the ground truth. Following a washout period, pathologists were provided with the results of the AI digital image analysis (DIA) and asked to reassess the slides. Both rounds of assessment from the pathologists were compared to the AI results and ground truth for each slide. We observed an overall HER2 positivity rate of 15% in our study cohort. Moderate agreement (Cohen's κ 0.59) was observed between the ground truth and AI on IHC, with most discrepancies occurring between 0 and 1+ scores. Inter-observer agreement amongst pathologists was substantial (Fleiss´ κ 0.77) and pathologists' agreement with AI scores was 80.6%. Substantial agreement of the AI with the ground truth (Cohen´s κ 0.80) was detected on ISH-stained slides, and the accuracy of AI was similar for the primary and metastatic tumors. We demonstrated the feasibility of a combined HER2 IHC and ISH AI workflow, with a Cohen's κ of 0.94 when assessed in accordance with the ASCO/CAP recommendations.

Deep learning-based predictions of clear and eosinophilic phenotypes in clear cell renal cell carcinoma

We have recently shown that histological phenotypes focusing on clear and eosinophilic cytoplasm in clear cell renal cell carcinoma (ccRCC) correlated with prognosis and the response to angiogenesis inhibition and checkpoint blockade. This study aims to objectively show the diagnostic utility of clear or eosinophilic phenotypes of ccRCC by developing an artificial intelligence (AI) model using the TCGA-ccRCC dataset and to demonstrate if the clear or eosinophilic predicted phenotypes correlate with pathological factors and gene signatures associated with angiogenesis and cancer immunity. Before the development of the AI model, histological evaluation using hematoxylin and eosin whole-slide images of the TCGA-ccRCC cohort (n = 435) was performed by a urologic pathologist. The AI model was developed as follows. First, the highest-grade area on each whole slide image was captured for image processing. Second, the selected regions were cropped into tiles. Third, the AI model was trained using transfer learning on a deep convolutional neural network, and clear or eosinophilic predictions were scaled as AI scores. Next, we verified the AI model using a validation cohort (n = 95). Finally, we evaluated the accuracy of the prognostic predictions of the AI model and revealed that the AI model detected clear and eosinophilic phenotypes with high accuracy. The AI model stratified the patients' outcomes, and the predicted eosinophilic phenotypes correlated with adverse clinicopathological characteristics and high immune-related gene signatures. In conclusion, the AI-based histologic subclassification accurately predicted clear or eosinophilic phenotypes of ccRCC, allowing for consistently reproducible stratification for prognostic and therapeutic stratification.

Beyond high hopes: A scoping review of the 2019-2021 scientific discourse on machine learning in medical imaging

Machine learning has become a key driver of the digital health revolution. That comes with a fair share of high hopes and hype. We conducted a scoping review on machine learning in medical imaging, providing a comprehensive outlook of the field's potential, limitations, and future directions. Most reported strengths and promises included: improved (a) analytic power, (b) efficiency (c) decision making, and (d) equity. Most reported challenges included: (a) structural barriers and imaging heterogeneity, (b) scarcity of well-annotated, representative and interconnected imaging datasets (c) validity and performance limitations, including bias and equity issues, and (d) the still missing clinical integration. The boundaries between strengths and challenges, with cross-cutting ethical and regulatory implications, remain blurred. The literature emphasizes explainability and trustworthiness, with a largely missing discussion about the specific technical and regulatory challenges surrounding these concepts. Future trends are expected to shift towards multi-source models, combining imaging with an array of other data, in a more open access, and explainable manner.

A CT-based transfer learning approach to predict NSCLC recurrence: The added-value of peritumoral region

Non-small cell lung cancer (NSCLC) represents 85% of all new lung cancer diagnoses and presents a high recurrence rate after surgery. Thus, an accurate prediction of recurrence risk in NSCLC patients at diagnosis could be essential to designate risk patients to more aggressive medical treatments. In this manuscript, we apply a transfer learning approach to predict recurrence in NSCLC patients, exploiting only data acquired during its screening phase. Particularly, we used a public radiogenomic dataset of NSCLC patients having a primary tumor CT image and clinical information. Starting from the CT slice containing the tumor with maximum area, we considered three different dilatation sizes to identify three Regions of Interest (ROIs): CROP (without dilation), CROP 10 and CROP 20. Then, from each ROI, we extracted radiomic features by means of different pre-trained CNNs. The latter have been combined with clinical information; thus, we trained a Support Vector Machine classifier to predict the NSCLC recurrence. The classification performances of the devised models were finally evaluated on both the hold-out training and hold-out test sets, in which the original sample has been previously divided. The experimental results showed that the model obtained analyzing CROP 20 images, which are the ROIs containing more peritumoral area, achieved the best performances on both the hold-out training set, with an AUC of 0.73, an Accuracy of 0.61, a Sensitivity of 0.63, and a Specificity of 0.60, and on the hold-out test set, with an AUC value of 0.83, an Accuracy value of 0.79, a Sensitivity value of 0.80, and a Specificity value of 0.78. The proposed model represents a promising procedure for early predicting recurrence risk in NSCLC patients.

A collaborative workflow between pathologists and deep learning for the evaluation of tumour cellularity in lung adenocarcinoma

AIMS

The reporting of tumour cellularity in cancer samples has become a mandatory task for pathologists. However, the estimation of tumour cellularity is often inaccurate. Therefore, we propose a collaborative workflow between pathologists and artificial intelligence (AI) models to evaluate tumour cellularity in lung cancer samples and propose a protocol to apply it to routine practice.

METHODS AND RESULTS

We developed a quantitative model of lung adenocarcinoma that was validated and tested on 50 cases, and a collaborative workflow where pathologists could access the AI results and adjust their original tumour cellularity scores (adjusted-score) that we tested on 151 cases. The adjusted-score was validated by comparing them with a ground truth established by manual annotation of haematoxylin and eosin slides with reference to immunostains with thyroid transcription factor-1 and napsin A. For training, validation, testing the AI and testing the collaborative workflow, we used 40, 10, 50 and 151 whole slide images of lung adenocarcinoma, respectively. The sensitivity and specificity of tumour segmentation were 97 and 87%, respectively, and the accuracy of nuclei recognition was 99%. One pathologist's visually estimated scores were compared to the adjusted-score, and the pathologist's scores were altered in 87% of cases. Comparison with the ground truth revealed that the adjusted-score was more precise than the pathologists' scores (P < 0.05).

CONCLUSION

We proposed a collaborative workflow between AI and pathologists as a model to improve daily practice and enhance the prediction of tumour cellularity for genetic tests.

An Automatic Segmentation Method for Lung Tumor Based on Improved Region Growing Algorithm

In medical image processing, accurate segmentation of lung tumors is very important. Computer-aided accurate segmentation can effectively assist doctors in surgery planning and treatment decisions. Although the accurate segmentation results of lung tumors can provide a reliable basis for clinical treatment, the key to obtaining accurate segmentation results is how to improve the segmentation performance of the algorithm. We propose an automatic segmentation method for lung tumors based on an improved region growing algorithm, which uses the prior information on lung tumors to achieve an automatic selection of the initial seed point. The proposed method includes a seed point expansion mechanism and an automatic threshold update mechanism and takes the combination of multiple segmentation results as the final segmentation result. In the lung image database consortium (LIDC-IDRI) dataset, we designed 10 experiments to test the proposed method and compare it with 4 popular segmentation methods. The experimental results show that the average dice coefficient obtained by the proposed method is 0.936 ± 0.027, and the average Jaccard distance is 0.114 ± 0.049. The average dice coefficient obtained by the proposed method is 0.107, 0.053, 0.040, and 0.156, higher than that of the other four methods, respectively. This study proves that the proposed method can automatically segment lung tumors in CT slices and has suitable segmentation performance.

Probe electrospray ionization mass spectrometry-based rapid diagnosis of liver tumors

BACKGROUND AND AIM

Prompt differential diagnosis of liver tumors is clinically important and sometimes difficult. A new diagnostic device that combines probe electrospray ionization-mass spectrometry (PESI-MS) and machine learning may help provide the differential diagnosis of liver tumors.

METHODS

We evaluated the diagnostic accuracy of this new PESI-MS device using tissues obtained and stored from previous surgically resected specimens. The following cancer tissues (with collection dates): hepatocellular carcinoma (HCC, 2016-2019), intrahepatic cholangiocellular carcinoma (ICC, 2014-2019), and colorectal liver metastasis (CRLM, 2014-2019) from patients who underwent hepatic resection were considered for use in this study. Non-cancerous liver tissues (NL) taken from CRLM cases were also incorporated into the analysis. Each mass spectrum provided by PESI-MS was tested using support vector machine, a type of machine learning, to evaluate the discriminatory ability of the device.

RESULTS

In this study, we used samples from 91 of 139 patients with HCC, all 24 ICC samples, and 103 of 202 CRLM samples; 80 NL from CRLM cases were also used. Each mass spectrum was obtained by PESI-MS in a few minutes and was evaluated by machine learning. The sensitivity, specificity, and diagnostic accuracy of the PESI-MS device for discriminating HCC, ICC, and CRLM from among a mix of all three tumors and from NL were 98.9%, 98.1%, and 98.3%; 87.5%, 93.1%, and 92.6%; and 99.0%, 97.9%, and 98.3%, respectively.

CONCLUSION

This study demonstrated that PESI-MS and machine learning could discriminate liver tumors accurately and rapidly.

Next Generation Digital Pathology: Emerging Trends and Measurement Challenges for Molecular Pathology

Digital pathology is revolutionising the analysis of histological features and is becoming more and more widespread in both the clinic and research. Molecular pathology extends the tissue morphology information provided by conventional histopathology by providing spatially resolved molecular information to complement the structural information provided by histopathology. The multidimensional nature of the molecular data poses significant challenge for data processing, mining, and analysis. One of the key challenges faced by new and existing pathology practitioners is how to choose the most suitable molecular pathology technique for a given diagnosis. By providing a comparison of different methods, this narrative review aims to introduce the field of molecular pathology, providing a high-level overview of many different methods. Since each pixel of an image contains a wealth of molecular information, data processing in molecular pathology is more complex. The key data processing steps and variables, and their effect on the data, are also discussed.

Pathological prognosis classification of patients with neuroblastoma using computational pathology analysis

Neuroblastoma is the most common extracranial solid tumor in early childhood. International Neuroblastoma Pathology Classification (INPC) is a commonly used classification system that provides clinicians with a reference for treatment stratification. However, given the complex and subjective assessment of the INPC, there will be inconsistencies in the analysis of the same patient by multiple pathologists. An automated, comprehensive and objective classification method is needed to identify different prognostic groups in patients with neuroblastoma. In this study, we collected 563 hematoxylin and eosin-stained histopathology whole-slide images from 107 patients with neuroblastoma who underwent surgical resection. We proposed a novel processing pipeline for nuclear segmentation, cell-level image feature extraction, and patient-level feature aggregation. Logistic regression model was built to classify patients with favorable histology (FH) and patients with unfavorable histology (UH). On the training/test dataset, patient-level of nucleus morphological/intensity features and age could correctly classify patients with a mean area under the receiver operating characteristic curve (AUC) of 0.946, a mean accuracy of 0.856, and a mean Matthews Correlation Coefficient (MCC) of 0.703,respectively. On the independent validation dataset, the classification model achieved a mean AUC of 0.938, a mean accuracy of 0.865 and a mean MCC of 0.630, showing good generalizability. Our results suggested that automatically derived image features could identify the differences in nuclear morphological and intensity between different prognostic groups, which could provide a reference to pathologists and facilitate the evaluation of the pathological prognosis in patients with neuroblastoma.

The state of the art for artificial intelligence in lung digital pathology

Lung diseases carry a significant burden of morbidity and mortality worldwide. The advent of digital pathology (DP) and an increase in computational power have led to the development of artificial intelligence (AI)-based tools that can assist pathologists and pulmonologists in improving clinical workflow and patient management. While previous works have explored the advances in computational approaches for breast, prostate, and head and neck cancers, there has been a growing interest in applying these technologies to lung diseases as well. The application of AI tools on radiology images for better characterization of indeterminate lung nodules, fibrotic lung disease, and lung cancer risk stratification has been well documented. In this article, we discuss methodologies used to build AI tools in lung DP, describing the various hand-crafted and deep learning-based unsupervised feature approaches. Next, we review AI tools across a wide spectrum of lung diseases including cancer, tuberculosis, idiopathic pulmonary fibrosis, and COVID-19. We discuss the utility of novel imaging biomarkers for different types of clinical problems including quantification of biomarkers like PD-L1, lung disease diagnosis, risk stratification, and prediction of response to treatments such as immune checkpoint inhibitors. We also look briefly at some emerging applications of AI tools in lung DP such as multimodal data analysis, 3D pathology, and transplant rejection. Lastly, we discuss the future of DP-based AI tools, describing the challenges with regulatory approval, developing reimbursement models, planning clinical deployment, and addressing AI biases. © 2022 The Authors. The Journal of Pathology published by John Wiley & Sons Ltd on behalf of The Pathological Society of Great Britain and Ireland.

Validation of whole slide imaging for frozen section diagnosis of lymph node metastasis: A retrospective study from a tertiary care hospital in Thailand

BACKGROUND

The use of whole slide imaging (WSI) for frozen section (FS) diagnosis is helpful, particularly in the context of pathologist shortages. However, there is minimal data on such usage in resource-limited settings. This study aims to validate the use of WSI for FS diagnosis of lymph node metastasis using a low-cost virtual microscope scanner with consumer-grade laptops at a tertiary care hospital in Thailand.

METHODS

FS slides were retrieved for which the clinical query was to evaluate lymph node metastasis. They were digitized by a virtual microscope scanner (MoticEasyScan, Hong Kong) using up to 40× optical magnification. Three observers with different pathology experience levels diagnosed each slide, reviewing glass slides (GS) followed by digital slides (DS) after two weeks of a wash out period. WSI and GS diagnoses were compared. The time used for scanning and diagnosis of each slide was recorded.

RESULTS

295 FS slides were retrieved and digitized. The first-time successful scanning rate was 93.6 %. The mean scanning time was 2 min per slide. Both intraobserver agreement and interobserver agreement of WSI and GS diagnoses were high (Cohen's K; kappa value >0.84). The time used for DS diagnosis decreased as the observer's experience with WSI increased.

CONCLUSIONS

Despite varying pathological experiences, observers using WSI provided accurate FS diagnoses of lymph node metastasis. The time required for DS diagnoses decreased with additional observer's experience with WSI. Therefore, a WSI system containing low-cost scanners and consumer-grade laptops could be used for FS services in hospital laboratories lacking pathologists.

Automated whole-slide images assessment of immune infiltration in resected non-small-cell lung cancer: towards better risk-stratification

Background

High immune infiltration is associated with favourable prognosis in patients with non-small-cell lung cancer (NSCLC), but an automated workflow for characterizing immune infiltration, with high validity and reliability, remains to be developed.

Methods

We performed a multicentre retrospective study of patients with completely resected NSCLC. We developed an image analysis workflow for automatically evaluating the density of CD3⁺ and CD8⁺ T-cells in the tumour regions on immunohistochemistry (IHC)-stained whole-slide images (WSIs), and proposed an immune scoring system “I-score” based on the automated assessed cell density.

Results

A discovery cohort (n = 145) and a validation cohort (n = 180) were used to assess the prognostic value of the I-score for disease-free survival (DFS). The I-score (two-category) was an independent prognostic factor after adjusting for other clinicopathologic factors. Compared with a low I-score (two-category), a high I-score was associated with significantly superior DFS in the discovery cohort (adjusted hazard ratio [HR], 0.54; 95% confidence interval [CI] 0.33–0.86; P = 0.010) and validation cohort (adjusted HR, 0.57; 95% CI 0.36–0.92; P = 0.022). The I-score improved the prognostic stratification when integrating it into the Cox proportional hazard regression models with other risk factors (discovery cohort, C-index 0.742 vs. 0.728; validation cohort, C-index 0.695 vs. 0.685).

Conclusion

This automated workflow and immune scoring system would advance the clinical application of immune microenvironment evaluation and support the clinical decision making for patients with resected NSCLC.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12967-022-03458-9.

Substantial improvement of histopathological diagnosis by whole-slide image-based remote consultation

Consultation by subspecialty experts is the most common mode of rendering diagnosis in challenging cases in pathological practice. Our study aimed to highlight the diagnostic benefits of whole-slide image (WSI)-based remote consultation. We obtained diagnostically challenging cases from two institutions from the years 2010 and 2013, with histological diagnoses that contained keywords “probable,” “suggestive,” “suspicious,” “inconclusive,” and “uncertain.” A total of 270 cases were selected for remote consultation using WSIs scanned at 40 × . The consultation process consisted of three rounds: the first and second rounds each with 12 subspecialty experts and the third round with six multi-expertise senior pathologists. The first consultation yielded 44% concordance, and a change in diagnosis occurred in 56% of cases. The most frequent change was from inconclusive to definite diagnosis (30%), followed by minor discordance (14%), and major discordance (12%). Out of the 70 cases which reached the second round, 31 cases showed discrepancy between the two consultants. For these 31 cases, a consensus diagnosis was provided by six multi-expertise senior pathologists. Combining all WSI-based consultation rounds, the original inconclusive diagnosis was changed in 140 (52%) out of 266 cases. Among these cases, 80 cases (30%) upgraded the inconclusive diagnosis to a definite diagnosis, and 60 cases (22%) changed the diagnosis with major or minor discordance, accounting for 28 cases (10%) and 32 cases (12%), respectively. We observed significant improvement in the pathological diagnosis of difficult cases by remote consultation using WSIs, which can further assist in patient healthcare. A post-study survey highlighted various benefits of WSI-based consults.

Current Trend of Artificial Intelligence Patents in Digital Pathology: A Systematic Evaluation of the Patent Landscape

Simple Summary

The combination of digital pathology (DP) with artificial intelligence (AI) offers faster, more accurate, and more comprehensive diagnoses, resulting in more precise individualized treatment. As this technology is constantly evolving, it is critical to understand the current state of AI applications in DP. Thus, it is necessary to analyze AI patent applications, assignees, and leaders in the field. In this study, five major patent databases, namely, those of the USPTO, EPO, KIPO, JPO, and CNIPA, were searched using key phrases, such as DP, AI, machine learning, and deep learning, and 523 patents were shortlisted based on the inclusion criteria. Our data demonstrated that the key areas of the patents were whole-slide imaging, segmentation, classification, and detection. In the past five years, an increasing trend in patent filing has been observed, mainly in a few prominent countries, with a focus on the digitization of pathological images and AI technologies that support the critical role of pathologists.

Abstract

The integration of digital pathology (DP) with artificial intelligence (AI) enables faster, more accurate, and thorough diagnoses, leading to more precise personalized treatment. As technology is advancing rapidly, it is critical to understand the current state of AI applications in DP. Therefore, a patent analysis of AI in DP is required to assess the application and publication trends, major assignees, and leaders in the field. We searched five major patent databases, namely, those of the USPTO, EPO, KIPO, JPO, and CNIPA, from 1974 to 2021, using keywords such as DP, AI, machine learning, and deep learning. We discovered 6284 patents, 523 of which were used for trend analyses on time series, international distribution, top assignees; word cloud analysis; and subject category analyses. Patent filing and publication have increased exponentially over the past five years. The United States has published the most patents, followed by China and South Korea (248, 117, and 48, respectively). The top assignees were Paige.AI, Inc. (New York City, NY, USA) and Siemens, Inc. (Munich, Germany) The primary areas were whole-slide imaging, segmentation, classification, and detection. Based on these findings, we expect a surge in DP and AI patent applications focusing on the digitalization of pathological images and AI technologies that support the vital role of pathologists.

Models of Artificial Intelligence-Assisted Diagnosis of Lung Cancer Pathology Based on Deep Learning Algorithms

In this article, in order to explore the application of a diagnosis system for lung cancer, we use an auxiliary diagnostic system to predict and diagnose the good and evil attributes of chest CT pulmonary nodules. This research improves the new diagnosis method based on the convolutional neural network (CNN) and the recurrent neural network (RNN) and combines the dual effects of the two algorithms to process the classification of benign and malignant nodules. By collecting H-E-stained pathological slices of 652 patients' lung lesions from two hospitals between January 2018 and January 2019, the output results of the improved 3D U-net system and the consistent results of two-person reading were compared. This article analyzes the sensitivity, specificity, positive flammability rate, and negative flammability rate of different lung nodule detection methods. In addition, the artificial intelligence system's and the radiologist's judgment results of benign and malignant pulmonary nodules are used to draw ROC curves for further analysis. The improved model has an accuracy rate of 92.3% for predicting malignant lung nodules and an accuracy rate of 82.8% for benign lung nodules. The new diagnostic method using the convolutional neural network and the recurrent neural network can be very effective for improving the accuracy of predicting lung cancer diagnosis. It can play a very effective role in the disease prediction of lung cancer patients, thereby improving the treatment effect.

Deep-Learning-Aided Detection of Mycobacteria in Pathology Specimens Increases the Sensitivity in Early Diagnosis of Pulmonary Tuberculosis Compared with Bacteriology Tests

The histopathological diagnosis of mycobacterial infection may be improved by a comprehensive analysis using artificial intelligence. Two autopsy cases of pulmonary tuberculosis, and forty biopsy cases of undetected acid-fast bacilli (AFB) were used to train AI (convolutional neural network), and construct an AI to support AFB detection. Forty-two patients underwent bronchoscopy, and were evaluated using AI-supported pathology to detect AFB. The AI-supported pathology diagnosis was compared with bacteriology diagnosis from bronchial lavage fluid and the final definitive diagnosis of mycobacteriosis. Among the 16 patients with mycobacteriosis, bacteriology was positive in 9 patients (56%). Two patients (13%) were positive for AFB without AI assistance, whereas AI-supported pathology identified eleven positive patients (69%). When limited to tuberculosis, AI-supported pathology had significantly higher sensitivity compared with bacteriology (86% vs. 29%, p = 0.046). Seven patients diagnosed with mycobacteriosis had no consolidation or cavitary shadows in computed tomography; the sensitivity of bacteriology and AI-supported pathology was 29% and 86%, respectively (p = 0.046). The specificity of AI-supported pathology was 100% in this study. AI-supported pathology may be more sensitive than bacteriological tests for detecting AFB in samples collected via bronchoscopy.

Application of Artificial Intelligence in Lung Cancer

Simple Summary

Lung cancer is the leading cause of malignancy-related mortality worldwide. AI has the potential to help to treat lung cancer from detection, diagnosis and decision making to prognosis prediction. AI could reduce the labor work of LDCT, CXR, and pathology slides reading. AI as a second reader in LDCT and CXR reading reduces the effort of radiologists and increases the accuracy of nodule detection. Introducing AI to WSI in digital pathology increases the Kappa value of the pathologist and help to predict molecular phenotypes with radiomics and H&E staining. By extracting radiomics from image data and WSI from the histopathology field, clinicians could use AI to predict tumor properties such as gene mutation and PD-L1 expression. Furthermore, AI could help clinicians in decision-making by predicting treatment response, side effects, and prognosis prediction in medical treatment, surgery, and radiotherapy. Integrating AI in the future clinical workflow would be promising.

Abstract

Lung cancer is the leading cause of malignancy-related mortality worldwide due to its heterogeneous features and diagnosis at a late stage. Artificial intelligence (AI) is good at handling a large volume of computational and repeated labor work and is suitable for assisting doctors in analyzing image-dominant diseases like lung cancer. Scientists have shown long-standing efforts to apply AI in lung cancer screening via CXR and chest CT since the 1960s. Several grand challenges were held to find the best AI model. Currently, the FDA have approved several AI programs in CXR and chest CT reading, which enables AI systems to take part in lung cancer detection. Following the success of AI application in the radiology field, AI was applied to digitalized whole slide imaging (WSI) annotation. Integrating with more information, like demographics and clinical data, the AI systems could play a role in decision-making by classifying EGFR mutations and PD-L1 expression. AI systems also help clinicians to estimate the patient’s prognosis by predicting drug response, the tumor recurrence rate after surgery, radiotherapy response, and side effects. Though there are still some obstacles, deploying AI systems in the clinical workflow is vital for the foreseeable future.

CHAIMELEON Project: Creation of a Pan-European Repository of Health Imaging Data for the Development of AI-Powered Cancer Management Tools

The CHAIMELEON project aims to set up a pan-European repository of health imaging data, tools and methodologies, with the ambition to set a standard and provide resources for future AI experimentation for cancer management. The project is a 4 year long, EU-funded project tackling some of the most ambitious research in the fields of biomedical imaging, artificial intelligence and cancer treatment, addressing the four types of cancer that currently have the highest prevalence worldwide: lung, breast, prostate and colorectal. To allow this, clinical partners and external collaborators will populate the repository with multimodality (MR, CT, PET/CT) imaging and related clinical data. Subsequently, AI developers will enable a multimodal analytical data engine facilitating the interpretation, extraction and exploitation of the information stored at the repository. The development and implementation of AI-powered pipelines will enable advancement towards automating data deidentification, curation, annotation, integrity securing and image harmonization. By the end of the project, the usability and performance of the repository as a tool fostering AI experimentation will be technically validated, including a validation subphase by world-class European AI developers, participating in Open Challenges to the AI Community. Upon successful validation of the repository, a set of selected AI tools will undergo early in-silico validation in observational clinical studies coordinated by leading experts in the partner hospitals. Tool performance will be assessed, including external independent validation on hallmark clinical decisions in response to some of the currently most important clinical end points in cancer. The project brings together a consortium of 18 European partners including hospitals, universities, R&D centers and private research companies, constituting an ecosystem of infrastructures, biobanks, AI/in-silico experimentation and cloud computing technologies in oncology.

Construction and Multicenter Diagnostic Verification of Intelligent Recognition System for Endoscopic Images From Early Gastric Cancer Based on YOLO-V3 Algorithm

Introduction

Endoscopy is an important tool for the diagnosis of early gastric cancer. Therefore, a combination of artificial intelligence and endoscopy has the ability to increase the speed and efficiency of early gastric cancer diagnosis. YOU ONLY LOOK ONCE (YOLO) is an advanced object detection depth neural network algorithm that has not been widely used in gastrointestinal image recognition.

Objective

We developed an artificial intelligence system herein referred to as “EGC-YOLO” for the rapid and accurate diagnosis of endoscopic images from early gastric cancer.

Methods

More than 40000 gastroscopic images from 1653 patients in Yixing people’s Hospital were used as the training set for the system, while endoscopic images from the other two hospitals were used as external validation test sets. The sensitivity, specificity, positive predictive value, Youden index and ROC curve were analyzed to evaluate detection efficiencies for EGC-YOLO.

Results

EGC-YOLO was able to diagnose early gastric cancer in the two test sets with a high superiority and efficiency. The accuracy, sensitivity, specificity and positive predictive value for Test Sets 1 and 2 were 85.15% and 86.02%, 85.36% and 83.02%, 84.41% and 92.21%, and 95.22% and 95.65%, respectively. In Test Sets 1 and 2, the corresponding Threshold-values were 0.02, 0.16 and 0.17 at the maximum of the Youden index. An increase in Threshold-values was associated with a downward trend in sensitivity and accuracy, while specificity remained relatively stable at more than 80%.

Conclusions

The EGC-YOLO system is superior for the efficient, accurate and rapid detection of early gastric cancer lesions. For different data sets, it is important to select the appropriate threshold-value in advance to achieve the best performance of the EGC-YOLO system.

MIXTURE of human expertise and deep learning-developing an explainable model for predicting pathological diagnosis and survival in patients with interstitial lung disease

Interstitial pneumonia is a heterogeneous disease with a progressive course and poor prognosis, at times even worse than those in the main cancer types. Histopathological examination is crucial for its diagnosis and estimation of prognosis. However, the evaluation strongly depends on the experience of pathologists, and the reproducibility of diagnosis is low. Herein, we propose MIXTURE (huMan-In-the-loop eXplainable artificial intelligence Through the Use of REcurrent training), an original method to develop deep learning models for extracting pathologically significant findings based on an expert pathologist's perspective with a small annotation effort. The procedure of MIXTURE consists of three steps as follows. First, we created feature extractors for tiles from whole slide images using self-supervised learning. The similar looking tiles were clustered based on the output features and then pathologists integrated the pathologically synonymous clusters. Using the integrated clusters as labeled data, deep learning models to classify the tiles into pathological findings were created by transfer-learning the feature extractors. We developed three models for different magnifications. Using these extracted findings, our model was able to predict the diagnosis of usual interstitial pneumonia, a finding suggestive of progressive disease, with high accuracy (AUC 0.90 in validation set and AUC 0.86 in test set). This high accuracy could not be achieved without the integration of findings by pathologists. The patients predicted as UIP had poorer prognosis (5-year overall survival [OS]: 55.4%) than those predicted as non-UIP (OS: 95.2%). The Cox proportional hazards model for each microscopic finding and prognosis pointed out dense fibrosis, fibroblastic foci, elastosis, and lymphocyte aggregation as independent risk factors. We suggest that MIXTURE may serve as a model approach to different diseases evaluated by medical imaging, including pathology and radiology, and be the prototype for explainable artificial intelligence that can collaborate with humans.

Artificial Intelligence in Digital Pathology: What Is the Future? Part 2: An Investigation on the Insiders

Motivation: This study deals with the introduction of artificial intelligence (AI) in digital pathology (DP). The study starts from the highlights of a companion paper. Objective: The aim was to investigate the consensus and acceptance of the insiders on this issue. Procedure: An electronic survey based on the standardized package Microsoft Forms (Microsoft, Redmond, WA, USA) was proposed to a sample of biomedical laboratory technicians (149 admitted in the study, 76 males, 73 females, mean age 44.2 years). Results: The survey showed no criticality. It highlighted (a) the good perception of the basic training on both groups, and (b) a uniformly low perceived knowledge of AI (as arisen from the graded questions). Expectations, perceived general impact, perceived changes in the work-flow, and worries clearly emerged in the study. Conclusions: The of AI in DP is an unstoppable process, as well as the increase of the digitalization in the health domain. Stakeholders must not look with suspicion towards AI, which can represent an important resource, but should invest in monitoring and consensus training initiatives based also on electronic surveys.

Deep Learning Fast Screening Approach on Cytological Whole Slides for Thyroid Cancer Diagnosis

Simple Summary

Papillary thyroid carcinoma is the most common type of thyroid cancer and could be cured if diagnosed and treated early. In clinical practice, the primary method for determining diagnosis of papillary thyroid carcinoma is manual visual inspection of cytopathology slides, which is difficult, time consuming and subjective with a high inter-observer variability and sometimes causes suboptimal patient management due to false-positive and false-negative results. This study presents a fast, fully automatic and efficient deep learning framework for fast screening of cytological slides for thyroid cancer diagnosis. We confirmed the robustness and effectiveness of the proposed method based on evaluation results from two different types of slides: thyroid fine needle aspiration smears and ThinPrep slides.

Abstract

Thyroid cancer is the most common cancer in the endocrine system, and papillary thyroid carcinoma (PTC) is the most prevalent type of thyroid cancer, accounting for 70 to 80% of all thyroid cancer cases. In clinical practice, visual inspection of cytopathological slides is an essential initial method used by the pathologist to diagnose PTC. Manual visual assessment of the whole slide images is difficult, time consuming, and subjective, with a high inter-observer variability, which can sometimes lead to suboptimal patient management due to false-positive and false-negative. In this study, we present a fully automatic, efficient, and fast deep learning framework for fast screening of papanicolaou-stained thyroid fine needle aspiration (FNA) and ThinPrep (TP) cytological slides. To the authors’ best of knowledge, this work is the first study to build an automated deep learning framework for identification of PTC from both FNA and TP slides. The proposed deep learning framework is evaluated on a dataset of 131 WSIs, and the results show that the proposed method achieves an accuracy of 99%, precision of 85%, recall of 94% and F1-score of 87% in segmentation of PTC in FNA slides and an accuracy of 99%, precision of 97%, recall of 98%, F1-score of 98%, and Jaccard-Index of 96% in TP slides. In addition, the proposed method significantly outperforms the two state-of-the-art deep learning methods, i.e., U-Net and SegNet, in terms of accuracy, recall, F1-score, and Jaccard-Index (p<0.001). Furthermore, for run-time analysis, the proposed fast screening method takes 0.4 min to process a WSI and is 7.8 times faster than U-Net and 9.1 times faster than SegNet, respectively.

Artificial Intelligence in Digital Pathology: What Is the Future? Part 1: From the Digital Slide Onwards

This commentary aims to address the field of Artificial intelligence (AI) in Digital Pathology (DP) both in terms of the global situation and research perspectives. It has four polarities. First, it revisits the evolutions of digital pathology with particular care to the two fields of the digital cytology and the digital histology. Second, it illustrates the main fields in the employment of AI in DP. Third, it looks at the future directions of the research challenges from both a clinical and technological point of view. Fourth, it discusses the transversal problems among these challenges and implications and introduces the immediate work to implement.

Requirement of artificial intelligence technology awareness for thoracic surgeons

Background

We have recently witnessed incredible interest in computer-based, internet web-dependent mechanisms and artificial intelligence (AI)-dependent technique emergence in our day-to-day lives. In the recent era of COVID-19 pandemic, this nonhuman, machine-based technology has gained a lot of momentum.

Main body of the abstract

The supercomputers and robotics with AI technology have shown the potential to equal or even surpass human experts' accuracy in some tasks in the future. Artificial intelligence (AI) is prompting massive data interweaving with elements from many digital sources such as medical imaging sorting, electronic health records, and transforming healthcare delivery. But in thoracic surgical and our counterpart pulmonary medical field, AI's main applications are still for interpretation of thoracic imaging, lung histopathological slide evaluation, physiological data interpretation, and biosignal testing only. The query arises whether AI-enabled technology-based or autonomous robots could ever do or provide better thoracic surgical procedures than current surgeons but it seems like an impossibility now.

Short conclusion

This review article aims to provide information pertinent to the use of AI to thoracic surgical specialists. In this review article, we described AI and related terminologies, current utilisation, challenges, potential, and current need for awareness of this technology.

Artificial intelligence-based analysis for immunohistochemistry staining of immune checkpoints to predict resected non-small cell lung cancer survival and relapse

Background

Conventional analysis of single-plex chromogenic immunohistochemistry (IHC) focused on quantitative but spatial analysis. How immune checkpoints localization related to non-small cell lung cancer (NSCLC) prognosis remained unclear.

Methods

Here, we analyzed ten immune checkpoints on 1,859 tumor microarrays (TMAs) from 121 NSCLC patients and recruited an external cohort of 30 NSCLC patients with 214 whole-slide IHC. EfficientUnet was applied to segment tumor cells (TCs) and tumor-infiltrating lymphocytes (TILs), while ResNet was performed to extract prognostic features from IHC images.

Results

The features of galectin-9, OX40, OX40L, KIR2D, and KIR3D played an un-negatable contribution to overall survival (OS) and relapse-free survival (RFS) in the internal cohort, validated in public databases (GEPIA, HPA, and STRING). The IC-Score and Res-Score were two predictive models established by EfficientUnet and ResNet. Based on the IC-Score, Res-Score, and clinical features, the integrated score presented the highest AUC for OS and RFS, which could achieve 0.9 and 0.85 in the internal testing cohort. The robustness of Res-Score was validated in the external cohort (AUC: 0.80–0.87 for OS, and 0.83–0.94 for RFS). Additionally, the neutrophil-to-lymphocyte ratio (NLR) combined with the PD-1/PD-L1 signature established by EfficientUnet can be a predictor for RFS in the external cohort.

Conclusions

Overall, we established a reliable model to risk-stratify relapse and death in NSCLC with a generalization ability, which provided a convenient approach to spatial analysis of single-plex chromogenic IHC.

RPB5-mediating protein promotes the progression of non-small cell lung cancer by regulating the proliferation and invasion

Background

This study aimed to investigate the relationship between RNA polymerase II subunit 5 (RPB5)-mediating protein (RMP) and clinicopathological characteristics of non-small cell lung cancer (NSCLC) patients by measuring the expression level of RMP in human NSCLC tissues and cell lines. At the same time, we studied the impact of RMP on the biological function of cancer, providing strong support for gene targeted therapy of NSCLC.

Methods

Real-time quantitative reverse transcription polymerase chain reaction (qRT-PCR) and Western blot were used to determine the expression levels of messenger (m)RNA and protein in NSCLC cell lines and tissues. Cell counting kit 8 (CCK8) assay and flow cytometry were selected to detect cell proliferation, cycle and apoptosis. The wound healing assay was chosen to detect the migration and invasion ability of cells. The xenograft model was performed to study the function of RMP in vivo. Immunohistochemical (IHC) staining showed the levels of RMP, Bcl-2, Bax and caspase-3.

Results

First, mRNA and protein levels of RMP were relatively overexpressed in NSCLC cells. Compared with the corresponding normal tissues, the mRNA and protein levels of RMP were significantly higher in human NSCLC tissues. Concurrently, we found that the expression of RMP was related to the status of lymph nodes (LNs) in cancer tissues and T stage. Then, RMP overexpression promoted the proliferation of A549. At the same time, RMP provided A549 cells the ability to resist chemotherapy and radiotherapy; when A549 cells were treated with gefitinib and radiation, RMP reduced apoptosis. We also found that RMP can protect A549 from G2 block caused by radiation. Over-irradiated RMP-overexpressed A549 cells had lower Bcl2-associated X protein (Bax) levels and higher B-cell lymphoma 2 (Bcl-2) levels. The migration and invasion ability of A549 cells was increased by RMP. Finally, RMP can promote tumor growth by increasing Bcl-2 levels and decreasing Bax and caspase-3 levels in the xenograft model.

Conclusions

There is potential for RMP to develop into a diagnostic and therapeutic target for NSCLC.