Current Trend of Artificial Intelligence Patents in Digital Pathology: A Systematic Evaluation of the Patent Landscape

Whole slide image-level classification of malignant effusion cytology using clustering-constrained attention multiple instance learning

BACKGROUND

Cytological diagnosis of pleural effusion plays an important role in the early detection and diagnosis of lung cancers. Recently, attempts have been made to overcome low diagnostic accuracy and interobserver variability using artificial intelligence-based image analysis. However, such analysis is primarily performed at the image-patch level and not at the whole-slide image (WSI) level. This study aims to develop a WSI-level classification of malignant effusions in metastatic lung cancer based on pleural fluid cytology using a quality-controlled, nationwide dataset.

METHODS

The dataset was collected by a consortium research group that included three major university hospitals and the Quality Assurance Program Committee of the Korean Society of Cytopathology. It contains 576 normal and 309 cancer WSIs from pleural fluids. A clustering-constrained attention multiple-instance learning (CLAM) model was used for WSI-level classification.

RESULTS

The CLAM model achieved a high accuracy of 97%, with an area under the curve of 0.97, representing a 13% improvement over the image patch classification model-based WSI classification. It also significantly reduced the analysis time and computing resources compared to those required during image patch-level classification and heat map generation on the WSIs.

CONCLUSION

The CLAM model successfully demonstrated high performance in differentiating malignant effusion at the WSI level using a large, quality-controlled, nationwide dataset. Further external validation is required to ensure generalizability.

Artificial intelligence for diagnosis and predictive biomarkers in Non-Small cell lung cancer Patients: New promises but also new hurdles for the pathologist

The rapid development of artificial intelligence (AI) based tools in pathology laboratories has brought forward unlimited opportunities for pathologists. Promising AI applications used for accomplishing diagnostic, prognostic and predictive tasks are being developed at a high pace. This is notably true in thoracic oncology, given the significant and rapid therapeutic progress made recently for lung cancer patients. Advances have been based on drugs targeting molecular alterations, immunotherapies, and, more recently antibody-drug conjugates which are soon to be introduced. For over a decade, many proof-of-concept studies have explored the use of AI algorithms in thoracic oncology to improve lung cancer patient care. However, despite the enthusiasm in this domain, the set-up and use of AI algorithms in daily practice of thoracic pathologists has not been operative until now, due to several constraints. The purpose of this review is to describe the potential but also the current barriers of AI applications in routine thoracic pathology for non-small cell lung cancer patient care and to suggest practical solutions for rapid future implementation.

Deep learning methods for improving the accuracy and efficiency of pathological image analysis

This study presents a novel integration of two advanced deep learning models, U-Net and EfficientNetV2, to achieve high-precision segmentation and rapid classification of pathological images. A key innovation is the development of a new heatmap generation algorithm, which leverages meticulous image preprocessing, data enhancement strategies, ensemble learning, attention mechanisms, and deep feature fusion techniques. This algorithm not only produces highly accurate and interpretatively rich heatmaps but also significantly improves the accuracy and efficiency of pathological image analysis. Unlike existing methods, our approach integrates these advanced techniques into a cohesive framework, enhancing its ability to reveal critical features in pathological images. Rigorous experimental validation demonstrated that our algorithm excels in key performance indicators such as accuracy, recall rate, and processing speed, underscoring its potential for broader applications in pathological image analysis and beyond.

Utility of artificial intelligence in a binary classification of soft tissue tumors

Soft tissue tumors (STTs) pose diagnostic and therapeutic challenges due to their rarity, complexity, and morphological overlap. Accurate differentiation between benign and malignant STTs is important to set treatment directions, however, this task can be difficult. The integration of machine learning and artificial intelligence (AI) models can potentially be helpful in classifying these tumors. The aim of this study was to investigate AI and machine learning tools in the classification of STT into benign and malignant categories. This study consisted of three components: (1) Evaluation of whole-slide images (WSIs) to classify STT into benign and malignant entities. Five specialized soft tissue pathologists from different medical centers independently reviewed 100 WSIs, representing 100 different cases, with limited clinical information and no additional workup. The results showed an overall concordance rate of 70.4% compared to the reference diagnosis. (2) Identification of cell-specific parameters that can distinguish benign and malignant STT. Using an image analysis software (QuPath) and a cohort of 95 cases, several cell-specific parameters were found to be statistically significant, most notably cell count, nucleus/cell area ratio, nucleus hematoxylin density mean, and cell max caliper. (3) Evaluation of machine learning library (Scikit-learn) in differentiating benign and malignant STTs. A total of 195 STT cases (156 cases in the training group and 39 cases in the validation group) achieved approximately 70% sensitivity and specificity, and an AUC of 0.68. Our limited study suggests that the use of WSI and AI in soft tissue pathology has the potential to enhance diagnostic accuracy and identify parameters that can differentiate between benign and malignant STTs. We envision the integration of AI as a supportive tool to augment the pathologists' diagnostic capabilities.

Artificial intelligence in digital pathology: a systematic review and meta-analysis of diagnostic test accuracy

Ensuring diagnostic performance of artificial intelligence (AI) before introduction into clinical practice is essential. Growing numbers of studies using AI for digital pathology have been reported over recent years. The aim of this work is to examine the diagnostic accuracy of AI in digital pathology images for any disease. This systematic review and meta-analysis included diagnostic accuracy studies using any type of AI applied to whole slide images (WSIs) for any disease. The reference standard was diagnosis by histopathological assessment and/or immunohistochemistry. Searches were conducted in PubMed, EMBASE and CENTRAL in June 2022. Risk of bias and concerns of applicability were assessed using the QUADAS-2 tool. Data extraction was conducted by two investigators and meta-analysis was performed using a bivariate random effects model, with additional subgroup analyses also performed. Of 2976 identified studies, 100 were included in the review and 48 in the meta-analysis. Studies were from a range of countries, including over 152,000 whole slide images (WSIs), representing many diseases. These studies reported a mean sensitivity of 96.3% (CI 94.1-97.7) and mean specificity of 93.3% (CI 90.5-95.4). There was heterogeneity in study design and 99% of studies identified for inclusion had at least one area at high or unclear risk of bias or applicability concerns. Details on selection of cases, division of model development and validation data and raw performance data were frequently ambiguous or missing. AI is reported as having high diagnostic accuracy in the reported areas but requires more rigorous evaluation of its performance.

A systematic review of machine learning-based tumor-infiltrating lymphocytes analysis in colorectal cancer: Overview of techniques, performance metrics, and clinical outcomes

The incidence of colorectal cancer (CRC), one of the deadliest cancers around the world, is increasing. Tissue microenvironment (TME) features such as tumor-infiltrating lymphocytes (TILs) can have a crucial impact on diagnosis or decision-making for treating patients with CRC. While clinical studies showed that TILs improve the host immune response, leading to a better prognosis, inter-observer agreement for quantifying TILs is not perfect. Incorporating machine learning (ML) based applications in clinical routine may promote diagnosis reliability. Recently, ML has shown potential for making progress in routine clinical procedures. We aim to systematically review the TILs analysis based on ML in CRC histological images. Deep learning (DL) and non-DL techniques can aid pathologists in identifying TILs, and automated TILs are associated with patient outcomes. However, a large multi-institutional CRC dataset with a diverse and multi-ethnic population is necessary to generalize ML methods.

Examining labelling guidelines for AI-based software as a medical device: A review and analysis of dermatology mobile applications in Australia

In recent years, there has been a surge in the development of AI-based Software as a Medical Device (SaMD), particularly in visual specialties such as dermatology. In Australia, the Therapeutic Goods Administration (TGA) regulates AI-based SaMD to ensure its safe use. Proper labelling of these devices is crucial to ensure that healthcare professionals and the general public understand how to use them and interpret results accurately. However, guidelines for labelling AI-based SaMD in dermatology are lacking, which may result in products failing to provide essential information about algorithm development and performance metrics. This review examines existing labelling guidelines for AI-based SaMD across visual medical specialties, with a specific focus on dermatology. Common recommendations for labelling are identified and applied to currently available dermatology AI-based SaMD mobile applications to determine usage of these labels. Of the 21 AI-based SaMD mobile applications identified, none fully comply with common labelling recommendations. Results highlight the need for standardized labelling guidelines. Ensuring transparency and accessibility of information is essential for the safe integration of AI into health care and preventing potential risks associated with inaccurate clinical decisions.

Advances in NIR-Responsive Natural Macromolecular Hydrogel Assembly Drugs for Cancer Treatment

Cancer is a serious disease with an abnormal proliferation of organ tissues; it is characterized by malignant infiltration and growth that affects human life. Traditional cancer therapies such as resection, radiotherapy and chemotherapy have a low cure rate and often cause irreversible damage to the body. In recent years, since the traditional treatment of cancer is still very far from perfect, researchers have begun to focus on non-invasive near-infrared (NIR)-responsive natural macromolecular hydrogel assembly drugs (NIR-NMHADs). Due to their unique biocompatibility and extremely high drug encapsulation, coupling with the spatiotemporal controllability of NIR, synergistic photothermal therapy (PTT), photothermal therapy (PDT), chemotherapy (CT) and immunotherapy (IT) has created excellent effects and good prospects for cancer treatment. In addition, some emerging bioengineering technologies can also improve the effectiveness of drug delivery systems. This review will discuss the properties of NIR light, the NIR-functional hydrogels commonly used in current research, the cancer therapy corresponding to the materials encapsulated in them and the bioengineering technology that can assist drug delivery systems. The review provides a constructive reference for the optimization of NIR-NMHAD experimental ideas and its application to human body.

Building Automation Pipeline for Diagnostic Classification of Sporadic Odontogenic Keratocysts and Non-Keratocysts Using Whole-Slide Images

The microscopic diagnostic differentiation of odontogenic cysts from other cysts is intricate and may cause perplexity for both clinicians and pathologists. Of particular interest is the odontogenic keratocyst (OKC), a developmental cyst with unique histopathological and clinical characteristics. Nevertheless, what distinguishes this cyst is its aggressive nature and high tendency for recurrence. Clinicians encounter challenges in dealing with this frequently encountered jaw lesion, as there is no consensus on surgical treatment. Therefore, the accurate and early diagnosis of such cysts will benefit clinicians in terms of treatment management and spare subjects from the mental agony of suffering from aggressive OKCs, which impact their quality of life. The objective of this research is to develop an automated OKC diagnostic system that can function as a decision support tool for pathologists, whether they are working locally or remotely. This system will provide them with additional data and insights to enhance their decision-making abilities. This research aims to provide an automation pipeline to classify whole-slide images of OKCs and non-keratocysts (non-KCs: dentigerous and radicular cysts). OKC diagnosis and prognosis using the histopathological analysis of tissues using whole-slide images (WSIs) with a deep-learning approach is an emerging research area. WSIs have the unique advantage of magnifying tissues with high resolution without losing information. The contribution of this research is a novel, deep-learning-based, and efficient algorithm that reduces the trainable parameters and, in turn, the memory footprint. This is achieved using principal component analysis (PCA) and the ReliefF feature selection algorithm (ReliefF) in a convolutional neural network (CNN) named P-C-ReliefF. The proposed model reduces the trainable parameters compared to standard CNN, achieving 97% classification accuracy.

CTM and QFD analysis: Framework for fintech adoption priority in commercial banks

As financial technology (fintech) is developing rapidly, many commercial banks experience difficulty deciding what kind of fintech to primarily focus on when managing their business. Owing to limited resources and assets, there is a practical need for guidelines for banks' investments in fintech. This study provides a systemic procedure to identify promising fintech groups and their investment priorities. We propose a QFD-based decision support framework for banks by considering both aspects of the emerging fintech push identified using patent topic modeling and the market pull of banking services obtained from a survey of the literature and experts. An empirical application of the proposed QFD framework to major South Korean banks shows that transaction support technology, secure transactions, and trading platforms are the three most important fintech categories. The QFD results are utilized to guide individual banks for further investment strategies such as mergers and acquisitions, strategic partnerships, and spin-off operations. The proposed framework can be generalized and applied to other financial service firms.

Deep Learning-Based Computational Cytopathologic Diagnosis of Metastatic Breast Carcinoma in Pleural Fluid

A Pleural effusion cytology is vital for treating metastatic breast cancer; however, concerns have arisen regarding the low accuracy and inter-observer variability in cytologic diagnosis. Although artificial intelligence-based image analysis has shown promise in cytopathology research, its application in diagnosing breast cancer in pleural fluid remains unexplored. To overcome these limitations, we evaluate the diagnostic accuracy of an artificial intelligence-based model using a large collection of cytopathological slides, to detect the malignant pleural effusion cytology associated with breast cancer. This study includes a total of 569 cytological slides of malignant pleural effusion of metastatic breast cancer from various institutions. We extracted 34,221 augmented image patches from whole-slide images and trained and validated a deep convolutional neural network model (DCNN) (Inception-ResNet-V2) with the images. Using this model, we classified 845 randomly selected patches, which were reviewed by three pathologists to compare their accuracy. The DCNN model outperforms the pathologists by demonstrating higher accuracy, sensitivity, and specificity compared to the pathologists (81.1% vs. 68.7%, 95.0% vs. 72.5%, and 98.6% vs. 88.9%, respectively). The pathologists reviewed the discordant cases of DCNN. After re-examination, the average accuracy, sensitivity, and specificity of the pathologists improved to 87.9, 80.2, and 95.7%, respectively. This study shows that DCNN can accurately diagnose malignant pleural effusion cytology in breast cancer and has the potential to support pathologists.

Biomimicry Industry and Patent Trends

This study examines the current technological level and industrial/technical trends in the field of biomimicry technology, as well as recent technological and research and development trends. Patent analysis was conducted, focusing on technology that uses design elements and biological/ecological characteristics to provide solutions to technological problems. The technological scope of the analysis included the field of technologies and materials that apply to the conditions found in ecology, as well as robot machines and devices designed to mimic certain animals and ecological elements. The search for patents was conducted in Korea, the United States, Japan, and Europe from 1975 to 2021, resulting in a total of 8278 raw data cases, from which 940 valid patents were selected. The percentage of patent document and the status of both domestic and foreign applicants varied among the countries of Korea, the United States, Japan, and Europe. Based on the results of the patent analysis, it was found that biomimicry technology is in a growth phase that is expected to continue in the future and that Korea and the United States are leading the development of this technology.

Recent application of artificial intelligence on histopathologic image-based prediction of gene mutation in solid cancers

PURPOSE

Evaluation of genetic mutations in cancers is important because distinct mutational profiles help determine individualized drug therapy. However, molecular analyses are not routinely performed in all cancers because they are expensive, time-consuming and not universally available. Artificial intelligence (AI) has shown the potential to determine a wide range of genetic mutations on histologic image analysis. Here, we assessed the status of mutation prediction AI models on histologic images by a systematic review.

METHODS

A literature search using the MEDLINE, Embase and Cochrane databases was conducted in August 2021. The articles were shortlisted by titles and abstracts. After a full-text review, publication trends, study characteristic analysis and comparison of performance metrics were performed.

RESULTS

Twenty-four studies were found mostly from developed countries, and their number is increasing. The major targets were gastrointestinal, genitourinary, gynecological, lung and head and neck cancers. Most studies used the Cancer Genome Atlas, with a few using an in-house dataset. The area under the curve of some of the cancer driver gene mutations in particular organs was satisfactory, such as 0.92 of BRAF in thyroid cancers and 0.79 of EGFR in lung cancers, whereas the average of all gene mutations was 0.64, which is still suboptimal.

CONCLUSION

AI has the potential to predict gene mutations on histologic images with appropriate caution. Further validation with larger datasets is still required before AI models can be used in clinical practice to predict gene mutations.

Development of quality assurance program for digital pathology by the Korean Society of Pathologists

Background: Digital pathology (DP) using whole slide imaging is a recently emerging game changer technology that can fundamentally change the way of working in pathology. The Digital Pathology Study Group (DPSG) of the Korean Society of Pathologists (KSP) published a consensus report on the recommendations for pathologic practice using DP. Accordingly, the need for the development and implementation of a quality assurance program (QAP) for DP has been raised.Methods: To provide a standard baseline reference for internal and external QAP for DP, the members of the Committee of Quality Assurance of the KSP developed a checklist for the Redbook and a QAP trial for DP based on the prior DPSG consensus report. Four leading institutes participated in the QAP trial in the first year, and we gathered feedback from these institutes afterwards.Results: The newly developed checklists of QAP for DP contain 39 items (216 score): eight items for quality control of DP systems; three for DP personnel; nine for hardware and software requirements for DP systems; 15 for validation, operation, and management of DP systems; and four for data security and personal information protection. Most participants in the QAP trial replied that continuous education on unfamiliar terminology and more practical experience is demanding.Conclusions: The QAP for DP is essential for the safe implementation of DP in pathologic practice. Each laboratory should prepare an institutional QAP according to this checklist, and consecutive revision of the checklist with feedback from the QAP trial for DP needs to follow.

Recent Application of Artificial Intelligence in Non-Gynecological Cancer Cytopathology: A Systematic Review

State-of-the-art artificial intelligence (AI) has recently gained considerable interest in the healthcare sector and has provided solutions to problems through automated diagnosis. Cytological examination is a crucial step in the initial diagnosis of cancer, although it shows limited diagnostic efficacy. Recently, AI applications in the processing of cytopathological images have shown promising results despite the elementary level of the technology. Here, we performed a systematic review with a quantitative analysis of recent AI applications in non-gynecological (non-GYN) cancer cytology to understand the current technical status. We searched the major online databases, including MEDLINE, Cochrane Library, and EMBASE, for relevant English articles published from January 2010 to January 2021. The searched query terms were: "artificial intelligence", "image processing", "deep learning", "cytopathology", and "fine-needle aspiration cytology." Out of 17,000 studies, only 26 studies (26 models) were included in the full-text review, whereas 13 studies were included for quantitative analysis. There were eight classes of AI models treated of according to target organs: thyroid (n = 11, 39%), urinary bladder (n = 6, 21%), lung (n = 4, 14%), breast (n = 2, 7%), pleural effusion (n = 2, 7%), ovary (n = 1, 4%), pancreas (n = 1, 4%), and prostate (n = 1, 4). Most of the studies focused on classification and segmentation tasks. Although most of the studies showed impressive results, the sizes of the training and validation datasets were limited. Overall, AI is also promising for non-GYN cancer cytopathology analysis, such as pathology or gynecological cytology. However, the lack of well-annotated, large-scale datasets with Z-stacking and external cross-validation was the major limitation found across all studies. Future studies with larger datasets with high-quality annotations and external validation are required.

Recent Applications of Artificial Intelligence from Histopathologic Image-Based Prediction of Microsatellite Instability in Solid Cancers: A Systematic Review

Cancers with high microsatellite instability (MSI-H) have a better prognosis and respond well to immunotherapy. However, MSI is not tested in all cancers because of the additional costs and time of diagnosis. Therefore, artificial intelligence (AI)-based models have been recently developed to evaluate MSI from whole slide images (WSIs). Here, we aimed to assess the current state of AI application to predict MSI based on WSIs analysis in MSI-related cancers and suggest a better study design for future studies. Studies were searched in online databases and screened by reference type, and only the full texts of eligible studies were reviewed. The included 14 studies were published between 2018 and 2021, and most of the publications were from developed countries. The commonly used dataset is The Cancer Genome Atlas dataset. Colorectal cancer (CRC) was the most common type of cancer studied, followed by endometrial, gastric, and ovarian cancers. The AI models have shown the potential to predict MSI with the highest AUC of 0.93 in the case of CRC. The relatively limited scale of datasets and lack of external validation were the limitations of most studies. Future studies with larger datasets are required to implicate AI models in routine diagnostic practice for MSI prediction.