Improving Generalization of Deep Learning Models for Diagnostic Pathology by Increasing Variability in Training Data: Experiments on Osteosarcoma Subtypes

Seeing the random forest through the decision trees. Supporting learning health systems from histopathology with machine learning models: Challenges and opportunities

This paper discusses some overlooked challenges faced when working with machine learning models for histopathology and presents a novel opportunity to support "Learning Health Systems" with them. Initially, the authors elaborate on these challenges after separating them according to their mitigation strategies: those that need innovative approaches, time, or future technological capabilities and those that require a conceptual reappraisal from a critical perspective. Then, a novel opportunity to support "Learning Health Systems" by integrating hidden information extracted by ML models from digitalized histopathology slides with other healthcare big data is presented.

[The model transferability of AI in digital pathology : Potential and reality]

OBJECTIVE

Artificial intelligence (AI) holds the potential to make significant advancements in pathology. However, its actual implementation and certification for practical use are currently limited, often due to challenges related to model transferability. In this context, we investigate the factors influencing transferability and present methods aimed at enhancing the utilization of AI algorithms in pathology.

MATERIALS AND METHODS

Various convolutional neural networks (CNNs) and vision transformers (ViTs) were trained using datasets from two institutions, along with the publicly available TCGA-MIBC dataset. These networks conducted predictions in urothelial tissue and intrahepatic cholangiocarcinoma (iCCA). The objective was to illustrate the impact of stain normalization, the influence of various artifacts during both training and testing, as well as the effects of the NoisyEnsemble method.

RESULTS

We were able to demonstrate that stain normalization of slides from different institutions has a significant positive effect on the inter-institutional transferability of CNNs and ViTs (respectively +13% and +10%). In addition, ViTs usually achieve a higher accuracy in the external test (here +1.5%). Similarly, we showcased how artifacts in test data can negatively affect CNN predictions and how incorporating these artifacts during training leads to improvements. Lastly, NoisyEnsembles of CNNs (better than ViTs) were shown to enhance transferability across different tissues and research questions (+7% Bladder, +15% iCCA).

DISCUSSION

It is crucial to be aware of the transferability challenge: achieving good performance during development does not necessarily translate to good performance in real-world applications. The inclusion of existing methods to enhance transferability, such as stain normalization and NoisyEnsemble, and their ongoing refinement, is of importance.

Detection of Large-Droplet Macrovesicular Steatosis in Donor Livers Based on Segment-Anything Model

Liver transplantation is an effective treatment for end-stage liver disease, acute liver failure, and primary hepatic malignancy. However, the limited availability of donor organs remains a challenge. Severe large-droplet fat (LDF) macrovesicular steatosis, characterized by cytoplasmic replacement with large fat vacuoles, can lead to liver transplant complications. Artificial intelligence models, such as segmentation and detection models, are being developed to detect LDF hepatocytes. The Segment-Anything Model, utilizing the DEtection TRansformer architecture, has the ability to segment objects without prior knowledge of size or shape. We investigated the Segment-Anything Model's potential to detect LDF hepatocytes in liver biopsies. Pathologist-annotated specimens were used to evaluate model performance. The model showed high sensitivity but compromised specificity due to similarities with other structures. Filtering algorithms were developed to improve specificity. Integration of the Segment-Anything Model with rule-based algorithms accurately detected LDF hepatocytes. Improved diagnosis and treatment of liver diseases can be achieved through advancements in artificial intelligence algorithms for liver histology analysis.

A Deep Learning Approach for Automatic and Objective Grading of the Motor Impairment Severity in Parkinson's Disease for Use in Tele-Assessments

Wearable sensors provide a tool for at-home monitoring of motor impairment progression in neurological conditions such as Parkinson's disease (PD). This study examined the ability of deep learning approaches to grade the motor impairment severity in a modified version of the Movement Disorders Society-sponsored revision of the Unified Parkinson's Disease Rating Scale (MDS-UPDRS) using low-cost wearable sensors. We hypothesized that expanding training datasets with motion data from healthy older adults (HOAs) and initializing classifiers with weights learned from unsupervised pre-training would lead to an improvement in performance when classifying lower vs. higher motor impairment relative to a baseline deep learning model (XceptionTime). This study evaluated the change in classification performance after using expanded training datasets with HOAs and transferring weights from unsupervised pre-training compared to a baseline deep learning model (XceptionTime) using both upper extremity (finger tapping, hand movements, and pronation-supination movements of the hands) and lower extremity (toe tapping and leg agility) tasks consistent with the MDS-UPDRS. Overall, we found a 12.2% improvement in accuracy after expanding the training dataset and pre-training using max-vote inference on hand movement tasks. Moreover, we found that the classification performance improves for every task except toe tapping after the addition of HOA training data. These findings suggest that learning from HOA motion data can implicitly improve the representations of PD motion data for the purposes of motor impairment classification. Further, our results suggest that unsupervised pre-training can improve the performance of motor impairment classifiers without any additional annotated PD data, which may provide a viable solution for a widely deployable telemedicine solution.

Improved artificial intelligence discrimination of minor histological populations by supplementing with color-adjusted images

Despite the dedicated research of artificial intelligence (AI) for pathological images, the construction of AI applicable to histopathological tissue subtypes, is limited by insufficient dataset collection owing to disease infrequency. Here, we present a solution involving the addition of supplemental tissue array (TA) images that are adjusted to the tonality of the main data using a cycle-consistent generative adversarial network (CycleGAN) to the training data for rare tissue types. F1 scores of rare tissue types that constitute < 1.2% of the training data were significantly increased by improving recall values after adding color-adjusted TA images constituting < 0.65% of total training patches. The detector also enabled the equivalent discrimination of clinical images from two distinct hospitals and the capability was more increased following color-correction of test data before AI identification (F1 score from 45.2 ± 27.1 to 77.1 ± 10.3, p < 0.01). These methods also classified intraoperative frozen sections, while excessive supplementation paradoxically decreased F1 scores. These results identify strategies for building an AI that preserves the imbalance between training data with large differences in actual disease frequencies, which is important for constructing AI for practical histopathological classification.

Unleashing the potential of AI for pathology: challenges and recommendations

Computational pathology is currently witnessing a surge in the development of AI techniques, offering promise for achieving breakthroughs and significantly impacting the practices of pathology and oncology. These AI methods bring with them the potential to revolutionize diagnostic pipelines as well as treatment planning and overall patient care. Numerous peer-reviewed studies reporting remarkable performance across diverse tasks serve as a testimony to the potential of AI in the field. However, widespread adoption of these methods in clinical and pre-clinical settings still remains a challenge. In this review article, we present a detailed analysis of the major obstacles encountered during the development of effective models and their deployment in practice. We aim to provide readers with an overview of the latest developments, assist them with insights into identifying some specific challenges that may require resolution, and suggest recommendations and potential future research directions. © 2023 The Authors. The Journal of Pathology published by John Wiley & Sons Ltd on behalf of The Pathological Society of Great Britain and Ireland.

Machine learning to predict overall short-term mortality in cutaneous melanoma

BACKGROUND

Cutaneous malignant melanoma (CMM) ranks among the ten most frequent malignancies, clinicopathological staging being of key importance to predict prognosis. Artificial intelligence (AI) has been recently applied to develop prognostically reliable staging systems for CMM. This study aims to provide a useful machine learning based tool to predict the overall CMM short-term survival.

METHODS

CMM records as collected at the Veneto Cancer Registry (RTV) and at the Veneto regional health service were considered. A univariate Cox regression validated the strength and direction of each independent variable with overall mortality. A range of machine learning models (Logistic Regression classifier, Support-Vector Machine, Random Forest, Gradient Boosting, and k-Nearest Neighbors) and a Deep Neural Network were then trained to predict the 3-years mortality probability. Five-fold cross-validation and Grid Search were performed to test the best data preprocessing procedures, features selection, and to optimize models hyperparameters. A final evaluation was carried out on a separate test set in terms of balanced accuracy, precision, recall and F1 score. The best model was deployed as online tool.

RESULTS

The univariate analysis confirmed the significant prognostic value of TNM staging. Adjunctive clinicopathological variables not included in the AJCC 8th melanoma staging system, i.e., sex, tumor site, histotype, growth phase, and age, were significantly linked to overall survival. Among the models, the Neural Network and the Random Forest models featured the best prognostic performance, achieving a balanced accuracy of 91% and 88%, respectively. According to the Gini importance score, age, T and M stages, mitotic count, and ulceration appeared to be the variables with the greatest impact on survival prediction.

CONCLUSIONS

Using data from patients with CMM, we developed an AI algorithm with high staging reliability, on top of which a web tool was implemented ( unipd.link/melanomaprediction ). Being essentially based on routinely recorded clinicopathological variables, it can already be implemented with minimal effort and further tested in the current clinical practice, an essential phase for validating the model's accuracy beyond the original research context.

Design of a Honey Badger Optimization Algorithm with a Deep Transfer Learning-Based Osteosarcoma Classification Model

Osteosarcoma is one of the aggressive bone tumors with numerous histologic patterns. Histopathological inspection is a crucial criterion in the medical diagnosis of Osteosarcoma. Due to the advancement of computing power and hardware technology, pathological image analysis system based on artificial intelligence (AI) were more commonly used. But classifying many intricate pathology images by hand will be challenging for pathologists. The lack of labeling data makes the system difficult to build and costly. This article designs a Honey Badger Optimization with Deep Learning based Automated Osteosarcoma Classification (HBODL-AOC) model. The HBODL-AOC technique's goal is to identify osteosarcoma's existence using medical images. In the presented HBODL-AOC technique, image preprocessing is initially performed by contrast enhancement technique. For feature extraction, the HBODL-AOC technique employs a deep convolutional neural network-based Mobile networks (MobileNet) model with an Adam optimizer for hyperparameter tuning. Finally, the adaptive neuro-fuzzy inference system (ANFIS) approach is implemented for the HBO (Honey Badger Optimization) algorithm can tune osteosarcoma classification and the membership function (MF). To demonstrate the enhanced classification performance of the HBODL-AOC approach, a sequence of simulations was performed. The extensive simulation analysis portrayed the improved performance of the HBODL-AOC technique over existing DL models.

Deep learning-based age estimation from chest X-rays indicates cardiovascular prognosis

BACKGROUND

In recent years, there has been considerable research on the use of artificial intelligence to estimate age and disease status from medical images. However, age estimation from chest X-ray (CXR) images has not been well studied and the clinical significance of estimated age has not been fully determined.

METHODS

To address this, we trained a deep neural network (DNN) model using more than 100,000 CXRs to estimate the patients' age solely from CXRs. We applied our DNN to CXRs of 1562 consecutive hospitalized heart failure patients, and 3586 patients admitted to the intensive care unit with cardiovascular disease.

RESULTS

The DNN's estimated age (X-ray age) showed a strong significant correlation with chronological age on the hold-out test data and independent test data. Elevated X-ray age is associated with worse clinical outcomes (heart failure readmission and all-cause death) for heart failure. Additionally, elevated X-ray age was associated with a worse prognosis in 3586 patients admitted to the intensive care unit with cardiovascular disease.

CONCLUSIONS

Our results suggest that X-ray age can serve as a useful indicator of cardiovascular abnormalities, which will help clinicians to predict, prevent and manage cardiovascular diseases.

John Charnley Award: Deep Learning Prediction of Hip Joint Center on Standard Pelvis Radiographs

BACKGROUND

Accurate hip joint center (HJC) determination is critical for preoperative planning, intraoperative execution, clinical outcomes after total hip arthroplasty, and commonly used classification systems in primary and revision hip replacement. However, current methods of preoperative HJC estimation are prone to subjectivity and human error. The purpose of the study was to leverage deep learning (DL) to develop a rapid and objective HJC estimation tool on anteroposterior (AP) pelvis radiographs.

METHODS

Radiographs from 3,965 patients (7,930 hips) were included. A DL model workflow was created to detect bony landmarks and estimate HJC based on a pelvic height ratio method. The workflow was utilized to conduct a grid-search for optimal nonspecific, sex-specific, and patient-specific (using contralateral hip) pelvic height ratios on the training/validation cohort (6,344 hips). Algorithm performance was assessed on an independent testing cohort for HJC estimation comparison.

RESULTS

The algorithm estimated HJC for the testing cohort at a rate of 0.65 seconds/hip based on features in AP radiographs alone. The model predicted HJC within 5 mm of error for 80% of hips using nonspecific ratios, which increased to 83% with sex-specific and 91% with patient-specific pelvic height ratio models. Mean error decreased utilizing the patient-specific model (3.09 ± 1.69 mm, P < .001).

CONCLUSION

Using DL, we developed nonspecific, sex-specific, and patient-specific models capable of estimating native HJC on AP pelvis radiographs. This tool may provide clinical value when considering preoperative component position in patients planned to undergo THA and in reducing the subjective variability in HJC estimation.

LEVEL OF EVIDENCE

Diagnostic, level IV.

Optimal Deep Stacked Sparse Autoencoder Based Osteosarcoma Detection and Classification Model

Osteosarcoma is a kind of bone cancer which generally starts to develop in the lengthy bones in the legs and arms. Because of an increase in occurrence of cancer and patient-specific treatment options, the detection and classification of cancer becomes a difficult process. The manual recognition of osteosarcoma necessitates expert knowledge and is time consuming. An earlier identification of osteosarcoma can reduce the death rate. With the development of new technologies, automated detection models can be exploited for medical image classification, thereby decreasing the expert’s reliance and resulting in timely identification. In recent times, an amount of Computer-Aided Detection (CAD) systems are available in the literature for the segmentation and detection of osteosarcoma using medicinal images. In this view, this research work develops a wind driven optimization with deep transfer learning enabled osteosarcoma detection and classification (WDODTL-ODC) method. The presented WDODTL-ODC model intends to determine the presence of osteosarcoma in the biomedical images. To accomplish this, the osteosarcoma model involves Gaussian filtering (GF) based on pre-processing and contrast enhancement techniques. In addition, deep transfer learning using a SqueezNet model is utilized as a featured extractor. At last, the Wind Driven Optimization (WDO) algorithm with a deep-stacked sparse auto-encoder (DSSAE) is employed for the classification process. The simulation outcome demonstrated that the WDODTL-ODC technique outperformed the existing models in the detection of osteosarcoma on biomedical images.

Recommendations on compiling test datasets for evaluating artificial intelligence solutions in pathology

Artificial intelligence (AI) solutions that automatically extract information from digital histology images have shown great promise for improving pathological diagnosis. Prior to routine use, it is important to evaluate their predictive performance and obtain regulatory approval. This assessment requires appropriate test datasets. However, compiling such datasets is challenging and specific recommendations are missing. A committee of various stakeholders, including commercial AI developers, pathologists, and researchers, discussed key aspects and conducted extensive literature reviews on test datasets in pathology. Here, we summarize the results and derive general recommendations on compiling test datasets. We address several questions: Which and how many images are needed? How to deal with low-prevalence subsets? How can potential bias be detected? How should datasets be reported? What are the regulatory requirements in different countries? The recommendations are intended to help AI developers demonstrate the utility of their products and to help pathologists and regulatory agencies verify reported performance measures. Further research is needed to formulate criteria for sufficiently representative test datasets so that AI solutions can operate with less user intervention and better support diagnostic workflows in the future.