Deep learning as a tool for increased accuracy and efficiency of histopathological diagnosis

From molecular mechanisms of prostate cancer to translational applications: based on multi-omics fusion analysis and intelligent medicine

Prostate cancer is the most common cancer in men worldwide and has a high mortality rate. The complex and heterogeneous development of prostate cancer has become a core obstacle in the treatment of prostate cancer. Simultaneously, the issues of overtreatment in early-stage diagnosis, oligometastasis and dormant tumor recognition, as well as personalized drug utilization, are also specific concerns that require attention in the clinical management of prostate cancer. Some typical genetic mutations have been proved to be associated with prostate cancer's initiation and progression. However, single-omic studies usually are not able to explain the causal relationship between molecular alterations and clinical phenotypes. Exploration from a systems genetics perspective is also lacking in this field, that is, the impact of gene network, the environmental factors, and even lifestyle behaviors on disease progression. At the meantime, current trend emphasizes the utilization of artificial intelligence (AI) and machine learning techniques to process extensive multidimensional data, including multi-omics. These technologies unveil the potential patterns, correlations, and insights related to diseases, thereby aiding the interpretable clinical decision making and applications, namely intelligent medicine. Therefore, there is a pressing need to integrate multidimensional data for identification of molecular subtypes, prediction of cancer progression and aggressiveness, along with perosonalized treatment performing. In this review, we systematically elaborated the landscape from molecular mechanism discovery of prostate cancer to clinical translational applications. We discussed the molecular profiles and clinical manifestations of prostate cancer heterogeneity, the identification of different states of prostate cancer, as well as corresponding precision medicine practices. Taking multi-omics fusion, systems genetics, and intelligence medicine as the main perspectives, the current research results and knowledge-driven research path of prostate cancer were summarized.

Computational pathology: A survey review and the way forward

Computational Pathology (CPath) is an interdisciplinary science that augments developments of computational approaches to analyze and model medical histopathology images. The main objective for CPath is to develop infrastructure and workflows of digital diagnostics as an assistive CAD system for clinical pathology, facilitating transformational changes in the diagnosis and treatment of cancer that are mainly address by CPath tools. With evergrowing developments in deep learning and computer vision algorithms, and the ease of the data flow from digital pathology, currently CPath is witnessing a paradigm shift. Despite the sheer volume of engineering and scientific works being introduced for cancer image analysis, there is still a considerable gap of adopting and integrating these algorithms in clinical practice. This raises a significant question regarding the direction and trends that are undertaken in CPath. In this article we provide a comprehensive review of more than 800 papers to address the challenges faced in problem design all-the-way to the application and implementation viewpoints. We have catalogued each paper into a model-card by examining the key works and challenges faced to layout the current landscape in CPath. We hope this helps the community to locate relevant works and facilitate understanding of the field's future directions. In a nutshell, we oversee the CPath developments in cycle of stages which are required to be cohesively linked together to address the challenges associated with such multidisciplinary science. We overview this cycle from different perspectives of data-centric, model-centric, and application-centric problems. We finally sketch remaining challenges and provide directions for future technical developments and clinical integration of CPath. For updated information on this survey review paper and accessing to the original model cards repository, please refer to GitHub. Updated version of this draft can also be found from arXiv.

Histopathological evaluation and grading for prostate cancer: current issues and crucial aspects

A crucial aspect of prostate cancer grading, especially in low- and intermediate-risk cancer, is the accurate identification of Gleason pattern 4 glands, which includes ill-formed or fused glands. However, there is notable inconsistency among pathologists in recognizing these glands, especially when mixed with pattern 3 glands. This inconsistency has significant implications for patient management and treatment decisions. Conversely, the recognition of glomeruloid and cribriform architecture has shown higher reproducibility. Cribriform architecture, in particular, has been linked to the worst prognosis among pattern 4 subtypes. Intraductal carcinoma of the prostate (IDC-P) is also associated with high-grade cancer and poor prognosis. Accurate identification, classification, and tumor size evaluation by pathologists are vital for determining patient treatment. This review emphasizes the importance of prostate cancer grading, highlighting challenges like distinguishing between pattern 3 and pattern 4 and the prognostic implications of cribriform architecture and intraductal proliferations. It also addresses the inherent grading limitations due to interobserver variability and explores the potential of computational pathology to enhance pathologist accuracy and consistency.

Computer-assisted urine cytology: Faster, cheaper, better?

Recent advancements in computer-assisted diagnosis (CAD) have catalysed significant progress in pathology, particularly in the realm of urine cytopathology. This review synthesizes the latest developments and challenges in CAD for diagnosing urothelial carcinomas, addressing the limitations of traditional urinary cytology. Through a literature review, we identify and analyse CAD models and algorithms developed for urine cytopathology, highlighting their methodologies and performance metrics. We discuss the potential of CAD to improve diagnostic accuracy, efficiency and patient outcomes, emphasizing its role in streamlining workflow and reducing errors. Furthermore, CAD tools have shown potential in exploring pathological conditions, uncovering novel biomarkers and prognostic/predictive features previously unknown or unseen. Finally, we examine the practical issues surrounding the integration of CAD into clinical practice, including regulatory approval, validation and training for pathologists. Despite the promising results, challenges remain, necessitating further research and validation efforts. Overall, CAD presents a transformative opportunity to revolutionize diagnostic practices in urine cytopathology, paving the way for enhanced patient care and outcomes.

Role of Machine Learning Assisted Biosensors in Point-of-Care-Testing For Clinical Decisions

Point-of-Care-Testing (PoCT) has emerged as an essential component of modern healthcare, providing rapid, low-cost, and simple diagnostic options. The integration of Machine Learning (ML) into biosensors has ushered in a new era of innovation in the field of PoCT. This article investigates the numerous uses and transformational possibilities of ML in improving biosensors for PoCT. ML algorithms, which are capable of processing and interpreting complicated biological data, have transformed the accuracy, sensitivity, and speed of diagnostic procedures in a variety of healthcare contexts. This review explores the multifaceted applications of ML models, including classification and regression, displaying how they contribute to improving the diagnostic capabilities of biosensors. The roles of ML-assisted electrochemical sensors, lab-on-a-chip sensors, electrochemiluminescence/chemiluminescence sensors, colorimetric sensors, and wearable sensors in diagnosis are explained in detail. Given the increasingly important role of ML in biosensors for PoCT, this study serves as a valuable reference for researchers, clinicians, and policymakers interested in understanding the emerging landscape of ML in point-of-care diagnostics.

Artificial intelligence integration in the drug lifecycle and in regulatory science: policy implications, challenges and opportunities

Artificial intelligence tools promise transformative impacts in drug development. Regulatory agencies face challenges in integrating AI while ensuring reliability and safety in clinical trial approvals, drug marketing authorizations, and post-market surveillance. Incorporating these technologies into the existing regulatory framework and agency practices poses notable challenges, particularly in evaluating the data and models employed for these purposes. Rapid adaptation of regulations and internal processes is essential for agencies to keep pace with innovation, though achieving this requires collective stakeholder collaboration. This article thus delves into the need for adaptations of regulations throughout the drug development lifecycle, as well as the utilization of AI within internal processes of medicine agencies.

Prediction of molecular subclasses of uveal melanoma by deep learning using routine haematoxylin-eosin-stained tissue slides

AIMS

Uveal melanoma has a high propensity to metastasize. Prognosis is associated with specific driver mutations and copy number variations, and these can only be obtained after genetic testing. In this study we evaluated the efficacy of patient outcome prediction using deep learning on haematoxylin and eosin (HE)-stained primary uveal melanoma slides in comparison to molecular testing.

METHODS

In this retrospective study of patients with uveal melanoma, 113 patients from the Erasmus Medical Centre who underwent enucleation had tumour tissue analysed for molecular classification between 1993 and 2020. Routine HE-stained slides were scanned to obtain whole-slide images (WSI). After annotation of regions of interest, tiles of 1024 × 1024 pixels were extracted at a magnification of 40×. An ablation study to select the best-performing deep-learning model was carried out using three state-of-the-art deep-learning models (EfficientNet, Vision Transformer, and Swin Transformer).

RESULTS

Deep-learning models were subjected to a training cohort (n = 40), followed by a validation cohort (n = 20), and finally underwent a test cohort (n = 48). A k-fold cross-validation (k = 3) of validation and test cohorts (n = 113 of three classes: BAP1, SF3B1, EIF1AX) demonstrated Swin Transformer as the best-performing deep-learning model to predict molecular subclasses based on HE stains. The model achieved an accuracy of 0.83 ± 0.09 on the validation cohort and 0.75 ± 0.04 on the test cohort. Within the subclasses, this model correctly predicted 70% BAP1-mutated, 61% SF3B1-mutated and 80% EIF1AX-mutated UM in the test set.

CONCLUSIONS

This study showcases the potential of the deep-learning methodology for predicting molecular subclasses in a multiclass manner using HE-stained WSI. This development holds promise for advanced prognostication of UM patients without the need of molecular or immunohistochemical testing. Additionally, this study suggests there are distinct histopathological features per subclass; mainly utilizing epithelioid cellular morphology for BAP1-classification, but an unknown feature distinguishes EIF1AX and SF3B1.

Combination of DCE-MRI and NME-DWI via Deep Neural Network for Predicting Breast Cancer Molecular Subtypes

BACKGROUND

To explore whether the combination of dynamic contrast-enhanced (DCE) magnetic resonance imaging (MRI) and nonmono-exponential (NME) model-based diffusion-weighted imaging (DWI) via deep neural network (DNN) can improve the prediction of breast cancer molecular subtypes compared to either imaging technique used alone.

PATIENTS AND METHODS

This prospective study examined 480 breast cancers in 475 patients undergoing DCE-MRI and NME-DWI at 3.0 T. Breast cancers were classified as follows: human epidermal growth factor receptor 2 enriched (HER2-enriched), luminal A, luminal B (HER2-), luminal B (HER2+), and triple-negative subtypes. A total of 20% cases were withheld as an independent test dataset, and the remaining cases were used to train DNN with an 80% to 20% training-validation split and 5-fold cross-validation. The diagnostic accuracies of DNN in 5-way subtype classification between the DCE-MRI, NME-DWI, and their combined multiparametric-MRI datasets were compared using analysis of variance with least significant difference posthoc test. Areas under the receiver-operating characteristic curves were calculated to assess the performances of DNN in binary subtype classification between the 3 datasets.

RESULTS

The 5-way classification accuracies of DNN on both DCE-MRI (0.71) and NME-DWI (0.64) were significantly lower (P < .05) than on multiparametric-MRI (0.76), while on DCE-MRI was significantly higher (P < .05) than on NME-DWI. The comparative results of binary classification between the 3 datasets were consistent with the 5-way classification.

CONCLUSION

The combination of DCE-MRI and NME-DWI via DNN achieved a significant improvement in breast cancer molecular subtype prediction compared to either imaging technique used alone. Additionally, DCE-MRI outperformed NME-DWI in differentiating subtypes.

Predicting uniaxial compressive strength of rocks using ANN models: Incorporating porosity, compressional wave velocity, and schmidt hammer data

The unconfined compressive strength (UCS) of intact rocks is crucial for engineering applications, but traditional laboratory testing is often impractical, especially for historic buildings lacking sufficient core samples. Non-destructive tests like the Schmidt hammer rebound number and compressional wave velocity offer solutions, but correlating these with UCS requires complex mathematical models. This paper introduces a novel approach using an artificial neural network (ANN) to simultaneously correlate UCS with three non-destructive test indexes: Schmidt hammer rebound number, compressional wave velocity, and open-effective porosity. The proposed ANN model outperforms existing methods, providing accurate UCS predictions for various rock types. Contour maps generated from the model offer practical tools for geotechnical and geological engineers, facilitating decision-making in the field and enhancing educational resources. This integrated approach promises to streamline UCS estimation, improving efficiency and accuracy in engineering assessments of intact rock materials.

A fully automated and explainable algorithm for predicting malignant transformation in oral epithelial dysplasia

Oral epithelial dysplasia (OED) is a premalignant histopathological diagnosis given to lesions of the oral cavity. Its grading suffers from significant inter-/intra-observer variability, and does not reliably predict malignancy progression, potentially leading to suboptimal treatment decisions. To address this, we developed an artificial intelligence (AI) algorithm, that assigns an Oral Malignant Transformation (OMT) risk score based on the Haematoxylin and Eosin (H&E) stained whole slide images (WSIs). Our AI pipeline leverages an in-house segmentation model to detect and segment both nuclei and epithelium. Subsequently, a shallow neural network utilises interpretable morphological and spatial features, emulating histological markers, to predict progression. We conducted internal cross-validation on our development cohort (Sheffield; n = 193 cases) and independent validation on two external cohorts (Birmingham and Belfast; n = 89 cases). On external validation, the proposed OMTscore achieved an AUROC = 0.75 (Recall = 0.92) in predicting OED progression, outperforming other grading systems (Binary: AUROC = 0.72, Recall = 0.85). Survival analyses showed the prognostic value of our OMTscore (C-index = 0.60, p = 0.02), compared to WHO (C-index = 0.64, p = 0.003) and binary grades (C-index = 0.65, p < 0.001). Nuclear analyses elucidated the presence of peri-epithelial and intra-epithelial lymphocytes in highly predictive patches of transforming cases (p < 0.001). This is the first study to propose a completely automated, explainable, and externally validated algorithm for predicting OED transformation. Our algorithm shows comparable-to-human-level performance, offering a promising solution to the challenges of grading OED in routine clinical practice.

Group-informed attentive framework for enhanced diabetes mellitus progression prediction

The increasing prevalence of Diabetes Mellitus (DM) as a global health concern highlights the paramount importance of accurately predicting its progression. This necessity has propelled the use of deep learning's advanced analytical and predictive capabilities to the forefront of current research. However, this approach is confronted with significant challenges, notably the prevalence of incomplete data and the need for more robust predictive models. Our research aims to address these critical issues, leveraging deep learning to enhance the precision and reliability of diabetes progression predictions. We address the issue of missing data by first locating individuals with data gaps within specific patient clusters, and then applying targeted imputation strategies for effective data imputation. To enhance the robustness of our model, we implement strategies such as data augmentation and the development of advanced group-level feature analysis. A cornerstone of our approach is the implementation of a deep attentive transformer that is sensitive to group characteristics. This framework excels in processing a wide array of data, including clinical and physical examination information, to accurately predict the progression of DM. Beyond its predictive capabilities, our model is engineered to perform advanced feature selection and reasoning. This is crucial for understanding the impact of both individual and group-level factors on deep models' predictions, providing invaluable insights into the dynamics of DM progression. Our approach not only marks a significant advancement in the prediction of diabetes progression but also contributes to a deeper understanding of the multifaceted factors influencing this chronic disease, thereby aiding in more effective diabetes management and research.

Machine-learning-based integrative -'omics analyses reveal immunologic and metabolic dysregulation in environmental enteric dysfunction

Environmental enteric dysfunction (EED) is a subclinical enteropathy challenging to diagnose due to an overlap of tissue features with other inflammatory enteropathies. EED subjects (n = 52) from Pakistan, controls (n = 25), and a validation EED cohort (n = 30) from Zambia were used to develop a machine-learning-based image analysis classification model. We extracted histologic feature representations from the Pakistan EED model and correlated them to transcriptomics and clinical biomarkers. In-silico metabolic network modeling was used to characterize alterations in metabolic flux between EED and controls and validated using untargeted lipidomics. Genes encoding beta-ureidopropionase, CYP4F3, and epoxide hydrolase 1 correlated to numerous tissue feature representations. Fatty acid and glycerophospholipid metabolism-related reactions showed altered flux. Increased phosphatidylcholine, lysophosphatidylcholine (LPC), and ether-linked LPCs, and decreased ester-linked LPCs were observed in the duodenal lipidome of Pakistan EED subjects, while plasma levels of glycine-conjugated bile acids were significantly increased. Together, these findings elucidate a multi-omic signature of EED.

Intracerebral Hemorrhage Prognosis Classification via Joint-Attention Cross-Modal Network

Intracerebral hemorrhage (ICH) is a critical condition characterized by a high prevalence, substantial mortality rates, and unpredictable clinical outcomes, which results in a serious threat to human health. Improving the timeliness and accuracy of prognosis assessment is crucial to minimizing mortality and long-term disability associated with ICH. Due to the complexity of ICH, the diagnosis of ICH in clinical practice heavily relies on the professional expertise and clinical experience of physicians. Traditional prognostic methods largely depend on the specialized knowledge and subjective judgment of healthcare professionals. Meanwhile, existing artificial intelligence (AI) methodologies, which predominantly utilize features derived from computed tomography (CT) scans, fall short of capturing the multifaceted nature of ICH. Although existing methods are capable of integrating clinical information and CT images for prognosis, the effectiveness of this fusion process still requires improvement. To surmount these limitations, the present study introduces a novel AI framework, termed the ICH Network (ICH-Net), which employs a joint-attention cross-modal network to synergize clinical textual data with CT imaging features. The architecture of ICH-Net consists of three integral components: the Feature Extraction Module, which processes and abstracts salient characteristics from the clinical and imaging data, the Feature Fusion Module, which amalgamates the diverse data streams, and the Classification Module, which interprets the fused features to deliver prognostic predictions. Our evaluation, conducted through a rigorous five-fold cross-validation process, demonstrates that ICH-Net achieves a commendable accuracy of up to 87.77%, outperforming other state-of-the-art methods detailed within our research. This evidence underscores the potential of ICH-Net as a formidable tool in prognosticating ICH, promising a significant advancement in clinical decision-making and patient care.

Multi-Scale Digital Pathology Patch-Level Prostate Cancer Grading Using Deep Learning: Use Case Evaluation of DiagSet Dataset

Prostate cancer remains a prevalent health concern, emphasizing the critical need for early diagnosis and precise treatment strategies to mitigate mortality rates. The accurate prediction of cancer grade is paramount for timely interventions. This paper introduces an approach to prostate cancer grading, framing it as a classification problem. Leveraging ResNet models on multi-scale patch-level digital pathology and the Diagset dataset, the proposed method demonstrates notable success, achieving an accuracy of 0.999 in identifying clinically significant prostate cancer. The study contributes to the evolving landscape of cancer diagnostics, offering a promising avenue for improved grading accuracy and, consequently, more effective treatment planning. By integrating innovative deep learning techniques with comprehensive datasets, our approach represents a step forward in the pursuit of personalized and targeted cancer care.

The return of natural dyes: the case of logwood tree (Haematoxylum campechianum L.)

In recent years, a worldwide reassessment of natural dyes has occurred, driven by the health and environmental issues associated with synthetic dyes. Haematoxylum campechianum L. is a tropical tree from which wood extracts were widely used in the textile industry during the 16th century. The logwood tree extract serves as a contemporary source of hematoxylin, a key dye in the globally prevalent hematoxylin-eosin staining method, a cornerstone in histopathological procedures. This paper will initially explore the re-emergence of natural dyes. Subsequently, it will focus on the historical, conventional, and innovative applications of logwood in the fields of medicine, histopathology, and nanotechnology, along with the status and alternative uses of the hematoxylin-eosin stain. Lastly, this paper will examine the current state of conservation and utilization of Haematoxylum campechianum in Campeche, Mexico, a leading global producer of hematoxylin.

Quantification of follicles in human ovarian tissue using image processing software and trained artificial intelligence†

Cancer survival rates in prepubertal girls and young women have risen in recent decades due to increasingly efficient treatments. However, many such treatments are gonadotoxic, causing premature ovarian insufficiency, loss of fertility, and ovarian endocrine function. Implantation of donor ovarian tissue encapsulated in immune-isolating capsules is a promising method to restore physiological endocrine function without immunosuppression or risk of reintroducing cancer cells harbored by the tissue. The success of this approach is largely determined by follicle density in the implanted ovarian tissue, which is analyzed manually from histologic sections and necessitates specialized, time-consuming labor. To address this limitation, we developed a fully automated method to quantify follicle density that does not require additional coding. We first analyzed ovarian tissue from 12 human donors between 16 and 37 years old using semi-automated image processing with manual follicle annotation and then trained artificial intelligence program based on follicle identification and object classification. One operator manually analyzed 102 whole slide images from serial histologic sections. Of those, 77 images were assessed by a second manual operator, followed with an automated method utilizing artificial intelligence. Of the 1181 follicles the control operator counted, the comparison operator counted 1178, and the artificial intelligence counted 927 follicles with 80% of those being correctly identified as follicles. The three-stage artificial intelligence pipeline finished 33% faster than manual annotation. Collectively, this report supports the use of artificial intelligence and automation to select tissue donors and grafts with the greatest follicle density to ensure graft longevity for premature ovarian insufficiency treatment.

Development and deployment of a histopathology-based deep learning algorithm for patient prescreening in a clinical trial

Accurate identification of genetic alterations in tumors, such as Fibroblast Growth Factor Receptor, is crucial for treating with targeted therapies; however, molecular testing can delay patient care due to the time and tissue required. Successful development, validation, and deployment of an AI-based, biomarker-detection algorithm could reduce screening cost and accelerate patient recruitment. Here, we develop a deep-learning algorithm using >3000 H&E-stained whole slide images from patients with advanced urothelial cancers, optimized for high sensitivity to avoid ruling out trial-eligible patients. The algorithm is validated on a dataset of 350 patients, achieving an area under the curve of 0.75, specificity of 31.8% at 88.7% sensitivity, and projected 28.7% reduction in molecular testing. We successfully deploy the system in a non-interventional study comprising 89 global study clinical sites and demonstrate its potential to prioritize/deprioritize molecular testing resources and provide substantial cost savings in the drug development and clinical settings.

3-Dimensional Reconstruction From Histopathological Sections: A Systematic Review

Although pathological tissue analysis is typically performed on single 2-dimensional (2D) histologic reference slides, 3-dimensional (3D) reconstruction from a sequence of histologic sections could provide novel opportunities for spatial analysis of the extracted tissue. In this review, we analyze recent works published after 2018 and report information on the extracted tissue types, the section thickness, and the number of sections used for reconstruction. By analyzing the technological requirements for 3D reconstruction, we observe that software tools exist, both free and commercial, which include the functionality to perform 3D reconstruction from a sequence of histologic images. Through the analysis of the most recent works, we provide an overview of the workflows and tools that are currently used for 3D reconstruction from histologic sections and address points for future work, such as a missing common file format or computer-aided analysis of the reconstructed model.

GMIM: Self-supervised pre-training for 3D medical image segmentation with adaptive and hierarchical masked image modeling

Self-supervised pre-training and fully supervised fine-tuning paradigms have received much attention to solve the data annotation problem in deep learning fields. Compared with traditional pre-training on large natural image datasets, medical self-supervised learning methods learn rich representations derived from unlabeled data itself thus avoiding the distribution shift between different image domains. However, nowadays state-of-the-art medical pre-training methods were specifically designed for downstream tasks making them less flexible and difficult to apply to new tasks. In this paper, we propose grid mask image modeling, a flexible and general self-supervised method to pre-train medical vision transformers for 3D medical image segmentation. Our goal is to guide networks to learn the correlations between organs and tissues by reconstructing original images based on partial observations. The relationships are consistent within the human body and invariant to disease type or imaging modality. To achieve this, we design a Siamese framework consisting of an online branch and a target branch. An adaptive and hierarchical masking strategy is employed in the online branch to (1) learn the boundaries or small contextual mutation regions within images; (2) to learn high-level semantic representations from deeper layers of the multiscale encoder. In addition, the target branch provides representations for contrastive learning to further reduce representation redundancy. We evaluate our method through segmentation performance on two public datasets. The experimental results demonstrate our method outperforms other self-supervised methods. Codes are available at https://github.com/mobiletomb/Gmim.

Morphometric Analysis of Lateral Cervical Lymph Node Metastasis in Papillary Thyroid Carcinoma Using Digital Pathology

Digital pathology uses digitized images for cancer research. We aimed to assess morphometric parameters using digital pathology for predicting recurrence in patients with papillary thyroid carcinoma (PTC) and lateral cervical lymph node (LN) metastasis. We analyzed 316 PTC patients and assessed the longest diameter and largest area of metastatic focus in LNs using a whole slide imaging scanner. In digital pathology assessment, the longest diameters and largest areas of metastatic foci in LNs were positively correlated with traditional optically measured diameters (R = 0.928 and R2 = 0.727, p < 0.001 and p < 0.001, respectively). The optimal cutoff diameter was 8.0 mm in both traditional microscopic (p = 0.009) and digital pathology (p = 0.016) evaluations, with significant differences in progression-free survival (PFS) observed at this cutoff (p = 0.006 and p = 0.002, respectively). The predictive area's cutoff was 35.6 mm2 (p = 0.005), which significantly affected PFS (p = 0.015). Using an 8.0-mm cutoff in traditional microscopic evaluation and a 35.6-mm2 cutoff in digital pathology showed comparable predictive results using the proportion of variation explained (PVE) methods (2.6% vs. 2.4%). Excluding cases with predominant cystic changes in LNs, the largest metastatic areas by digital pathology had the highest PVE at 3.9%. Furthermore, high volume of LN metastasis (p = 0.001), extranodal extension (p = 0.047), and high ratio of metastatic LNs (p = 0.006) were associated with poor prognosis. Both traditional microscopic and digital pathology evaluations effectively measured the longest diameter of metastatic foci in LNs. Moreover, digital pathology offers limited advantages in predicting PFS of patients with lateral cervical LN metastasis of PTC, especially those without predominant cystic changes in LNs.

Applications of Artificial Intelligence in Prostate Cancer Care: A Path to Enhanced Efficiency and Outcomes

The landscape of prostate cancer care has rapidly evolved. We have transitioned from the use of conventional imaging, radical surgeries, and single-agent androgen deprivation therapy to an era of advanced imaging, precision diagnostics, genomics, and targeted treatment options. Concurrently, the emergence of large language models (LLMs) has dramatically transformed the paradigm for artificial intelligence (AI). This convergence of advancements in prostate cancer management and AI provides a compelling rationale to comprehensively review the current state of AI applications in prostate cancer care. Here, we review the advancements in AI-driven applications across the continuum of the journey of a patient with prostate cancer from early interception to survivorship care. We subsequently discuss the role of AI in prostate cancer drug discovery, clinical trials, and clinical practice guidelines. In the localized disease setting, deep learning models demonstrated impressive performance in detecting and grading prostate cancer using imaging and pathology data. For biochemically recurrent diseases, machine learning approaches are being tested for improved risk stratification and treatment decisions. In advanced prostate cancer, deep learning can potentially improve prognostication and assist in clinical decision making. Furthermore, LLMs are poised to revolutionize information summarization and extraction, clinical trial design and operations, drug development, evidence synthesis, and clinical practice guidelines. Synergistic integration of multimodal data integration and human-AI integration are emerging as a key strategy to unlock the full potential of AI in prostate cancer care.

Application of Machine Learning in Multimorbidity Research: Protocol for a Scoping Review

BACKGROUND

Multimorbidity, defined as the coexistence of multiple chronic conditions, poses significant challenges to health care systems on a global scale. It is associated with increased mortality, reduced quality of life, and increased health care costs. The burden of multimorbidity is expected to worsen if no effective intervention is taken. Machine learning has the potential to assist in addressing these challenges since it offers advanced analysis and decision-making capabilities, such as disease prediction, treatment development, and clinical strategies.

OBJECTIVE

This paper represents the protocol of a scoping review that aims to identify and explore the current literature concerning the use of machine learning for patients with multimorbidity. More precisely, the objective is to recognize various machine learning models, the patient groups involved, features considered, types of input data, the maturity of the machine learning algorithms, and the outcomes from these machine learning models.

METHODS

The scoping review will be based on the guidelines of the PRISMA-ScR (Preferred Reporting Items for Systematic Reviews and Meta-Analyses Extension for Scoping Reviews). Five databases (PubMed, Embase, IEEE, Web of Science, and Scopus) are chosen to conduct a literature search. Two reviewers will independently screen the titles, abstracts, and full texts of identified studies based on predefined eligibility criteria. Covidence (Veritas Health Innovation Ltd) will be used as a tool for managing and screening papers. Only studies that examine more than 1 chronic disease or individuals with a single chronic condition at risk of developing another will be included in the scoping review. Data from the included studies will be collected using Microsoft Excel (Microsoft Corp). The focus of the data extraction will be on bibliographical information, objectives, study populations, types of input data, types of algorithm, performance, maturity of the algorithms, and outcome.

RESULTS

The screening process will be presented in a PRISMA-ScR flow diagram. The findings of the scoping review will be conveyed through a narrative synthesis. Additionally, data extracted from the studies will be presented in more comprehensive formats, such as charts or tables. The results will be presented in a forthcoming scoping review, which will be published in a peer-reviewed journal.

CONCLUSIONS

To our knowledge, this may be the first scoping review to investigate the use of machine learning in multimorbidity research. The goal of the scoping review is to summarize the field of literature on machine learning in patients with multiple chronic conditions, highlight different approaches, and potentially discover research gaps. The results will offer insights for future research within this field, contributing to developments that can enhance patient outcomes.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

PRR1-10.2196/53761.

Precise and automated lung cancer cell classification using deep neural network with multiscale features and model distillation

Lung diseases globally impose a significant pathological burden and mortality rate, particularly the differential diagnosis between adenocarcinoma, squamous cell carcinoma, and small cell lung carcinoma, which is paramount in determining optimal treatment strategies and improving clinical prognoses. Faced with the challenge of improving diagnostic precision and stability, this study has developed an innovative deep learning-based model. This model employs a Feature Pyramid Network (FPN) and Squeeze-and-Excitation (SE) modules combined with a Residual Network (ResNet18), to enhance the processing capabilities for complex images and conduct multi-scale analysis of each channel's importance in classifying lung cancer. Moreover, the performance of the model is further enhanced by employing knowledge distillation from larger teacher models to more compact student models. Subjected to rigorous five-fold cross-validation, our model outperforms existing models on all performance metrics, exhibiting exceptional diagnostic accuracy. Ablation studies on various model components have verified that each addition effectively improves model performance, achieving an average accuracy of 98.84% and a Matthews Correlation Coefficient (MCC) of 98.83%. Collectively, the results indicate that our model significantly improves the accuracy of disease diagnosis, providing physicians with more precise clinical decision-making support.

Artificial intelligence in digital pathology: a systematic review and meta-analysis of diagnostic test accuracy

Ensuring diagnostic performance of artificial intelligence (AI) before introduction into clinical practice is essential. Growing numbers of studies using AI for digital pathology have been reported over recent years. The aim of this work is to examine the diagnostic accuracy of AI in digital pathology images for any disease. This systematic review and meta-analysis included diagnostic accuracy studies using any type of AI applied to whole slide images (WSIs) for any disease. The reference standard was diagnosis by histopathological assessment and/or immunohistochemistry. Searches were conducted in PubMed, EMBASE and CENTRAL in June 2022. Risk of bias and concerns of applicability were assessed using the QUADAS-2 tool. Data extraction was conducted by two investigators and meta-analysis was performed using a bivariate random effects model, with additional subgroup analyses also performed. Of 2976 identified studies, 100 were included in the review and 48 in the meta-analysis. Studies were from a range of countries, including over 152,000 whole slide images (WSIs), representing many diseases. These studies reported a mean sensitivity of 96.3% (CI 94.1-97.7) and mean specificity of 93.3% (CI 90.5-95.4). There was heterogeneity in study design and 99% of studies identified for inclusion had at least one area at high or unclear risk of bias or applicability concerns. Details on selection of cases, division of model development and validation data and raw performance data were frequently ambiguous or missing. AI is reported as having high diagnostic accuracy in the reported areas but requires more rigorous evaluation of its performance.

Revolutionizing diagnostic pathology: The emergence and impact of artificial intelligence-what doesn't kill you makes you stronger?

This study explored the integration and impact of artificial intelligence (AI) in diagnostic pathology, particularly dermatopathology, assessing its challenges and potential solutions for global health care enhancement. A comprehensive literature search in PubMed and Google Scholar, conducted on March 30, 2023, and using terms related to AI, pathology, and machine learning, yielded 44 relevant publications. These were analyzed under themes including the evolution of deep learning in pathology, AI's role in replacing pathologists, development challenges of diagnostic algorithms, clinical implementation hurdles, strategies for practical application in dermatopathology, and future prospects of AI in this field. The findings highlight AI's transformative potential in pathology, underscore the need for ongoing research, collaboration, and regulatory dialogue, and emphasize the importance of addressing the ethical and practical challenges in AI implementation for improved global health care outcomes.

Pros and cons of artificial intelligence implementation in diagnostic pathology

The rapid introduction of digital pathology has greatly facilitated development of artificial intelligence (AI) models in pathology that have shown great promise in assisting morphological diagnostics and quantitation of therapeutic targets. We are now at a tipping point where companies have started to bring algorithms to the market, and questions arise whether the pathology community is ready to implement AI in routine workflow. However, concerns also arise about the use of AI in pathology. This article reviews the pros and cons of introducing AI in diagnostic pathology.

DRA-Net: Medical image segmentation based on adaptive feature extraction and region-level information fusion

Medical image segmentation is a key task in computer aided diagnosis. In recent years, convolutional neural network (CNN) has made some achievements in medical image segmentation. However, the convolution operation can only extract features in a fixed size region at a time, which leads to the loss of some key features. The recently popular Transformer has global modeling capabilities, but it does not pay enough attention to local information and cannot accurately segment the edge details of the target area. Given these issues, we proposed dynamic regional attention network (DRA-Net). Different from the above methods, it first measures the similarity of features and concentrates attention on different dynamic regions. In this way, the network can adaptively select different modeling scopes for feature extraction, reducing information loss. Then, regional feature interaction is carried out to better learn local edge details. At the same time, we also design ordered shift multilayer perceptron (MLP) blocks to enhance communication within different regions, further enhancing the network's ability to learn local edge details. After several experiments, the results indicate that our network produces more accurate segmentation performance compared to other CNN and Transformer based networks.

Predicting rectal cancer prognosis from histopathological images and clinical information using multi-modal deep learning

Objective

The objective of this study was to provide a multi-modal deep learning framework for forecasting the survival of rectal cancer patients by utilizing both digital pathological images data and non-imaging clinical data.

Materials and methods

The research included patients diagnosed with rectal cancer by pathological confirmation from January 2015 to December 2016. Patients were allocated to training and testing sets in a randomized manner, with a ratio of 4:1. The tissue microarrays (TMAs) and clinical indicators were obtained. Subsequently, we selected distinct deep learning models to individually forecast patient survival. We conducted a scanning procedure on the TMAs in order to transform them into digital pathology pictures. Additionally, we performed pre-processing on the clinical data of the patients. Subsequently, we selected distinct deep learning algorithms to conduct survival prediction analysis using patients' pathological images and clinical data, respectively.

Results

A total of 292 patients with rectal cancer were randomly allocated into two groups: a training set consisting of 234 cases, and a testing set consisting of 58 instances. Initially, we make direct predictions about the survival status by using pre-processed Hematoxylin and Eosin (H&E) pathological images of rectal cancer. We utilized the ResNest model to extract data from histopathological images of patients, resulting in a survival status prediction with an AUC (Area Under the Curve) of 0.797. Furthermore, we employ a multi-head attention fusion (MHAF) model to combine image features and clinical features in order to accurately forecast the survival rate of rectal cancer patients. The findings of our experiment show that the multi-modal structure works better than directly predicting from histopathological images. It achieves an AUC of 0.837 in predicting overall survival (OS).

Conclusions

Our study highlights the potential of multi-modal deep learning models in predicting survival status from histopathological images and clinical information, thus offering valuable insights for clinical applications.

DiagSet: a dataset for prostate cancer histopathological image classification

Cancer diseases constitute one of the most significant societal challenges. In this paper, we introduce a novel histopathological dataset for prostate cancer detection. The proposed dataset, consisting of over 2.6 million tissue patches extracted from 430 fully annotated scans, 4675 scans with assigned binary diagnoses, and 46 scans with diagnoses independently provided by a group of histopathologists can be found at https://github.com/michalkoziarski/DiagSet . Furthermore, we propose a machine learning framework for detection of cancerous tissue regions and prediction of scan-level diagnosis, utilizing thresholding to abstain from the decision in uncertain cases. The proposed approach, composed of ensembles of deep neural networks operating on the histopathological scans at different scales, achieves 94.6% accuracy in patch-level recognition and is compared in a scan-level diagnosis with 9 human histopathologists showing high statistical agreement.

Highly accurate and effective deep neural networks in pathological diagnosis of prostate cancer

PURPOSE

To established an AI system to make the pathological diagnosis of prostate cancer.

METHODS

Prostate histopathological whole mount (WM) sections from patients underwent robot-assisted laparoscopic prostatectomy were prepared. All the prostate WM pathological sections were converted to digital image data and marked with different colors on the basis of the ISUP Gleason grade group. The image was then fed into a segmentation algorithm. We chose modified U-Net as our fundamental network architecture.

RESULTS

172 patients were involved in this study. 896 pieces of prostate WM pathological sections from 160 patients, in which 826 pieces of WM sections from 148 patients were assigned to the training set randomly. After image segmentation there were totally 2,138,895 patches, of which 1,646,535 patches were valid for training. The other WM section was arranged for testing. Based on the whole image testing, AI and pathologists presented the same answers among 21 of 22 pieces of sections. To evaluate the diagnostic results at the pixel level, we anticipated correct cancer or non-cancer diagnose from this AI system. The area under the ROC curve as 96.8%. The value of pixel accuracy of three methods (binary analysis, clinically oriented analysis and analysis for different ISUP Gleason grade) were 96.93%, 95.43% and 93.88%, respectively. The value of frequency weighted IoU were 94.32%, 92.13% and 90.21%, respectively.

CONCLUSIONS

This AI system is able to assist pathologists to make a final diagnosis, indicating the great potential and a wide-range of applications of AI in the medical field.

Transfer Learning with Pretrained Convolutional Neural Network for Automated Gleason Grading of Prostate Cancer Tissue Microarrays

Background

The Gleason grading system has been the most effective prediction for prostate cancer patients. This grading system provides this possibility to assess prostate cancer's aggressiveness and then constitutes an important factor for stratification and therapeutic decisions. However, determining Gleason grade requires highly-trained pathologists and is time-consuming and tedious, and suffers from inter-pathologist variability. To remedy these limitations, this paper introduces an automatic methodology based on transfer learning with pretrained convolutional neural networks (CNNs) for automatic Gleason grading of prostate cancer tissue microarray (TMA).

Methods

Fifteen pretrained (CNNs): Efficient Nets (B0-B5), NasNetLarge, NasNetMobile, InceptionV3, ResNet-50, SeResnet 50, Xception, DenseNet121, ResNext50, and inception_resnet_v2 were fine-tuned on a dataset of prostate carcinoma TMA images. Six pathologists separately identified benign and cancerous areas for each prostate TMA image by allocating benign, 3, 4, or 5 Gleason grade for 244 patients. The dataset was labeled by these pathologists and majority vote was applied on pixel-wise annotations to obtain a unified label.

Results

Results showed the NasnetLarge architecture is the best model among them in the classification of prostate TMA images of 244 patients with accuracy of 0.93 and area under the curve of 0.98.

Conclusion

Our study can act as a highly trained pathologist to categorize the prostate cancer stages with more objective and reproducible results.

Applications of artificial intelligence in the field of oral and maxillofacial pathology: a systematic review and meta-analysis

BACKGROUND

Since AI algorithms can analyze patient data, medical records, and imaging results to suggest treatment plans and predict outcomes, they have the potential to support pathologists and clinicians in the diagnosis and treatment of oral and maxillofacial pathologies, just like every other area of life in which it is being used. The goal of the current study was to examine all of the trends being investigated in the area of oral and maxillofacial pathology where AI has been possibly involved in helping practitioners.

METHODS

We started by defining the important terms in our investigation's subject matter. Following that, relevant databases like PubMed, Scopus, and Web of Science were searched using keywords and synonyms for each concept, such as "machine learning," "diagnosis," "treatment planning," "image analysis," "predictive modelling," and "patient monitoring." For more papers and sources, Google Scholar was also used.

RESULTS

The majority of the 9 studies that were chosen were on how AI can be utilized to diagnose malignant tumors of the oral cavity. AI was especially helpful in creating prediction models that aided pathologists and clinicians in foreseeing the development of oral and maxillofacial pathology in specific patients. Additionally, predictive models accurately identified patients who have a high risk of developing oral cancer as well as the likelihood of the disease returning after treatment.

CONCLUSIONS

In the field of oral and maxillofacial pathology, AI has the potential to enhance diagnostic precision, personalize care, and ultimately improve patient outcomes. The development and application of AI in healthcare, however, necessitates careful consideration of ethical, legal, and regulatory challenges. Additionally, because AI is still a relatively new technology, caution must be taken when applying it to this industry.

STS-TransUNet: Semi-supervised Tooth Segmentation Transformer U-Net for dental panoramic image

In this paper, we introduce a novel deep learning method for dental panoramic image segmentation, which is crucial in oral medicine and orthodontics for accurate diagnosis and treatment planning. Traditional methods often fail to effectively combine global and local context, and struggle with unlabeled data, limiting performance in varied clinical settings. We address these issues with an advanced TransUNet architecture, enhancing feature retention and utilization by connecting the input and output layers directly. Our architecture further employs spatial and channel attention mechanisms in the decoder segments for targeted region focus, and deep supervision techniques to overcome the vanishing gradient problem for more efficient training. Additionally, our network includes a self-learning algorithm using unlabeled data, boosting generalization capabilities. Named the Semi-supervised Tooth Segmentation Transformer U-Net (STS-TransUNet), our method demonstrated superior performance on the MICCAI STS-2D dataset, proving its effectiveness and robustness in tooth segmentation tasks.

Shedding light on the black box of a neural network used to detect prostate cancer in whole slide images by occlusion-based explainability

Diagnostic histopathology faces increasing demands due to aging populations and expanding healthcare programs. Semi-automated diagnostic systems employing deep learning methods are one approach to alleviate this pressure. The learning models for histopathology are inherently complex and opaque from the user's perspective. Hence different methods have been developed to interpret their behavior. However, relatively limited attention has been devoted to the connection between interpretation methods and the knowledge of experienced pathologists. The main contribution of this paper is a method for comparing morphological patterns used by expert pathologists to detect cancer with the patterns identified as important for inference of learning models. Given the patch-based nature of processing large-scale histopathological imaging, we have been able to show statistically that the VGG16 model could utilize all the structures that are observable by the pathologist, given the patch size and scan resolution. The results show that the neural network approach to recognizing prostatic cancer is similar to that of a pathologist at medium optical resolution. The saliency maps identified several prevailing histomorphological features characterizing carcinoma, e.g., single-layered epithelium, small lumina, and hyperchromatic nuclei with halo. A convincing finding was the recognition of their mimickers in non-neoplastic tissue. The method can also identify differences, i.e., standard patterns not used by the learning models and new patterns not yet used by pathologists. Saliency maps provide added value for automated digital pathology to analyze and fine-tune deep learning systems and improve trust in computer-based decisions.

Binary semantic segmentation for detection of prostate adenocarcinoma using an ensemble with attention and residual U-Net architectures

An accurate determination of the Gleason Score (GS) or Gleason Pattern (GP) is crucial in the diagnosis of prostate cancer (PCa) because it is one of the criterion used to guide treatment decisions for prognostic-risk groups. However, the manually designation of GP by a pathologist using a microscope is prone to error and subject to significant inter-observer variability. Deep learning has been used to automatically differentiate GP on digitized slides, aiding pathologists and reducing inter-observer variability, especially in the early GP of cancer. This article presents a binary semantic segmentation for the GP of prostate adenocarcinoma. The segmentation separates benign and malignant tissues, with the malignant class consisting of adenocarcinoma GP3 and GP4 tissues annotated from 50 unique digitized whole slide images (WSIs) of prostate needle core biopsy specimens stained with hematoxylin and eosin. The pyramidal digitized WSIs were extracted into image patches with a size of 256 × 256 pixels at a magnification of 20×. An ensemble approach is proposed combining U-Net-based architectures, including traditional U-Net, attention-based U-Net, and residual attention-based U-Net. This work initially considers a PCa tissue analysis using a combination of attention gate units with residual convolution units. The performance evaluation revealed a mean Intersection-over-Union of 0.79 for the two classes, 0.88 for the benign class, and 0.70 for the malignant class. The proposed method was then used to produce pixel-level segmentation maps of PCa adenocarcinoma tissue slides in the testing set. We developed a screening tool to discriminate between benign and malignant prostate tissue in digitized images of needle biopsy samples using an AI approach. We aimed to identify malignant adenocarcinoma tissues from our own collected, annotated, and organized dataset. Our approach returned the performance which was accepted by the pathologists.

Immunohistochemical Stain-Aided Annotation Accelerates Machine Learning and Deep Learning Model Development in the Pathologic Diagnosis of Nasopharyngeal Carcinoma

Nasopharyngeal carcinoma (NPC) is an epithelial cancer originating in the nasopharynx epithelium. Nevertheless, annotating pathology slides remains a bottleneck in the development of AI-driven pathology models and applications. In the present study, we aim to demonstrate the feasibility of using immunohistochemistry (IHC) for annotation by non-pathologists and to develop an efficient model for distinguishing NPC without the time-consuming involvement of pathologists. For this study, we gathered NPC slides from 251 different patients, comprising hematoxylin and eosin (H&E) slides, pan-cytokeratin (Pan-CK) IHC slides, and Epstein-Barr virus-encoded small RNA (EBER) slides. The annotation of NPC regions in the H&E slides was carried out by a non-pathologist trainee who had access to corresponding Pan-CK IHC slides, both with and without EBER slides. The training process utilized ResNeXt, a deep neural network featuring a residual and inception architecture. In the validation set, NPC exhibited an AUC of 0.896, with a sensitivity of 0.919 and a specificity of 0.878. This study represents a significant breakthrough: the successful application of deep convolutional neural networks to identify NPC without the need for expert pathologist annotations. Our results underscore the potential of laboratory techniques to substantially reduce the workload of pathologists.

Application of cloud server-based machine learning for assisting pathological structure recognition in IgA nephropathy

BACKGROUND

Machine learning (ML) models can help assisting diagnosis by rapidly localising and classifying regions of interest (ROIs) within whole slide images (WSIs). Effective ML models for clinical decision support require a substantial dataset of 'real' data, and in reality, it should be robust, user-friendly and universally applicable.

METHODS

WSIs of primary IgAN were collected and annotated. The H-AI-L algorithm which could facilitate direct WSI viewing and potential ROI detection for clinicians was built on the cloud server of matpool, a shared internet-based service platform. Model performance was evaluated using F1-score, precision, recall and Matthew's correlation coefficient (MCC).

RESULTS

The F1-score of glomerular localisation in WSIs was 0.85 and 0.89 for the initial and pretrained models, respectively, with corresponding recall values of 0.79 and 0.83, and precision scores of 0.92 and 0.97. Dichotomous differentiation between global sclerotic (GS) and other glomeruli revealed F1-scores of 0.70 and 0.91, and MCC values of 0.55 and 0.87, for the initial and pretrained models, respectively. The overall F1-score of multiclassification was 0.81 for the pretrained models. The total glomerular recall rate was 0.96, with F1-scores of 0.68, 0.56 and 0.26 for GS, segmental glomerulosclerosis and crescent (C), respectively. Interstitial fibrosis/tubular atrophy lesion similarity between the true label and model predictions was 0.75.

CONCLUSIONS

Our results underscore the efficacy of the ML integration algorithm in segmenting ROIs in IgAN WSIs, and the internet-based model deployment is in favour of widespread adoption and utilisation across multiple centres and increased volumes of WSIs.

Advancing Colorectal Cancer Diagnosis with AI-Powered Breathomics: Navigating Challenges and Future Directions

Early detection of colorectal cancer is crucial for improving outcomes and reducing mortality. While there is strong evidence of effectiveness, currently adopted screening methods present several shortcomings which negatively impact the detection of early stage carcinogenesis, including low uptake due to patient discomfort. As a result, developing novel, non-invasive alternatives is an important research priority. Recent advancements in the field of breathomics, the study of breath composition and analysis, have paved the way for new avenues for non-invasive cancer detection and effective monitoring. Harnessing the utility of Volatile Organic Compounds in exhaled breath, breathomics has the potential to disrupt colorectal cancer screening practices. Our goal is to outline key research efforts in this area focusing on machine learning methods used for the analysis of breathomics data, highlight challenges involved in artificial intelligence application in this context, and suggest possible future directions which are currently considered within the framework of the European project ONCOSCREEN.

Artificial intelligence in neuro-oncology

Artificial intelligence (AI) describes the application of computer algorithms to the solution of problems that have traditionally required human intelligence. Although formal work in AI has been slowly advancing for almost 70 years, developments in the last decade, and particularly in the last year, have led to an explosion of AI applications in multiple fields. Neuro-oncology has not escaped this trend. Given the expected integration of AI-based methods to neuro-oncology practice over the coming years, we set to provide an overview of existing technologies as they are applied to the neuropathology and neuroradiology of brain tumors. We highlight current benefits and limitations of these technologies and offer recommendations on how to appraise novel AI-tools as they undergo consideration for integration into clinical workflows.

AI in Rehabilitation Medicine: Opportunities and Challenges

Artificial intelligence (AI) tools are increasingly able to learn from larger and more complex data, thus allowing clinicians and scientists to gain new insights from the information they collect about their patients every day. In rehabilitation medicine, AI can be used to find patterns in huge amounts of healthcare data. These patterns can then be leveraged at the individual level, to design personalized care strategies and interventions to optimize each patient's outcomes. However, building effective AI tools requires many careful considerations about how we collect and handle data, how we train the models, and how we interpret results. In this perspective, we discuss some of the current opportunities and challenges for AI in rehabilitation. We first review recent trends in AI for the screening, diagnosis, treatment, and continuous monitoring of disease or injury, with a special focus on the different types of healthcare data used for these applications. We then examine potential barriers to designing and integrating AI into the clinical workflow, and we propose an end-to-end framework to address these barriers and guide the development of effective AI for rehabilitation. Finally, we present ideas for future work to pave the way for AI implementation in real-world rehabilitation practices.

Machine Learning in Invasive and Noninvasive Coronary Angiography

PURPOSE OF REVIEW

The objective of this review is to shed light on the transformative potential of machine learning (ML) in coronary angiography. We aim to understand existing developments in using ML for coronary angiography and discuss broader implications for the future of coronary angiography and cardiovascular medicine.

RECENT FINDINGS

The developments in invasive and noninvasive imaging have revolutionized diagnosis and treatment of coronary artery disease (CAD). However, CAD remains underdiagnosed and undertreated. ML has emerged as a powerful tool to further improve image analysis, hemodynamic assessment, lesion detection, and predictive modeling. These advancements have enabled more accurate identification of CAD, streamlined workflows, reduced the need for invasive diagnostic procedures, and improved the diagnostic value of invasive procedures when they are needed. Further integration of ML with coronary angiography will advance the prevention, diagnosis, and treatment of CAD. The integration of ML with coronary angiography is ushering in a new era in cardiovascular medicine. We highlight five use cases to leverage ML in coronary angiography: (1) improvement of quality and efficacy, (2) characterization of plaque, (3) hemodynamic assessment, (4) prediction of future outcomes, and (5) diagnosis of non-atherosclerotic coronary disease.

Quantitative interpretation of bone marrow biopsies in MPN-What's the point in a molecular age?

The diagnosis of myeloproliferative neoplasms (MPN) requires the integration of clinical, morphological, genetic and immunophenotypic findings. Recently, there has been a transformation in our understanding of the cellular and molecular mechanisms underlying disease initiation and progression in MPN. This has been accompanied by the widespread application of high-resolution quantitative molecular techniques. By contrast, microscopic interpretation of bone marrow biopsies by haematologists/haematopathologists remains subjective and qualitative. However, advances in tissue image analysis and artificial intelligence (AI) promise to transform haematopathology. Pioneering studies in bone marrow image analysis offer to refine our understanding of the boundaries between reactive samples and MPN subtypes and better capture the morphological correlates of high-risk disease. They also demonstrate potential to improve the evaluation of current and novel therapeutics for MPN and other blood cancers. With increased therapeutic targeting of diverse molecular, cellular and extra-cellular components of the marrow, these approaches can address the unmet need for improved objective and quantitative measures of disease modification in the context of clinical trials. This review focuses on the state-of-the-art in image analysis/AI of bone marrow tissue, with an emphasis on its potential to complement and inform future clinical studies and research in MPN.

The Utility of Artificial Intelligence in the Diagnosis and Management of Pancreatic Cancer

Artificial intelligence (AI) has made significant advancements in the medical domain in recent years. AI, an expansive field comprising Machine Learning (ML) and, within it, Deep Learning (DL), seeks to emulate the intricate operations of the human brain. It examines vast amounts of data and plays a crucial role in decision-making, overcoming limitations related to human evaluation. DL utilizes complex algorithms to analyze data. ML and DL are subsets of AI that utilize hard statistical techniques that help machines consistently improve at tasks with experience. Pancreatic cancer is more common in developed countries and is one of the leading causes of cancer-related mortality worldwide. Managing pancreatic cancer remains a challenge despite significant advancements in diagnosis and treatment. AI has secured an almost ubiquitous presence in the field of oncological workup and management, especially in gastroenterology malignancies. AI is particularly useful for various investigations of pancreatic carcinoma because it has specific radiological features that enable diagnostic procedures without the requirement of a histological study. However, interpreting and evaluating resulting images is not always simple since images vary as the disease progresses. Secondly, a number of factors may impact prognosis and response to the treatment process. Currently, AI models have been created for diagnosing, grading, staging, and predicting prognosis and treatment response. This review presents the most up-to-date knowledge on the use of AI in the diagnosis and treatment of pancreatic carcinoma.

A Survey on Artificial Intelligence in Pulmonary Imaging

Over the last decade, deep learning (DL) has contributed a paradigm shift in computer vision and image recognition creating widespread opportunities of using artificial intelligence in research as well as industrial applications. DL has been extensively studied in medical imaging applications, including those related to pulmonary diseases. Chronic obstructive pulmonary disease, asthma, lung cancer, pneumonia, and, more recently, COVID-19 are common lung diseases affecting nearly 7.4% of world population. Pulmonary imaging has been widely investigated toward improving our understanding of disease etiologies and early diagnosis and assessment of disease progression and clinical outcomes. DL has been broadly applied to solve various pulmonary image processing challenges including classification, recognition, registration, and segmentation. This paper presents a survey of pulmonary diseases, roles of imaging in translational and clinical pulmonary research, and applications of different DL architectures and methods in pulmonary imaging with emphasis on DL-based segmentation of major pulmonary anatomies such as lung volumes, lung lobes, pulmonary vessels, and airways as well as thoracic musculoskeletal anatomies related to pulmonary diseases.

Comparison of Dynamic Contrast-Enhanced MRI and Non-Mono-Exponential Model-Based Diffusion-Weighted Imaging for the Prediction of Prognostic Biomarkers and Molecular Subtypes of Breast Cancer Based on Radiomics

BACKGROUND

Dynamic contrast-enhanced (DCE) MRI and non-mono-exponential model-based diffusion-weighted imaging (NME-DWI) that does not require contrast agent can both characterize breast cancer. However, which technique is superior remains unclear.

PURPOSE

To compare the performances of DCE-MRI, NME-DWI and their combination as multiparametric MRI (MP-MRI) in the prediction of breast cancer prognostic biomarkers and molecular subtypes based on radiomics.

STUDY TYPE

Prospective.

POPULATION

A total of 477 female patients with 483 breast cancers (5-fold cross-validation: training/validation, 80%/20%).

FIELD STRENGTH/SEQUENCE

A 3.0 T/DCE-MRI (6 dynamic frames) and NME-DWI (13 b values).

ASSESSMENT

After data preprocessing, high-throughput features were extracted from each tumor volume of interest, and optimal features were selected using recursive feature elimination method. To identify ER+ vs. ER-, PR+ vs. PR-, HER2+ vs. HER2-, Ki-67+ vs. Ki-67-, luminal A/B vs. nonluminal A/B, and triple negative (TN) vs. non-TN, the following models were implemented: random forest, adaptive boosting, support vector machine, linear discriminant analysis, and logistic regression.

STATISTICAL TESTS

Student's t, chi-square, and Fisher's exact tests were applied on clinical characteristics to confirm whether significant differences exist between different statuses (±) of prognostic biomarkers or molecular subtypes. The model performances were compared between the DCE-MRI, NME-DWI, and MP-MRI datasets using the area under the receiver-operating characteristic curve (AUC) and the DeLong test. P < 0.05 was considered significant.

RESULTS

With few exceptions, no significant differences (P = 0.062-0.984) were observed in the AUCs of models for six classification tasks between the DCE-MRI (AUC = 0.62-0.87) and NME-DWI (AUC = 0.62-0.91) datasets, while the model performances on the two imaging datasets were significantly poorer than on the MP-MRI dataset (AUC = 0.68-0.93). Additionally, the random forest and adaptive boosting models (AUC = 0.62-0.93) outperformed other three models (AUC = 0.62-0.90).

DATA CONCLUSION

NME-DWI was comparable with DCE-MRI in predictive performance and could be used as an alternative technique. Besides, MP-MRI demonstrated significantly higher AUCs than either DCE-MRI or NME-DWI.

EVIDENCE LEVEL

2.

TECHNICAL EFFICACY

Stage 2.

Artificial intelligence in breast cancer: application and future perspectives

Breast cancer is one of the most common cancers and is one of the leading causes of cancer-related deaths in women worldwide. Early diagnosis and treatment are the key for a favorable prognosis. The application of artificial intelligence technology in the medical field is increasingly extensive, including image analysis, automated diagnosis, intelligent pharmaceutical system, personalized treatment and so on. AI-based breast cancer imaging, pathology and adjuvant therapy technology cannot only reduce the workload of clinicians, but also continuously improve the accuracy and sensitivity of breast cancer diagnosis and treatment. This paper reviews the application of AI in breast cancer, as well as looks ahead and poses challenges to the future development of AI for breast cancer detection and therapeutic, so as to provide ideas for future research.

Prognostic significance of tumor microenvironment assessed by machine learning algorithm in surgically resected non-small cell lung cancer

BACKGROUND

A methodology to assess the immune microenvironment (IME) of non-small cell lung cancer (NSCLC) has not been established, and the prognostic impact of IME factors is not yet clear.

AIMS

This study aimed to assess the IME factors and evaluate their prognostic values.

METHODS AND RESULTS

We assessed CD8+ tumor-infiltrating lymphocyte (TIL) density, forkhead box protein P3+ (Foxp3+ ) TIL density, and programmed death receptor ligand-1 (PD-L1) tumor proportion score (TPS) using a machine-learning algorithm in whole-slide imaging (WSI). We dichotomized patients according to TIL density or TPS and compared their clinical outcomes. Between September 2014 and September 2015, 165 patients with NSCLC were enrolled in the study. We assessed IME factors in the epithelium, stroma, and their combination. An improvement in disease-free survival (DFS) was observed in the high CD8+ TIL density group in the epithelium, stroma, and the combination of both. Moreover, the group with high PD-L1 TPS in the epithelium showed better DFS than that with low PD-L1 TPS. In the multivariate analysis, the CD8+ TIL density in the combination of epithelium and stroma and PD-L1 TPS in the epithelium were independent prognostic factors (hazard ratio [HR] = 0.43; 95% confidence interval [CI] = 0.26-0.72; p = .001, HR = 0.49; 95% CI = 0.30-0.81; p = .005, respectively).

CONCLUSION

Our approach demonstrated that the IME factors are related to survival in patients with NSCLC. The quantitative assessment of IME factors enables to discriminate patients with high risk of recurrence, who can be the candidates for adjuvant therapy. Assessing the CD8+ TIL density in the combination of epithelium and stroma might be more useful than their individual assessment because it is a simple and time-saving analysis of TILs in WSI.

Adapted Deep Ensemble Learning-Based Voting Classifier for Osteosarcoma Cancer Classification

The study utilizes osteosarcoma hematoxylin and the Eosin-stained image dataset, which is unevenly dispersed, and it raises concerns about the potential impact on the overall performance and reliability of any analyses or models derived from the dataset. In this study, a deep-learning-based convolution neural network (CNN) and adapted heterogeneous ensemble-learning-based voting classifier have been proposed to classify osteosarcoma. The proposed methods can also resolve the issue and develop unbiased learning models by introducing an evenly distributed training dataset. Data augmentation is employed to boost the generalization abilities. Six different pre-trained CNN models, namely MobileNetV1, Mo-bileNetV2, ResNetV250, InceptionV2, EfficientNetV2B0, and NasNetMobile, are applied and evaluated in frozen and fine-tuned-based phases. In addition, a novel CNN model and adapted heterogeneous ensemble-learning-based voting classifier developed from the proposed CNN model, fine-tuned NasNetMobile model, and fine-tuned Efficient-NetV2B0 model are also introduced to classify osteosarcoma. The proposed CNN model outperforms other pre-trained models. The Kappa score obtained from the proposed CNN model is 93.09%. Notably, the proposed voting classifier attains the highest Kappa score of 96.50% and outperforms all other models. The findings of this study have practical implications in telemedicine, mobile healthcare systems, and as a supportive tool for medical professionals.

Clinical Validation of Artificial Intelligence-Augmented Pathology Diagnosis Demonstrates Significant Gains in Diagnostic Accuracy in Prostate Cancer Detection

CONTEXT.—

Prostate cancer diagnosis rests on accurate assessment of tissue by a pathologist. The application of artificial intelligence (AI) to digitized whole slide images (WSIs) can aid pathologists in cancer diagnosis, but robust, diverse evidence in a simulated clinical setting is lacking.

OBJECTIVE.—

To compare the diagnostic accuracy of pathologists reading WSIs of prostatic biopsy specimens with and without AI assistance.

DESIGN.—

Eighteen pathologists, 2 of whom were genitourinary subspecialists, evaluated 610 prostate needle core biopsy WSIs prepared at 218 institutions, with the option for deferral. Two evaluations were performed sequentially for each WSI: initially without assistance, and immediately thereafter aided by Paige Prostate (PaPr), a deep learning-based system that provides a WSI-level binary classification of suspicious for cancer or benign and pinpoints the location that has the greatest probability of harboring cancer on suspicious WSIs. Pathologists' changes in sensitivity and specificity between the assisted and unassisted modalities were assessed, together with the impact of PaPr output on the assisted reads.

RESULTS.—

Using PaPr, pathologists improved their sensitivity and specificity across all histologic grades and tumor sizes. Accuracy gains on both benign and cancerous WSIs could be attributed to PaPr, which correctly classified 100% of the WSIs showing corrected diagnoses in the PaPr-assisted phase.

CONCLUSIONS.—

This study demonstrates the effectiveness and safety of an AI tool for pathologists in simulated diagnostic practice, bridging the gap between computational pathology research and its clinical application, and resulted in the first US Food and Drug Administration authorization of an AI system in pathology.

Deep learning of renal scans in children with antenatal hydronephrosis

INTRODUCTION

Antenatal hydronephrosis (ANH) is one of the most common anomalies identified on prenatal ultrasound, found in up to 4.5% of all pregnancies. Children with ANH are surveilled with repeated renal ultrasound and when there is high suspicion for a ureteropelvic junction obstruction on renal ultrasound, a mercaptuacetyltriglycerine (MAG3) Lasix renal scan is performed to evaluate for obstruction. However, the challenging interpretation of MAG3 renal scans places patients at risk of misdiagnosis.

OBJECTIVE

Our objective was to analyze MAG3 renal scans using machine learning to predict renal complications. We hypothesized that our deep learning model would extract features from MAG3 renal scans that can predict renal complications in children with ANH.

STUDY DESIGN

We performed a case-control study of MAG3 studies drawn from a population of children with ANH concerning for ureteropelvic junction obstruction evaluated at our institution from January 2009 until June of 2021. The outcome was renal complications that occur ≥6 months after an equivocal MAG-3 renal scan. We created two machine learning models: a deep learning model using the radiotracer concentration versus time data from the kidney of interest and a random forest model created using clinical data. The performance of the models was assessed using measures of diagnostic accuracy.

RESULTS

We identified 152 eligible patients with available images of which 62 were cases and 90 were controls. The deep learning model predicted future renal complications with an overall accuracy of 73% (95% confidence inteveral [CI] 68-76%) and an AUC of 0.78 (95% CI 0.7, 0.84). The random forest model had an accuracy of 62% (95% CI 60-66%) and an AUC of 0.67 (95% CI. 0 64, 0.72) DISCUSSION: Our deep learning model predicted patients at high risk of developing renal complications following an equivocal renal scan and discriminate those at low risk with moderately high accuracy (73%). The deep learning model outperformed the clinical model built from clinical features classically used by urologists for surgical decision making.

CONCLUSION

Our models have the potential to influence clinical decision making by providing supplemental analytical data from MAG3 scans that would not otherwise be available to urologists. Future multi-institutional retrospective and prospective trials are needed to validate our model.