Deep learning systems detect dysplasia with human-like accuracy using histopathology and probe-based confocal laser endomicroscopy

Machine learning approaches to evaluate infants' general movements in the writhing stage-a pilot study

The goals of this study are to describe machine learning techniques employing computer-vision movement algorithms to automatically evaluate infants' general movements (GMs) in the writhing stage. This is a retrospective study of infants admitted 07/2019 to 11/2021 to a level IV neonatal intensive care unit (NICU). Infant GMs, classified by certified expert, were analyzed in two-steps (1) determination of anatomic key point location using a NICU-trained pose estimation model [accuracy determined using object key point similarity (OKS)]; (2) development of a preliminary movement model to distinguish normal versus cramped-synchronized (CS) GMs using cosine similarity and autocorrelation of major joints. GMs were analyzed using 85 videos from 74 infants; gestational age at birth 28.9 ± 4.1 weeks and postmenstrual age (PMA) at time of video 35.9 ± 4.6 weeks The NICU-trained pose estimation model was more accurate (0.91 ± 0.008 OKS) than a generic model (0.83 ± 0.032 OKS, p < 0.001). Autocorrelation values in the lower limbs were significantly different between normal (5 videos) and CS GMs (5 videos, p < 0.05). These data indicate that automated pose estimation of anatomical key points is feasible in NICU patients and that a NICU-trained model can distinguish between normal and CS GMs. These preliminary data indicate that machine learning techniques may represent a promising tool for earlier CP risk assessment in the writhing stage and prior to hospital discharge.

Machine Learning-Based Prediction of Responsiveness to Neoadjuvant Chemoradiotheapy in Locally Advanced Rectal Cancer Patients from Endomicroscopy

The protocol for treating locally advanced rectal cancer consists of the application of chemoradiotherapy (neoCRT) followed by surgical intervention. One issue for clinical oncologists is predicting the efficacy of neoCRT in order to adjust the dosage and avoid treatment toxicity in cases when surgery should be conducted promptly. Biomarkers may be used for this purpose along with in vivo cell-level images of the colorectal mucosa obtained by probe-based confocal laser endomicroscopy (pCLE) during colonoscopy. The aim of this article is to report our experience with Motiro, a computational framework that we developed for machine learning (ML) based analysis of pCLE videos for predicting neoCRT response in locally advanced rectal cancer patients. pCLE videos were collected from 47 patients who were diagnosed with locally advanced rectal cancer (T3/T4, or N+). The patients received neoCRT. Response to treatment by all patients was assessed by endoscopy along with biopsy and magnetic resonance imaging (MRI). Thirty-seven patients were classified as non-responsive to neoCRT because they presented a visible macroscopic neoplastic lesion, as confirmed by pCLE examination. Ten remaining patients were considered responsive to neoCRT because they presented lesions as a scar or small ulcer with negative biopsy, at post-treatment follow-up. Motiro was used for batch mode analysis of pCLE videos. It automatically characterized the tumoral region and its surroundings. That enabled classifying a patient as responsive or non-responsive to neoCRT based on pre-neoCRT pCLE videos. Motiro classified patients as responsive or non-responsive to neoCRT with an accuracy of ~ 0.62 when using images of the tumor. When using images of regions surrounding the tumor, it reached an accuracy of ~ 0.70. Feature analysis showed that spatial heterogeneity in fluorescence distribution within regions surrounding the tumor was the main contributor to predicting response to neoCRT. We developed a computational framework to predict response to neoCRT by locally advanced rectal cancer patients based on pCLE images acquired pre-neoCRT. We demonstrate that the analysis of the mucosa of the region surrounding the tumor provides stronger predictive power.

Deep Learning in Barrett's Esophagus Diagnosis: Current Status and Future Directions

Barrett's esophagus (BE) represents a pre-malignant condition characterized by abnormal cellular proliferation in the distal esophagus. A timely and accurate diagnosis of BE is imperative to prevent its progression to esophageal adenocarcinoma, a malignancy associated with a significantly reduced survival rate. In this digital age, deep learning (DL) has emerged as a powerful tool for medical image analysis and diagnostic applications, showcasing vast potential across various medical disciplines. In this comprehensive review, we meticulously assess 33 primary studies employing varied DL techniques, predominantly featuring convolutional neural networks (CNNs), for the diagnosis and understanding of BE. Our primary focus revolves around evaluating the current applications of DL in BE diagnosis, encompassing tasks such as image segmentation and classification, as well as their potential impact and implications in real-world clinical settings. While the applications of DL in BE diagnosis exhibit promising results, they are not without challenges, such as dataset issues and the "black box" nature of models. We discuss these challenges in the concluding section. Essentially, while DL holds tremendous potential to revolutionize BE diagnosis, addressing these challenges is paramount to harnessing its full capacity and ensuring its widespread application in clinical practice.

Artificial Intelligence in the Detection of Barrett's Esophagus: A Systematic Review

Barrett's esophagus (BE) remains a significant precursor to esophageal adenocarcinoma, requiring accurate and efficient diagnosis and management. The increasing application of machine learning (ML) technologies presents a transformative opportunity for diagnosing and treating BE. This systematic review evaluates the effectiveness and accuracy of machine learning technologies in BE diagnosis and management by conducting a comprehensive search across PubMed, Scopus, and Web of Science databases up to the year 2023. The studies were organized into five categories: computer-aided systems, natural language processing and text-based systems, deep learning on histology and biopsy images, real-time and video analysis, and miscellaneous studies. Results indicate high sensitivity and specificity across machine learning applications. Specifically, computer-aided systems showed sensitivities ranging from 84% to 100% and specificities from 64% to 90.7%. Natural language processing and text-based systems achieved an accuracy as high as 98.7%. Deep learning techniques applied to histology and biopsy images displayed sensitivities up to greater than 90% and a specificity of 100%. Furthermore, real-time and video analysis technologies demonstrated high performance with assessment speeds of up to 48 frames per second (fps) and a mean average precision of 75.3%. Overall, the reviewed literature underscores the growing capability and efficiency of machine learning technologies in diagnosing and managing Barrett's esophagus, often outperforming traditional diagnostic methods. These findings highlight the promising future role of machine learning in enhancing clinical practice and improving patient care for Barrett's esophagus.

The Multifaceted Role of Confocal Laser Endomicroscopy in Head and Neck Surgery: Oncologic and Functional Insights

(1) Background: Confocal laser endomicroscopy (CLE) has emerged as a transformative tool in head and neck surgery, with applications spanning oncologic insights and functional evaluations. This study delves into CLE's potential in these domains. (2) Methods: We performed CLE in head and neck oncologic surgery, focusing on tumor margin identification and precise resection. We also employed CLE for functional assessment in allergic rhinitis, observing real-time mucosal changes during nasal provocation testing. (3) Results: In oncologic surgery, CLE enabled real-time visualization of tumor margins and cellular patterns, aiding resection decisions. In allergic rhinitis assessment, CLE captured dynamic morphological alterations upon allergen exposure, enhancing understanding of mucosal reactions. (4) Conclusions: The integration of CLE with evolving technologies such as deep learning and AI holds promise for enhanced diagnostic accuracy. This study underscores CLE's expansive potential, highlighting its role in guiding surgical choices and illuminating inflammatory processes in the head and neck.

The intersection of video capsule endoscopy and artificial intelligence: addressing unique challenges using machine learning

Introduction

Technical burdens and time-intensive review processes limit the practical utility of video capsule endoscopy (VCE). Artificial intelligence (AI) is poised to address these limitations, but the intersection of AI and VCE reveals challenges that must first be overcome. We identified five challenges to address. Challenge #1: VCE data are stochastic and contains significant artifact. Challenge #2: VCE interpretation is cost-intensive. Challenge #3: VCE data are inherently imbalanced. Challenge #4: Existing VCE AIMLT are computationally cumbersome. Challenge #5: Clinicians are hesitant to accept AIMLT that cannot explain their process.

Methods

An anatomic landmark detection model was used to test the application of convolutional neural networks (CNNs) to the task of classifying VCE data. We also created a tool that assists in expert annotation of VCE data. We then created more elaborate models using different approaches including a multi-frame approach, a CNN based on graph representation, and a few-shot approach based on meta-learning.

Results

When used on full-length VCE footage, CNNs accurately identified anatomic landmarks (99.1%), with gradient weighted-class activation mapping showing the parts of each frame that the CNN used to make its decision. The graph CNN with weakly supervised learning (accuracy 89.9%, sensitivity of 91.1%), the few-shot model (accuracy 90.8%, precision 91.4%, sensitivity 90.9%), and the multi-frame model (accuracy 97.5%, precision 91.5%, sensitivity 94.8%) performed well.

Discussion

Each of these five challenges is addressed, in part, by one of our AI-based models. Our goal of producing high performance using lightweight models that aim to improve clinician confidence was achieved.

Intestinal Barrier Healing Is Superior to Endoscopic and Histologic Remission for Predicting Major Adverse Outcomes in Inflammatory Bowel Disease: The Prospective ERIca Trial

BACKGROUND & AIMS

Endoscopic and histologic remission have emerged as key therapeutic goals in the management of inflammatory bowel diseases (IBD) that are associated with favorable long-term disease outcomes. Here, we prospectively compared the predictive value of barrier healing with endoscopic and histologic remission for predicting long-term disease behavior in a large cohort of patients with IBD in clinical remission.

METHODS

At baseline, patients with IBD in clinical remission underwent ileocolonoscopy with assessment of intestinal barrier function by confocal endomicroscopy. Endoscopic and histologic disease activity, as well as barrier healing, was prospectively assessed along established scores. During subsequent follow-up, patients were closely monitored for clinical disease activity and the occurrence of major adverse outcomes (MAOs): disease flares, IBD-related hospitalization or surgery, and initiation or dose escalation of systemic steroids, immunosuppressants, small molecules, or biological therapy.

RESULTS

The final analysis included 181 patients, 100 with Crohn's disease [CD] and 81 with ulcerative colitis (UC). During a mean follow-up of 35 (CD) and 25 (UC) months, 73% of patients with CD and 69% of patients with UC experienced at least 1 MAO. The probability of MAO-free survival was significantly higher in patients with IBD with endoscopic remission compared with endoscopically active disease. In addition, histologic remission predicted MAO-free survival in patients with UC but not CD. Barrier healing on endomicroscopy was superior to endoscopic and histologic remission for predicting MAO-free survival in both UC and CD.

CONCLUSIONS

Barrier healing is associated with decreased risk of disease progression in patients with clinically remittent IBD, with superior predictive performance compared with endoscopic and histologic remission. Analysis of barrier function might be considered as a future treatment target in clinical trials.

CLINICALTRIALS

gov number, NCT05157750.

Confocal Laser Endomicroscopy for Detection of Early Upper Gastrointestinal Cancer

Esophageal and gastric cancers are common diseases with high morbidity and mortality; thus, early detection and treatment are beneficial to improve prognosis. Confocal laser endomicroscopy (CLE) is a novel imaging technique that permits the histological analysis of tissues during endoscopy. CLE has been shown to uniquely affect the diagnosis of early upper gastrointestinal cancers. Relevant literature was searched using PubMed and Google Scholar databases. Despite inherent flaws, CLE can reduce tissue damage and improve diagnostic accuracy to a certain extent. CLE in combination with other imaging methods can help enhance the detection rate and avoid unnecessary biopsies in the management of esophageal or gastric cancer and precancerous lesions. CLE is of great significance in the diagnosis and surveillance of early cancers of the upper gastrointestinal tract. Further technical innovations and the standardisation of CLE will make it more responsive to the needs of routine clinical applications.

Automated detection of premalignant oral lesions on whole slide images using convolutional neural networks

INTRODUCTION

Oral epithelial dysplasia (OED) is a precursor lesion to oral squamous cell carcinoma, a disease with a reported overall survival rate of 56 percent across all stages. Accurate detection of OED is critical as progression to oral cancer can be impeded with complete excision of premalignant lesions. However, previous research has demonstrated that the task of grading of OED, even when performed by highly trained experts, is subject to high rates of reader variability and misdiagnosis. Thus, our study aims to develop a convolutional neural network (CNN) model that can identify regions suspicious for OED whole-slide pathology images.

METHODS

During model development, we optimized key training hyperparameters including loss function on 112 pathologist annotated cases between the training and validation sets. Then, we compared OED segmentation and classification metrics between two well-established CNN architectures for medical imaging, DeepLabv3+ and UNet++. To further assess generalizability, we assessed case-level performance of a held-out test set of 44 whole-slide images.

RESULTS

DeepLabv3+ outperformed UNet++ in overall accuracy, precision, and segmentation metrics in a 4-fold cross validation study. When applied to the held-out test set, our best performing DeepLabv3+ model achieved an overall accuracy and F1-Score of 93.3 percent and 90.9 percent, respectively.

CONCLUSION

The present study trained and implemented a CNN-based deep learning model for identification and segmentation of oral epithelial dysplasia (OED) with reasonable success. Computer assisted detection was shown to be feasible in detecting premalignant/precancerous oral lesions, laying groundwork for eventual clinical implementation.

In vivo microscopy as an adjunctive tool to guide detection, diagnosis, and treatment

SIGNIFICANCE

There have been numerous academic and commercial efforts to develop high-resolution in vivo microscopes for a variety of clinical use cases, including early disease detection and surgical guidance. While many high-profile studies, commercialized products, and publications have resulted from these efforts, mainstream clinical adoption has been relatively slow other than for a few clinical applications (e.g., dermatology).

AIM

Here, our goals are threefold: (1) to introduce and motivate the need for in vivo microscopy (IVM) as an adjunctive tool for clinical detection, diagnosis, and treatment, (2) to discuss the key translational challenges facing the field, and (3) to propose best practices and recommendations to facilitate clinical adoption.

APPROACH

We will provide concrete examples from various clinical domains, such as dermatology, oral/gastrointestinal oncology, and neurosurgery, to reinforce our observations and recommendations.

RESULTS

While the incremental improvement and optimization of IVM technologies should and will continue to occur, future translational efforts would benefit from the following: (1) integrating clinical and industry partners upfront to define and maintain a compelling value proposition, (2) identifying multimodal/multiscale imaging workflows, which are necessary for success in most clinical scenarios, and (3) developing effective artificial intelligence tools for clinical decision support, tempered by a realization that complete adoption of such tools will be slow.

CONCLUSIONS

The convergence of imaging modalities, academic-industry-clinician partnerships, and new computational capabilities has the potential to catalyze rapid progress and adoption of IVM in the next few decades.

Machine Learning for Future Subtyping of the Tumor Microenvironment of Gastro-Esophageal Adenocarcinomas

Tumor progression involves an intricate interplay between malignant cells and their surrounding tumor microenvironment (TME) at specific sites. The TME is dynamic and is composed of stromal, parenchymal, and immune cells, which mediate cancer progression and therapy resistance. Evidence from preclinical and clinical studies revealed that TME targeting and reprogramming can be a promising approach to achieve anti-tumor effects in several cancers, including in GEA. Thus, it is of great interest to use modern technology to understand the relevant components of programming the TME. Here, we discuss the approach of machine learning, which recently gained increasing interest recently because of its ability to measure tumor parameters at the cellular level, reveal global features of relevance, and generate prognostic models. In this review, we discuss the relevant stromal composition of the TME in GEAs and discuss how they could be integrated. We also review the current progress in the application of machine learning in different medical disciplines that are relevant for the management and study of GEA.