Deep Convolutional Neural Network-Based Positron Emission Tomography Analysis Predicts Esophageal Cancer Outcome

Impact of artificial intelligence in the management of esophageal, gastric and colorectal malignancies

The incidence of gastrointestinal malignancies has increased over the past decade at an alarming rate. Colorectal and gastric cancers are the third and fifth most commonly diagnosed cancers worldwide but are cited as the second and third leading causes of mortality. Early institution of appropriate therapy from timely diagnosis can optimize patient outcomes. Artificial intelligence (AI)-assisted diagnostic, prognostic, and therapeutic tools can assist in expeditious diagnosis, treatment planning/response prediction, and post-surgical prognostication. AI can intercept neoplastic lesions in their primordial stages, accurately flag suspicious and/or inconspicuous lesions with greater accuracy on radiologic, histopathological, and/or endoscopic analyses, and eliminate over-dependence on clinicians. AI-based models have shown to be on par, and sometimes even outperformed experienced gastroenterologists and radiologists. Convolutional neural networks (state-of-the-art deep learning models) are powerful computational models, invaluable to the field of precision oncology. These models not only reliably classify images, but also accurately predict response to chemotherapy, tumor recurrence, metastasis, and survival rates post-treatment. In this systematic review, we analyze the available evidence about the diagnostic, prognostic, and therapeutic utility of artificial intelligence in gastrointestinal oncology.

Utilizing convolutional neural networks for discriminating cancer and stromal cells in three-dimensional cell culture images with nuclei counterstain

Significance

Accurate cell segmentation and classification in three-dimensional (3D) images are vital for studying live cell behavior and drug responses in 3D tissue culture. Evaluating diverse cell populations in 3D cell culture over time necessitates non-toxic staining methods, as specific fluorescent tags may not be suitable, and immunofluorescence staining can be cytotoxic for prolonged live cell cultures.

Aim

We aim to perform machine learning-based cell classification within a live heterogeneous cell culture population grown in a 3D tissue culture relying only on reflectance, transmittance, and nuclei counterstained images obtained by confocal microscopy.

Approach

In this study, we employed a supervised convolutional neural network (CNN) to classify tumor cells and fibroblasts within 3D-grown spheroids. These cells are first segmented using the marker-controlled watershed image processing method. Training data included nuclei counterstaining, reflectance, and transmitted light images, with stained fibroblast and tumor cells as ground-truth labels.

Results

Our results demonstrate the successful marker-controlled watershed segmentation of 84% of spheroid cells into single cells. We achieved a median accuracy of 67% (95% confidence interval of the median is 65-71%) in identifying cell types. We also recapitulate the original 3D images using the CNN-classified cells to visualize the original 3D-stained image's cell distribution.

Conclusion

This study introduces a non-invasive toxicity-free approach to 3D cell culture evaluation, combining machine learning with confocal microscopy, opening avenues for advanced cell studies.

Methodological evaluation of original articles on radiomics and machine learning for outcome prediction based on positron emission tomography (PET)

AIM

Despite a vast number of articles on radiomics and machine learning in positron emission tomography (PET) imaging, clinical applicability remains limited, partly owing to poor methodological quality. We therefore systematically investigated the methodology described in publications on radiomics and machine learning for PET-based outcome prediction.

METHODS

A systematic search for original articles was run on PubMed. All articles were rated according to 17 criteria proposed by the authors. Criteria with >2 rating categories were binarized into "adequate" or "inadequate". The association between the number of "adequate" criteria per article and the date of publication was examined.

RESULTS

One hundred articles were identified (published between 07/2017 and 09/2023). The median proportion of articles per criterion that were rated "adequate" was 65% (range: 23-98%). Nineteen articles (19%) mentioned neither a test cohort nor cross-validation to separate training from testing. The median number of criteria with an "adequate" rating per article was 12.5 out of 17 (range, 4-17), and this did not increase with later dates of publication (Spearman's rho, 0.094; p = 0.35). In 22 articles (22%), less than half of the items were rated "adequate". Only 8% of articles published the source code, and 10% made the dataset openly available.

CONCLUSION

Among the articles investigated, methodological weaknesses have been identified, and the degree of compliance with recommendations on methodological quality and reporting shows potential for improvement. Better adherence to established guidelines could increase the clinical significance of radiomics and machine learning for PET-based outcome prediction and finally lead to the widespread use in routine clinical practice.

Assessment of hyperspectral imaging and CycleGAN-simulated narrowband techniques to detect early esophageal cancer

The clinical signs and symptoms of esophageal cancer (EC) are often not discernible until the intermediate or advanced phases. The detection of EC in advanced stages significantly decreases the survival rate to below 20%. This study conducts a comparative analysis of the efficacy of several imaging techniques, including white light image (WLI), narrowband imaging (NBI), cycle-consistent adversarial network simulated narrowband image (CNBI), and hyperspectral imaging simulated narrowband image (HNBI), in the early detection of esophageal cancer (EC). In conjunction with Kaohsiung Armed Forces General Hospital, a dataset consisting of 1000 EC pictures was used, including 500 images captured using WLI and 500 images captured using NBI. The CycleGAN model was used to generate the CNBI dataset. Additionally, a novel method for HSI imaging was created with the objective of generating HNBI pictures. The evaluation of the efficacy of these four picture types in early detection of EC was conducted using three indicators: CIEDE2000, entropy, and the structural similarity index measure (SSIM). Results of the CIEDE2000, entropy, and SSIM analyses suggest that using CycleGAN to generate CNBI images and HSI model for creating HNBI images is superior in detecting early esophageal cancer compared to the use of conventional WLI and NBI techniques.

Systematic meta-analysis of computer-aided detection to detect early esophageal cancer using hyperspectral imaging

One of the leading causes of cancer deaths is esophageal cancer (EC) because identifying it in early stage is challenging. Computer-aided diagnosis (CAD) could detect the early stages of EC have been developed in recent years. Therefore, in this study, complete meta-analysis of selected studies that only uses hyperspectral imaging to detect EC is evaluated in terms of their diagnostic test accuracy (DTA). Eight studies are chosen based on the Quadas-2 tool results for systematic DTA analysis, and each of the methods developed in these studies is classified based on the nationality of the data, artificial intelligence, the type of image, the type of cancer detected, and the year of publishing. Deeks' funnel plot, forest plot, and accuracy charts were made. The methods studied in these articles show the automatic diagnosis of EC has a high accuracy, but external validation, which is a prerequisite for real-time clinical applications, is lacking.

Classification of head and neck cancer from PET images using convolutional neural networks

The aim of this study was to develop a convolutional neural network (CNN) for classifying positron emission tomography (PET) images of patients with and without head and neck squamous cell carcinoma (HNSCC) and other types of head and neck cancer. A PET/magnetic resonance imaging scan with 18F-fluorodeoxyglucose (18F-FDG) was performed for 200 head and neck cancer patients, 182 of which were diagnosed with HNSCC, and the location of cancer tumors was marked to the images with a binary mask by a medical doctor. The models were trained and tested with five-fold cross-validation with the primary data set of 1990 2D images obtained by dividing the original 3D images of 178 HNSCC patients into transaxial slices and with an additional test set with 238 images from the patients with head and neck cancer other than HNSCC. A shallow and a deep CNN were built by using the U-Net architecture for classifying the data into two groups based on whether an image contains cancer or not. The impact of data augmentation on the performance of the two CNNs was also considered. According to our results, the best model for this task in terms of area under receiver operator characteristic curve (AUC) is a deep augmented model with a median AUC of 85.1%. The four models had highest sensitivity for HNSCC tumors on the root of the tongue (median sensitivities of 83.3-97.7%), in fossa piriformis (80.2-93.3%), and in the oral cavity (70.4-81.7%). Despite the fact that the models were trained with only HNSCC data, they had also very good sensitivity for detecting follicular and papillary carcinoma of thyroid gland and mucoepidermoid carcinoma of the parotid gland (91.7-100%).

Artificial intelligence-based computer-aided diagnosis system supports diagnosis of lymph node metastasis in esophageal squamous cell carcinoma: A multicenter study

Background

This study aimed to develop an artificial intelligence-based computer-aided diagnosis system (AI-CAD) emulating the diagnostic logic of radiologists for lymph node metastasis (LNM) in esophageal squamous cell carcinoma (ESCC) patients, which contributed to clinical treatment decision-making.

Methods

A total of 689 ESCC patients with PET/CT images were enrolled from three hospitals and divided into a training cohort and two external validation cohorts. 452 CT images from three publicly available datasets were also included for pretraining the model. Anatomic information from CT images was first obtained automatically using a U-Net-based multi-organ segmentation model, and metabolic information from PET images was subsequently extracted using a gradient-based approach. AI-CAD was developed in the training cohort and externally validated in two validation cohorts.

Results

The AI-CAD achieved an accuracy of 0.744 for predicting pathological LNM in the external cohort and a good agreement with a human expert in two external validation cohorts (kappa = 0.674 and 0.587, p < 0.001). With the aid of AI-CAD, the human expert's diagnostic performance for LNM was significantly improved (accuracy [95% confidence interval]: 0.712 [0.669-0.758] vs. 0.833 [0.797-0.865], specificity [95% confidence interval]: 0.697 [0.636-0.753] vs. 0.891 [0.851-0.928]; p < 0.001) among patients underwent lymphadenectomy in the external validation cohorts.

Conclusions

The AI-CAD could aid in preoperative diagnosis of LNM in ESCC patients and thereby support clinical treatment decision-making.

The Oesophageal Cancer Multidisciplinary Team: Can Machine Learning Assist Decision-Making?

BACKGROUND

The complexity of the upper gastrointestinal (UGI) multidisciplinary team (MDT) is continually growing, leading to rising clinician workload, time pressures, and demands. This increases heterogeneity or 'noise' within decision-making for patients with oesophageal cancer (OC) and may lead to inconsistent treatment decisions. In recent decades, the application of artificial intelligence (AI) and more specifically the branch of machine learning (ML) has led to a paradigm shift in the perceived utility of statistical modelling within healthcare. Within oesophageal cancer (OC) care, ML techniques have already been applied with early success to the analyses of histological samples and radiology imaging; however, it has not yet been applied to the MDT itself where such models are likely to benefit from incorporating information-rich, diverse datasets to increase predictive model accuracy.

METHODS

This review discusses the current role the MDT plays in modern UGI cancer care as well as the utilisation of ML techniques to date using histological and radiological data to predict treatment response, prognostication, nodal disease evaluation, and even resectability within OC.

RESULTS

The review finds that an emerging body of evidence is growing in support of ML tools within multiple domains relevant to decision-making within OC including automated histological analysis and radiomics. However, to date, no specific application has been directed to the MDT itself which routinely assimilates this information.

CONCLUSIONS

The authors feel the UGI MDT offers an information-rich, diverse array of data from which ML offers the potential to standardise, automate, and produce more consistent, data-driven MDT decisions.

Deep learning based radiomics for gastrointestinal cancer diagnosis and treatment: A minireview

Gastrointestinal (GI) cancers are the major cause of cancer-related mortality globally. Medical imaging is an important auxiliary means for the diagnosis, assessment and prognostic prediction of GI cancers. Radiomics is an emerging and effective technology to decipher the encoded information within medical images, and traditional machine learning is the most commonly used tool. Recent advances in deep learning technology have further promoted the development of radiomics. In the field of GI cancer, although there are several surveys on radiomics, there is no specific review on the application of deep-learning-based radiomics (DLR). In this review, a search was conducted on Web of Science, PubMed, and Google Scholar with an emphasis on the application of DLR for GI cancers, including esophageal, gastric, liver, pancreatic, and colorectal cancers. Besides, the challenges and recommendations based on the findings of the review are comprehensively analyzed to advance DLR.

Prediction of Individual Lymph Node Metastatic Status in Esophageal Squamous Cell Carcinoma Using Routine Computed Tomography Imaging: Comparison of Size-Based Measurements and Radiomics-Based Models

BACKGROUND

Lymph node status is vital for prognosis and treatment decisions for esophageal squamous cell carcinoma (ESCC). This study aimed to construct and evaluate an optimal radiomics-based method for a more accurate evaluation of individual regional lymph node status in ESCC and to compare it with traditional size-based measurements.

METHODS

The study consecutively collected 3225 regional lymph nodes from 530 ESCC patients receiving upfront surgery from January 2011 to October 2015. Computed tomography (CT) scans for individual lymph nodes were analyzed. The study evaluated the predictive performance of machine-learning models trained on features extracted from two-dimensional (2D) and three-dimensional (3D) radiomics by different contouring methods. Robust and important radiomics features were selected, and classification models were further established and validated.

RESULTS

The lymph node metastasis rate was 13.2% (427/3225). The average short-axis diameter was 6.4 mm for benign lymph nodes and 7.9 mm for metastatic lymph nodes. The division of lymph node stations into five regions according to anatomic lymph node drainage (cervical, upper mediastinal, middle mediastinal, lower mediastinal, and abdominal regions) improved the predictive performance. The 2D radiomics method showed optimal diagnostic results, with more efficient segmentation of nodal lesions. In the test set, this optimal model achieved an area under the receiver operating characteristic curve of 0.841-0.891, an accuracy of 84.2-94.7%, a sensitivity of 65.7-83.3%, and a specificity of 84.4-96.7%.

CONCLUSIONS

The 2D radiomics-based models noninvasively predicted the metastatic status of an individual lymph node in ESCC and outperformed the conventional size-based measurement. The 2D radiomics-based model could be incorporated into the current clinical workflow to enable better decision-making for treatment strategies.

Locoregional recurrence prediction in head and neck cancer based on multi-modality and multi-view feature expansion

Objective. Locoregional recurrence (LRR) is one of the leading causes of treatment failure in head and neck (H&N) cancer. Accurately predicting LRR after radiotherapy is essential to achieving better treatment outcomes for patients with H&N cancer through developing personalized treatment strategies. We aim to develop an end-to-end multi-modality and multi-view feature extension method (MMFE) to predict LRR in H&N cancer. Approach. Deep learning (DL) has been widely used for building prediction models and has achieved great success. Nevertheless, 2D-based DL models inherently fail to utilize the contextual information from adjacent slices, while complicated 3D models have a substantially larger number of parameters, which require more training samples, memory and computing resources. In the proposed MMFE scheme, through the multi-view feature expansion and projection dimension reduction operations, we are able to reduce the model complexity while preserving volumetric information. Additionally, we designed a multi-modality convolutional neural network that can be trained in an end-to-end manner and can jointly optimize the use of deep features of CT, PET and clinical data to improve the model’s prediction ability. Main results. The dataset included 206 eligible patients, of which, 49 had LRR while 157 did not. The proposed MMFE method obtained a higher AUC value than the other four methods. The best prediction result was achieved when using all three modalities, which yielded an AUC value of 0.81. Significance. Comparison experiments demonstrated the superior performance of the MMFE as compared to other 2D/3D-DL-based methods. By combining CT, PET and clinical features, the MMFE could potentially identify H&N cancer patients at high risk for LRR such that personalized treatment strategy can be developed accordingly.

Evaluation of the Effect of Comprehensive Nursing Interventions on Plaque Control in Patients with Periodontal Disease in the Context of Artificial Intelligence

Plaque is a bacterial biofilm that adheres to each other and exists on the tooth surface, and new plaque can continuously reform after removing it from the tooth surface. The pathogenesis of periodontal disease is related to the bacteria, the host and the environment, with the bacteria and bacterial products in plaque being the main initiators of periodontal disease. The effective control of plaque is an effective method for the treatment and prevention of periodontal disease and is often underappreciated in clinical practice. For the traditional diagnostic method through experience and visual observation, it may lead to misdiagnosis and underdiagnosis. In order to accurately diagnose plaque disease, this study designed a convolutional neural network-based oral dental disease diagnosis system for oral care interventions to improve oral health awareness. Thus motivate patients to implement proper oral health care measures, and continuously and lifelong insist on thorough daily plaque removal to improve patients' physical health and quality of life in periodontal disease patients.

Machine Learning and Radiomics Applications in Esophageal Cancers Using Non-Invasive Imaging Methods-A Critical Review of Literature

Esophageal cancer (EC) is of public health significance as one of the leading causes of cancer death worldwide. Accurate staging, treatment planning and prognostication in EC patients are of vital importance. Recent advances in machine learning (ML) techniques demonstrate their potential to provide novel quantitative imaging markers in medical imaging. Radiomics approaches that could quantify medical images into high-dimensional data have been shown to improve the imaging-based classification system in characterizing the heterogeneity of primary tumors and lymph nodes in EC patients. In this review, we aim to provide a comprehensive summary of the evidence of the most recent developments in ML application in imaging pertinent to EC patient care. According to the published results, ML models evaluating treatment response and lymph node metastasis achieve reliable predictions, ranging from acceptable to outstanding in their validation groups. Patients stratified by ML models in different risk groups have a significant or borderline significant difference in survival outcomes. Prospective large multi-center studies are suggested to improve the generalizability of ML techniques with standardized imaging protocols and harmonization between different centers.

SCNET: A Novel UGI Cancer Screening Framework Based on Semantic-Level Multimodal Data Fusion

Upper gastrointestinal (UGI) cancer has been identified as one of the ten most common causes of cancer deaths globally. UGI cancer screening is critical to improving the survival rate of UGI cancer patients. While many approaches to UGI cancer screening rely on single-modality data such as gastroscope imaging, limited studies have been dedicated to UGI cancer screening exploiting multisource and multimodal medical data, which could potentially lead to improved screening results. In this paper, we propose semantic-level cancer-screening network (SCNET), a framework for UGI cancer screening based on semantic-level multimodal upper gastrointestinal data fusion. Specifically, the proposed SCNET consists of a gastrointestinal image recognition flow and a textual medical record processing flow. High-level features of upper gastrointestinal data are extracted by identifying effective feature channels according to the correlation between the textual features and the spatial structure of the image features. The final screening results are obtained after the data fusion step. The experimental results show that the improvement of our approach over the state-of-the-art ones reached 4.01% in average. The source code of SCNET is available at https://github.com/netflymachine/SCNET.

Predicting aggressive histopathological features in esophageal cancer with positron emission tomography using a deep convolutional neural network

Background

The presence of lymphovascular invasion (LVI) and perineural invasion (PNI) are of great prognostic importance in esophageal squamous cell carcinoma. Currently, positron emission tomography (PET) scans are the only means of functional assessment prior to treatment. We aimed to predict the presence of LVI and PNI in esophageal squamous cell carcinoma using PET imaging data by training a three-dimensional convolution neural network (3D-CNN).

Methods

Seven hundred and ninety-eight PET scans of patients with esophageal squamous cell carcinoma and 309 PET scans of patients with stage I lung cancer were collected. In the first part of this study, we built a 3D-CNN based on a residual network, ResNet, for a task to classify the scans into esophageal cancer or lung cancer. In the second stage, we collected the PET scans of 278 patients undergoing esophagectomy for a task to classify and predict the presence of LVI/PNI.

Results

In the first part, the model performance attained an area under the receiver operating characteristic curve (AUC) of 0.860. In the second part, we randomly split 80%, 10%, and 10% of our dataset into training set, validation set and testing set, respectively, for a task to classify the scans into the presence of LVI/PNI and evaluated the model performance on the testing set. Our 3D-CNN model attained an AUC of 0.668 in the testing set, which shows a better discriminative ability than random guessing.

Conclusions

A 3D-CNN can be trained, using PET imaging datasets, to predict LNV/PNI in esophageal cancer with acceptable accuracy.

Lung Nodule Sizes Are Encoded When Scaling CT Image for CNN's

Noninvasive diagnosis of lung cancer in early stages is one task where radiomics helps. Clinical practice shows that the size of a nodule has high predictive power for malignancy. In the literature, convolutional neural networks (CNNs) have become widely used in medical image analysis. We study the ability of a CNN to capture nodule size in computed tomography images after images are resized for CNN input. For our experiments, we used the National Lung Screening Trial data set. Nodules were labeled into 2 categories (small/large) based on the original size of a nodule. After all extracted patches were re-sampled into 100-by-100-pixel images, a CNN was able to successfully classify test nodules into small- and large-size groups with high accuracy. To show the generality of our discovery, we repeated size classification experiments using Common Objects in Context (COCO) data set. From the data set, we selected 3 categories of images, namely, bears, cats, and dogs. For all 3 categories a 5- × 2-fold cross-validation was performed to put them into small and large classes. The average area under receiver operating curve is 0.954, 0.952, and 0.979 for the bear, cat, and dog categories, respectively. Thus, camera image rescaling also enables a CNN to discover the size of an object. The source code for experiments with the COCO data set is publicly available in Github (https://github.com/VisionAI-USF/COCO_Size_Decoding/).

Automatic classification of esophageal lesions in endoscopic images using a convolutional neural network

Background

Using deep learning techniques in image analysis is a dynamically emerging field. This study aims to use a convolutional neural network (CNN), a deep learning approach, to automatically classify esophageal cancer (EC) and distinguish it from premalignant lesions.

Methods

A total of 1,272 white-light images were adopted from 748 subjects, including normal cases, premalignant lesions, and cancerous lesions; 1,017 images were used to train the CNN, and another 255 images were examined to evaluate the CNN architecture. Our proposed CNN structure consists of two subnetworks (O-stream and P-stream). The original images were used as the inputs of the O-stream to extract the color and global features, and the pre-processed esophageal images were used as the inputs of the P-stream to extract the texture and detail features.

Results

The CNN system we developed achieved an accuracy of 85.83%, a sensitivity of 94.23%, and a specificity of 94.67% after the fusion of the 2 streams was accomplished. The classification accuracy of normal esophagus, premalignant lesion, and EC were 94.23%, 82.5%, and 77.14%, respectively, which shows a better performance than the Local Binary Patterns (LBP) + Support Vector Machine (SVM) and Histogram of Gradient (HOG) + SVM methods. A total of 8 of the 35 (22.85%) EC lesions were categorized as premalignant lesions because of their slightly reddish and flat lesions.

Conclusions

The CNN system, with 2 streams, demonstrated high sensitivity and specificity with the endoscopic images. It obtained better detection performance than the currently used methods based on the same datasets and has great application prospects in assisting endoscopists to distinguish esophageal lesion subclasses.