Deep learning for lung cancer prognostication: A retrospective multi-cohort radiomics study

CT-derived radiomic analysis for predicting the survival rate of patients with non-small cell lung cancer receiving radiotherapy

BACKGROUND

Radiomics provides an opportunity to minimize adverse effects and optimize the efficacy of treatments noninvasively. This study aims to develop a computed tomography (CT) derived radiomic signature to predict radiological response for the patients with non-small cell lung cancer (NSCLC) receiving radiotherapy.

METHODS

Total 815 NSCLC patients receiving radiotherapy were sourced from public datasets. Using CT images of 281 NSCLC patients, we adopted genetic algorithm to establish a predictive radiomic signature for radiotherapy that had optimal C-index value by Cox model. Survival analysis and receiver operating characteristic curve were performed to estimate the predictive performance of the radiomic signature. Furthermore, radiogenomics analysis was performed in a dataset with matched images and transcriptome data.

RESULTS

Radiomic signature consisting of three features was established and then validated in the validation dataset (log-rank P = 0.0047) including 140 patient, and showed a significant predictive power in two independent datasets totaling 395 NSCLC patients with binary 2-year survival endpoint. Furthermore, the novel proposed radiomic nomogram significantly improved the prognostic performance (concordance index) of clinicopathological factors. Radiogenomics analysis linked our signature with important tumor biological processes (e.g. Mismatch repair, Cell adhesion molecules and DNA replication) associated with clinical outcomes.

CONCLUSIONS

The radiomic signature, reflecting tumor biological processes, could noninvasively predict therapeutic efficacy of NSCLC patients receiving radiotherapy and demonstrate unique advantage for clinical application.

Survival prediction for stage I-IIIA non-small cell lung cancer using deep learning

BACKGROUND AND PURPOSE

The aim of this study was to develop and evaluate a prediction model for 2-year overall survival (OS) in stage I-IIIA non-small cell lung cancer (NSCLC) patients who received definitive radiotherapy by considering clinical variables and image features from pre-treatment CT-scans.

MATERIALS AND METHODS

NSCLC patients who received stereotactic radiotherapy were prospectively collected at the UMCG and split into a training and a hold out test set including 189 and 81 patients, respectively. External validation was performed on 228 NSCLC patients who were treated with radiation or concurrent chemoradiation at the Maastro clinic (Lung1 dataset). A hybrid model that integrated both image and clinical features was implemented using deep learning. Image features were learned from cubic patches containing lung tumours extracted from pre-treatment CT scans. Relevant clinical variables were selected by univariable and multivariable analyses.

RESULTS

Multivariable analysis showed that age and clinical stage were significant prognostic clinical factors for 2-year OS. Using these two clinical variables in combination with image features from pre-treatment CT scans, the hybrid model achieved a median AUC of 0.76 [95 % CI: 0.65-0.86] and 0.64 [95 % CI: 0.58-0.70] on the complete UMCG and Maastro test sets, respectively. The Kaplan-Meier survival curves showed significant separation between low and high mortality risk groups on these two test sets (log-rank test: p-value < 0.001, p-value = 0.012, respectively) CONCLUSION: We demonstrated that a hybrid model could achieve reasonable performance by utilizing both clinical and image features for 2-year OS prediction. Such a model has the potential to identify patients with high mortality risk and guide clinical decision making.

Deep Learning-Based Model for Identifying Tumors in Endoscopic Images From Patients With Locally Advanced Rectal Cancer Treated With Total Neoadjuvant Therapy

BACKGROUND

A barrier to the widespread adoption of watch-and-wait management for locally advanced rectal cancer is the inaccuracy and variability of identifying tumor response endoscopically in patients who have completed total neoadjuvant therapy (chemoradiotherapy and systemic chemotherapy).

OBJECTIVE

This study aimed to develop a novel method of identifying the presence or absence of a tumor in endoscopic images using deep convolutional neural network-based automatic classification and to assess the accuracy of the method.

DESIGN

In this prospective pilot study, endoscopic images obtained before, during, and after total neoadjuvant therapy were grouped on the basis of tumor presence. A convolutional neural network was modified for probabilistic classification of tumor versus no tumor and trained with an endoscopic image set. After training, a testing endoscopic imaging set was applied to the network.

SETTINGS

The study was conducted at a comprehensive cancer center.

PATIENTS

Images were analyzed from 109 patients who were diagnosed with locally advanced rectal cancer between December 2012 and July 2017 and who underwent total neoadjuvant therapy.

MAIN OUTCOME MEASURES

The main outcomes were accuracy of identifying tumor presence or absence in endoscopic images measured as area under the receiver operating characteristic for the training and testing image sets.

RESULTS

A total of 1392 images were included; 1099 images (468 of no tumor and 631 of tumor) were for training and 293 images (151 of no tumor and 142 of tumor) for testing. The area under the receiver operating characteristic for training and testing was 0.83.

LIMITATIONS

The study had a limited number of images in each set and was conducted at a single institution.

CONCLUSIONS

The convolutional neural network method is moderately accurate in distinguishing tumor from no tumor. Further research should focus on validating the convolutional neural network on a large image set. See Video Abstract at http://links.lww.com/DCR/B959 .

MODELO BASADO EN APRENDIZAJE PROFUNDO PARA IDENTIFICAR TUMORES EN IMGENES ENDOSCPICAS DE PACIENTES CON CNCER DE RECTO LOCALMENTE AVANZADO TRATADOS CON TERAPIA NEOADYUVANTE TOTAL

ANTECEDENTES:Una barrera para la aceptación generalizada del tratamiento de Observar y Esperar para el cáncer de recto localmente avanzado, es la imprecisión y la variabilidad en la identificación de la respuesta tumoral endoscópica, en pacientes que completaron la terapia neoadyuvante total (quimiorradioterapia y quimioterapia sistémica).OBJETIVO:Desarrollar un método novedoso para identificar la presencia o ausencia de un tumor en imágenes endoscópicas utilizando una clasificación automática basada en redes neuronales convolucionales profundas y evaluar la precisión del método.DISEÑO:Las imágenes endoscópicas obtenidas antes, durante y después de la terapia neoadyuvante total se agruparon en base de la presencia del tumor. Se modificó una red neuronal convolucional para la clasificación probabilística de tumor versus no tumor y se entrenó con un conjunto de imágenes endoscópicas. Después del entrenamiento, se aplicó a la red un conjunto de imágenes endoscópicas de prueba.ENTORNO CLINICO:El estudio se realizó en un centro oncológico integral.PACIENTES:Analizamos imágenes de 109 pacientes que fueron diagnosticados de cáncer de recto localmente avanzado entre diciembre de 2012 y julio de 2017 y que se sometieron a terapia neoadyuvante total.PRINCIPALES MEDIDAS DE VALORACION:La precisión en la identificación de la presencia o ausencia de tumores en imágenes endoscópicas medidas como el área bajo la curva de funcionamiento del receptor para los conjuntos de imágenes de entrenamiento y prueba.RESULTADOS:Se incluyeron mil trescientas noventa y dos imágenes: 1099 (468 sin tumor y 631 con tumor) para entrenamiento y 293 (151 sin tumor y 142 con tumor) para prueba. El área bajo la curva operativa del receptor para entrenamiento y prueba fue de 0,83.LIMITACIONES:El estudio tuvo un número limitado de imágenes en cada conjunto y se realizó en una sola institución.CONCLUSIÓN:El método de la red neuronal convolucional es moderadamente preciso para distinguir el tumor de ningún tumor. La investigación adicional debería centrarse en validar la red neuronal convolucional en un conjunto de imágenes mayor. Consulte Video Resumen en http://links.lww.com/DCR/B959 . (Traducción -Dr. Fidel Ruiz Healy ).

MVI-TR: A Transformer-Based Deep Learning Model with Contrast-Enhanced CT for Preoperative Prediction of Microvascular Invasion in Hepatocellular Carcinoma

In this study, we considered preoperative prediction of microvascular invasion (MVI) status with deep learning (DL) models for patients with early-stage hepatocellular carcinoma (HCC) (tumor size ≤ 5 cm). Two types of DL models based only on venous phase (VP) of contrast-enhanced computed tomography (CECT) were constructed and validated. From our hospital (First Affiliated Hospital of Zhejiang University, Zhejiang, P.R. China), 559 patients, who had histopathological confirmed MVI status, participated in this study. All preoperative CECT were collected, and the patients were randomly divided into training and validation cohorts at a ratio of 4:1. We proposed a novel transformer-based end-to-end DL model, named MVI-TR, which is a supervised learning method. MVI-TR can capture features automatically from radiomics and perform MVI preoperative assessments. In addition, a popular self-supervised learning method, the contrastive learning model, and the widely used residual networks (ResNets family) were constructed for fair comparisons. With an accuracy of 99.1%, a precision of 99.3%, an area under the curve (AUC) of 0.98, a recalling rate of 98.8%, and an F1-score of 99.1% in the training cohort, MVI-TR achieved superior outcomes. Additionally, the validation cohort's MVI status prediction had the best accuracy (97.2%), precision (97.3%), AUC (0.935), recalling rate (93.1%), and F1-score (95.2%). MVI-TR outperformed other models for predicting MVI status, and showed great preoperative predictive value for early-stage HCC patients.

Exploring the opinion of Spanish medical specialists about the usefulness of radiomics in oncology

AIM

To describe the knowledge and opinion of health professionals regarding the usefulness of radiomics in oncology.

METHODS

A 12-question questionnaire (multiple-choice responses, Likert-type scale, and open response) was developed and sent to professionals related to diagnosis/treatment of oncological diseases (Oncology, Radiodiagnosis, Nuclear Medicine, Radiation Oncology, Hematology-Oncology, Radiophysics and Pathology). Participants were classified into two groups according to their level of training: attending physicians and residents.

RESULTS

114 professionals completed the survey (54% residents, mostly from Nuclear Medicine and Radiodiagnostic specialties). Attending physicians obtained a better performance in the area pf knowledge compared to residents. Both groups of respondents agreed regarding the usefulness of radiomics to help make more accurate diagnoses and promoting the work of medical teams and the most frequent disadvantages were related to the lack of systematization in the acquisition of images and extraction of parameters, the need for the training of professionals and concern about the replacement of human work by technological tools.

CONCLUSIONS

Radiomics is a novel field and the most general aspects are known by health professionals. The professionals surveyed were optimistic about the benefits provided by radiomics and other types of tools. The main problem detected was the lack of systematization in its implementation. The replacement of professionals and job loss is a concern, albeit less prevalent, and may respond to a generational phenomenon.

Integrated multi-dimensional deep neural network model improves prognosis prediction of advanced NSCLC patients receiving bevacizumab

Background

The addition of bevacizumab was found to be associated with prolonged survival whether in combination with chemotherapy, tyrosine kinase inhibitors or immune checkpoint inhibitors in the treatment landscape of advanced non-small cell lung cancer (NSCLC) patients. However, the biomarkers for efficacy of bevacizumab were still largely unknown. This study aimed to develop a deep learning model to provide individual assessment of survival in advanced NSCLC patients receiving bevacizumab.

Methods

All data were retrospectively collected from a cohort of 272 radiological and pathological proven advanced non-squamous NSCLC patients. A novel multi-dimensional deep neural network (DNN) models were trained based on clinicopathological, inflammatory and radiomics features using DeepSurv and N-MTLR algorithm. And concordance index (C-index) and bier score was used to demonstrate the discriminatory and predictive capacity of the model.

Results

The integration of clinicopathologic, inflammatory and radiomics features representation was performed using DeepSurv and N-MTLR with the C-index of 0.712 and 0.701 in testing cohort. And Cox proportional hazard (CPH) and random survival forest (RSF) models were also developed after data pre-processing and feature selection with the C-index of 0.665 and 0.679 respectively. DeepSurv prognostic model, indicated with best performance, was used for individual prognosis prediction. And patients divided in high-risk group were significantly associated with inferior PFS (median PFS: 5.4 vs 13.1 months, P<0.0001) and OS (median OS: 16.4 vs 21.3 months, P<0.0001).

Conclusions

The integration of clinicopathologic, inflammatory and radiomics features representation based on DeepSurv model exhibited superior predictive accuracy as non-invasive method to assist in patients counseling and guidance of optimal treatment strategies.

A comprehensive survey of deep learning research on medical image analysis with focus on transfer learning

This survey aims to identify commonly used methods, datasets, future trends, knowledge gaps, constraints, and limitations in the field to provide an overview of current solutions used in medical image analysis in parallel with the rapid developments in transfer learning (TL). Unlike previous studies, this survey grouped the last five years of current studies for the period between January 2017 and February 2021 according to different anatomical regions and detailed the modality, medical task, TL method, source data, target data, and public or private datasets used in medical imaging. Also, it provides readers with detailed information on technical challenges, opportunities, and future research trends. In this way, an overview of recent developments is provided to help researchers to select the most effective and efficient methods and access widely used and publicly available medical datasets, research gaps, and limitations of the available literature.

Role of artificial intelligence in risk prediction, prognostication, and therapy response assessment in colorectal cancer: current state and future directions

Artificial Intelligence (AI) is a branch of computer science that utilizes optimization, probabilistic and statistical approaches to analyze and make predictions based on a vast amount of data. In recent years, AI has revolutionized the field of oncology and spearheaded novel approaches in the management of various cancers, including colorectal cancer (CRC). Notably, the applications of AI to diagnose, prognosticate, and predict response to therapy in CRC, is gaining traction and proving to be promising. There have also been several advancements in AI technologies to help predict metastases in CRC and in Computer-Aided Detection (CAD) Systems to improve miss rates for colorectal neoplasia. This article provides a comprehensive review of the role of AI in predicting risk, prognosis, and response to therapies among patients with CRC.

Artificial intelligence-based non-small cell lung cancer transcriptome RNA-sequence analysis technology selection guide

The incidence and mortality rates of lung cancer are high worldwide, where non-small cell lung cancer (NSCLC) accounts for more than 85% of lung cancer cases. Recent non-small cell lung cancer research has been focused on analyzing patient prognosis after surgery and identifying mechanisms in connection with clinical cohort and ribonucleic acid (RNA) sequencing data, including single-cell ribonucleic acid (scRNA) sequencing data. This paper investigates statistical techniques and artificial intelligence (AI) based non-small cell lung cancer transcriptome data analysis methods divided into target and analysis technology groups. The methodologies of transcriptome data were schematically categorized so researchers can easily match analysis methods according to their goals. The most widely known and frequently utilized transcriptome analysis goal is to find essential biomarkers and classify carcinomas and cluster NSCLC subtypes. Transcriptome analysis methods are divided into three major categories: Statistical analysis, machine learning, and deep learning. Specific models and ensemble techniques typically used in NSCLC analysis are summarized in this paper, with the intent to lay a foundation for advanced research by converging and linking the various analysis methods available.

Artificial intelligence in lung cancer: current applications and perspectives

Artificial intelligence (AI) has been a very active research topic over the last years and thoracic imaging has particularly benefited from the development of AI and in particular deep learning. We have now entered a phase of adopting AI into clinical practice. The objective of this article was to review the current applications and perspectives of AI in thoracic oncology. For pulmonary nodule detection, computer-aided detection (CADe) tools have been commercially available since the early 2000s. The more recent rise of deep learning and the availability of large annotated lung nodule datasets have allowed the development of new CADe tools with fewer false-positive results per examination. Classical machine learning and deep-learning methods were also used for pulmonary nodule segmentation allowing nodule volumetry and pulmonary nodule characterization. For pulmonary nodule characterization, radiomics and deep-learning approaches were used. Data from the National Lung Cancer Screening Trial (NLST) allowed the development of several computer-aided diagnostic (CADx) tools for diagnosing lung cancer on chest computed tomography. Finally, AI has been used as a means to perform virtual biopsies and to predict response to treatment or survival. Thus, many detection, characterization and stratification tools have been proposed, some of which are commercially available.

Artificial intelligence: A critical review of applications for lung nodule and lung cancer

Artificial intelligence (AI) is a broad concept that usually refers to computer programs that can learn from data and perform certain specific tasks. In the recent years, the growth of deep learning, a successful technique for computer vision tasks that does not require explicit programming, coupled with the availability of large imaging databases fostered the development of multiple applications in the medical imaging field, especially for lung nodules and lung cancer, mostly through convolutional neural networks (CNN). Some of the first applications of AI is this field were dedicated to automated detection of lung nodules on X-ray and computed tomography (CT) examinations, with performances now reaching or exceeding those of radiologists. For lung nodule segmentation, CNN-based algorithms applied to CT images show excellent spatial overlap index with manual segmentation, even for irregular and ground glass nodules. A third application of AI is the classification of lung nodules between malignant and benign, which could limit the number of follow-up CT examinations for less suspicious lesions. Several algorithms have demonstrated excellent capabilities for the prediction of the malignancy risk when a nodule is discovered. These different applications of AI for lung nodules are particularly appealing in the context of lung cancer screening. In the field of lung cancer, AI tools applied to lung imaging have been investigated for distinct aims. First, they could play a role for the non-invasive characterization of tumors, especially for histological subtype and somatic mutation predictions, with a potential therapeutic impact. Additionally, they could help predict the patient prognosis, in combination to clinical data. Despite these encouraging perspectives, clinical implementation of AI tools is only beginning because of the lack of generalizability of published studies, of an inner obscure working and because of limited data about the impact of such tools on the radiologists' decision and on the patient outcome. Radiologists must be active participants in the process of evaluating AI tools, as such tools could support their daily work and offer them more time for high added value tasks.

A Comprehensive Survey on the Progress, Process, and Challenges of Lung Cancer Detection and Classification

Lung cancer is the primary reason of cancer deaths worldwide, and the percentage of death rate is increasing step by step. There are chances of recovering from lung cancer by detecting it early. In any case, because the number of radiologists is limited and they have been working overtime, the increase in image data makes it hard for them to evaluate the images accurately. As a result, many researchers have come up with automated ways to predict the growth of cancer cells using medical imaging methods in a quick and accurate way. Previously, a lot of work was done on computer-aided detection (CADe) and computer-aided diagnosis (CADx) in computed tomography (CT) scan, magnetic resonance imaging (MRI), and X-ray with the goal of effective detection and segmentation of pulmonary nodule, as well as classifying nodules as malignant or benign. But still, no complete comprehensive review that includes all aspects of lung cancer has been done. In this paper, every aspect of lung cancer is discussed in detail, including datasets, image preprocessing, segmentation methods, optimal feature extraction and selection methods, evaluation measurement matrices, and classifiers. Finally, the study looks into several lung cancer-related issues with possible solutions.

Application of nnU-Net for Automatic Segmentation of Lung Lesions on CT Images and Its Implication for Radiomic Models

Radiomics investigates the predictive role of quantitative parameters calculated from radiological images. In oncology, tumour segmentation constitutes a crucial step of the radiomic workflow. Manual segmentation is time-consuming and prone to inter-observer variability. In this study, a state-of-the-art deep-learning network for automatic segmentation (nnU-Net) was applied to computed tomography images of lung tumour patients, and its impact on the performance of survival radiomic models was assessed. In total, 899 patients were included, from two proprietary and one public datasets. Different network architectures (2D, 3D) were trained and tested on different combinations of the datasets. Automatic segmentations were compared to reference manual segmentations performed by physicians using the DICE similarity coefficient. Subsequently, the accuracy of radiomic models for survival classification based on either manual or automatic segmentations were compared, considering both hand-crafted and deep-learning features. The best agreement between automatic and manual contours (DICE = 0.78 ± 0.12) was achieved averaging 2D and 3D predictions and applying customised post-processing. The accuracy of the survival classifier (ranging between 0.65 and 0.78) was not statistically different when using manual versus automatic contours, both with hand-crafted and deep features. These results support the promising role nnU-Net can play in automatic segmentation, accelerating the radiomic workflow without impairing the models' accuracy. Further investigations on different clinical endpoints and populations are encouraged to confirm and generalise these findings.

Predictive performance of radiomic models based on features extracted from pretrained deep networks

OBJECTIVES

In radiomics, generic texture and morphological features are often used for modeling. Recently, features extracted from pretrained deep networks have been used as an alternative. However, extracting deep features involves several decisions, and it is unclear how these affect the resulting models. Therefore, in this study, we considered the influence of such choices on the predictive performance.

METHODS

On ten publicly available radiomic datasets, models were trained using feature sets that differed in terms of the utilized network architecture, the layer of feature extraction, the used set of slices, the use of segmentation, and the aggregation method. The influence of these choices on the predictive performance was measured using a linear mixed model. In addition, models with generic features were trained and compared in terms of predictive performance and correlation.

RESULTS

No single choice consistently led to the best-performing models. In the mixed model, the choice of architecture (AUC + 0.016; p < 0.001), the level of feature extraction (AUC + 0.016; p < 0.001), and using all slices (AUC + 0.023; p < 0.001) were highly significant; using the segmentation had a lower influence (AUC + 0.011; p = 0.023), while the aggregation method was insignificant (p = 0.774). Models based on deep features were not significantly better than those based on generic features (p > 0.05 on all datasets). Deep feature sets correlated moderately with each other (r = 0.4), in contrast to generic feature sets (r = 0.89).

CONCLUSIONS

Different choices have a significant effect on the predictive performance of the resulting models; however, for the highest performance, these choices should be optimized during cross-validation.

Machine Learning Radiomics Model for External and Internal Respiratory Motion Correlation Prediction in Lung Tumor

Objectives: The complexity and specificity of lung tumor motion render it necessary to determine the external and internal correlation individually before applying indirect tumor tracking. However, the correlation cannot be determined from patient respiratory and tumor clinical characteristics before treatment. The purpose of this study is to present a machine learning model for an external/internal correlation prediction that is based on computed tomography (CT) radiomic features. Methods: 4-dimensional computed tomography (4DCT) images of 67 patients were collected retrospectively, and the external/internal correlation of lung tumor was calculated based on Spearman's rank correlation coefficient. Radiomic features were extracted from average intensity projection and the light gradient boosting machine (LightGBM)-based cross-validation (the recursive elimination method) was used for feature selection. The LightGBM framework forecasting models with classification thresholds 0.7, 0.8, and 0.9 are established using stratified 5-fold cross-validation. Model performance was assessed using receiver operating characteristics, sensitivity, and specificity. Results: There were 16, 18, and 13 features selected for models 0.7, 0.8, and 0.9, respectively. Texture features are of great importance in external/internal correlation prediction compared to other features in all models. The sensitivities of the predictions in models 0.7, 0.8, and 0.9 were 0.800 ± 0.126, 0.829 ± 0.140, and 0.864 ± 0.086, respectively. The specificities were 0.771 ± 0.114, 0.936 ± 0.0581, and 0.839 ± 0.101, whereas the area under the curve (AUC) was 0.837, 0.946, and 0.877, respectively. Conclusions: Our findings indicate that radiomics is an effective tool for respiratory motion correlation prediction, which can extract tumor motion characteristics. We proposed a machine learning framework for correlation prediction in the motion management strategy for lung tumor patients.

Pattern Classification for Ovarian Tumors by Integration of Radiomics and Deep Learning Features

BACKGROUND

Ovarian tumor is a common female genital tumor, among which malignant tumors have a poor prognosis. The survival rate of 70% of patients with ovarian cancer is less than 5 years, while benign ovarian tumor is better, so the early diagnosis of ovarian cancer is important for the treatment and prognosis of patients.

OBJECTIVES

Our aim is to establish a classification model for ovarian tumors.

METHODS

We extracted radiomics and deep learning features from patients'CT images. The four-step feature selection algorithm proposed in this paper was used to obtain the optimal combination of features, then, a classification model was developed by combining those selected features and support vector machine. The receiver operating characteristic curve and an area under the curve (AUC) analysis were used to evaluate the performance of the classification model in both the training and test cohort.

RESULTS

The classification model, which combined radiomics features with deep learning features, demonstrated better classification performance with respect to the radiomics features model alone in training cohort (AUC 0.9289 vs. 0.8804, P < 0.0001, accuracy 0.8970 vs. 0.7993, P < 0.0001), and significantly improve the performance in the test cohort (AUC 0.9089 vs. 0.8446, P = 0.001, accuracy 0.8296 vs. 0.7259, P < 0.0001).

CONCLUSION

The experiments showed that deep learning features play an active role in the construction of classification model, and the proposed classification model achieved excellent classification performance, which can potentially become a new auxiliary diagnostic tool.

Application of Radiomics in Predicting Treatment Response to Neoadjuvant Chemoradiotherapy in Locally Advanced Rectal Cancer: Strategies and Challenges

Neoadjuvant chemoradiotherapy (nCRT) followed by total mesorectal excision is the standard treatment for locally advanced rectal cancer (LARC). A noninvasive preoperative prediction method should greatly assist in the evaluation of response to nCRT and for the development of a personalized strategy for patients with LARC. Assessment of nCRT relies on imaging and radiomics can extract valuable quantitative data from medical images. In this review, we examined the status of radiomic application for assessing response to nCRT in patients with LARC and indicated a potential direction for future research.

Deep multiple instance learning for predicting chemotherapy response in non-small cell lung cancer using pretreatment CT images

The individual prognosis of chemotherapy is quite different in non-small cell lung cancer (NSCLC). There is an urgent need to precisely predict and assess the treatment response. To develop a deep multiple-instance learning (DMIL) based model for predicting chemotherapy response in NSCLC in pretreatment CT images. Two datasets of NSCLC patients treated with chemotherapy as the first-line treatment were collected from two hospitals. Dataset 1 (163 response and 138 nonresponse) was used to train, validate, and test the DMIL model and dataset 2 (22 response and 20 nonresponse) was used as the external validation cohort. Five backbone networks in the feature extraction module and three pooling methods were compared. The DMIL with a pre-trained VGG16 backbone and an attention mechanism pooling performed the best, with an accuracy of 0.883 and area under the curve (AUC) of 0.982 on Dataset 1. While using max pooling and convolutional pooling, the AUC was 0.958 and 0.931, respectively. In Dataset 2, the best DMIL model produced an accuracy of 0.833 and AUC of 0.940. Deep learning models based on the MIL can predict chemotherapy response in NSCLC using pretreatment CT images and the pre-trained VGG16 with attention mechanism pooling yielded better predictions.

Accurate preoperative staging and HER2 status prediction of gastric cancer by the deep learning system based on enhanced computed tomography

Purpose

To construct the deep learning system (DLS) based on enhanced computed tomography (CT) images for preoperative prediction of staging and human epidermal growth factor receptor 2 (HER2) status in gastric cancer patients.

Methods

The raw enhanced CT image dataset consisted of CT images of 389 patients in the retrospective cohort, The Cancer Imaging Archive (TCIA) cohort, and the prospective cohort. DLS was developed by transfer learning for tumor detection, staging, and HER2 status prediction. The pre-trained Yolov5, EfficientNet, EfficientNetV2, Vision Transformer (VIT), and Swin Transformer (SWT) were studied. The tumor detection and staging dataset consisted of 4860 enhanced CT images and annotated tumor bounding boxes. The HER2 state prediction dataset consisted of 38900 enhanced CT images.

Results

The DetectionNet based on Yolov5 realized tumor detection and staging and achieved a mean Average Precision (IoU=0.5) (mAP_0.5) of 0.909 in the external validation cohort. The VIT-based PredictionNet performed optimally in HER2 status prediction with the area under the receiver operating characteristics curve (AUC) of 0.9721 and 0.9995 in the TCIA cohort and prospective cohort, respectively. DLS included DetectionNet and PredictionNet had shown excellent performance in CT image interpretation.

Conclusion

This study developed the enhanced CT-based DLS to preoperatively predict the stage and HER2 status of gastric cancer patients, which will help in choosing the appropriate treatment to improve the survival of gastric cancer patients.

CT images-based 3D convolutional neural network to predict early recurrence of solitary hepatocellular carcinoma after radical hepatectomy

PURPOSE The high rate of recurrence of hepatocellular carcinoma (HCC) after radical hepatectomy is an important factor that affects the long-term survival of patients. This study aimed to develop a computed tomography (CT) images-based 3-dimensional (3D) convolutional neural network (CNN) for the preoperative prediction of early recurrence (ER) (≤2 years) after radical hepatectomy in patients with solitary HCC and to compare the effects of segmentation sampling (SS) and non-segmentation sampling (NSS) on the prediction performance of 3D-CNN. METHODS Contrast-enhanced CT images of 220 HCC patients were used in this study (training group=178 and test group=42). We used SS and NSS to select the volume-of-interest to train SS-3D-CNN and NSS-3D-CNN separately. The prediction accuracy was evaluated using the test group. Finally, gradient-weighted class activation mappings (Grad-CAMs) were plotted to analyze the difference of prediction logic between the SS-3D-CNN and NSS-3D-CNN. RESULTS The areas under the receiver operating characteristic curves (AUCs) of the SS-3D-CNN and NSS3D-CNN in the training group were 0.824 (95% CI: 0.764-0.885) and 0.868 (95% CI: 0.815-0.921). The AUC of the SS-3D-CNN and NSS-3D-CNN in the test group were 0.789 (95% CI: 0.637-0.941) and 0.560 (95% CI: 0.378-0.742). The SS-3D-CNN could stratify patients into low- and high-risk groups, with significant differences in recurrence-free survival (RFS) (P < .001). But NSS-3D-CNN could not effectively stratify them in the test group. According to the Grad-CAMs, compared with SS-3D-CNN, NSS-3D-CNN was obviously interfered by the nearby tissues. CONCLUSION SS-3D-CNN may be of clinical use for identifying high-risk patients and formulating individualized treatment and follow-up strategies. SS is better than NSS in improving the performance of 3D-CNN in our study.

Big data in basic and translational cancer research

Historically, the primary focus of cancer research has been molecular and clinical studies of a few essential pathways and genes. Recent years have seen the rapid accumulation of large-scale cancer omics data catalysed by breakthroughs in high-throughput technologies. This fast data growth has given rise to an evolving concept of 'big data' in cancer, whose analysis demands large computational resources and can potentially bring novel insights into essential questions. Indeed, the combination of big data, bioinformatics and artificial intelligence has led to notable advances in our basic understanding of cancer biology and to translational advancements. Further advances will require a concerted effort among data scientists, clinicians, biologists and policymakers. Here, we review the current state of the art and future challenges for harnessing big data to advance cancer research and treatment.

Gross tumour volume radiomics for prognostication of recurrence & death following radical radiotherapy for NSCLC

Recurrence occurs in up to 36% of patients treated with curative-intent radiotherapy for NSCLC. Identifying patients at higher risk of recurrence for more intensive surveillance may facilitate the earlier introduction of the next line of treatment. We aimed to use radiotherapy planning CT scans to develop radiomic classification models that predict overall survival (OS), recurrence-free survival (RFS) and recurrence two years post-treatment for risk-stratification. A retrospective multi-centre study of >900 patients receiving curative-intent radiotherapy for stage I-III NSCLC was undertaken. Models using radiomic and/or clinical features were developed, compared with 10-fold cross-validation and an external test set, and benchmarked against TNM-stage. Respective validation and test set AUCs (with 95% confidence intervals) for the radiomic-only models were: (1) OS: 0.712 (0.592–0.832) and 0.685 (0.585–0.784), (2) RFS: 0.825 (0.733–0.916) and 0.750 (0.665–0.835), (3) Recurrence: 0.678 (0.554–0.801) and 0.673 (0.577–0.77). For the combined models: (1) OS: 0.702 (0.583–0.822) and 0.683 (0.586–0.78), (2) RFS: 0.805 (0.707–0.903) and 0·755 (0.672–0.838), (3) Recurrence: 0·637 (0.51–0.·765) and 0·738 (0.649–0.826). Kaplan-Meier analyses demonstrate OS and RFS difference of >300 and >400 days respectively between low and high-risk groups. We have developed validated and externally tested radiomic-based prediction models. Such models could be integrated into the routine radiotherapy workflow, thus informing a personalised surveillance strategy at the point of treatment. Our work lays the foundations for future prospective clinical trials for quantitative personalised risk-stratification for surveillance following curative-intent radiotherapy for NSCLC.

Radiomics combined with clinical features in distinguishing non-calcifying tuberculosis granuloma and lung adenocarcinoma in small pulmonary nodules

Aim

To evaluate the performance of radiomics models with the combination of clinical features in distinguishing non-calcified tuberculosis granuloma (TBG) and lung adenocarcinoma (LAC) in small pulmonary nodules.

Methodology

We conducted a retrospective analysis of 280 patients with pulmonary nodules confirmed by surgical biopsy from January 2017 to December 2020. Samples were divided into LAC group (n = 143) and TBG group (n = 137). We assigned them to a training dataset (n = 196) and a testing dataset (n = 84). Clinical features including gender, age, smoking, CT appearance (size, location, spiculated sign, lobulated shape, vessel convergence, and pleural indentation) were extracted and included in the radiomics models. 3D slicer and FAE software were used to delineate the Region of Interest (ROI) and extract clinical features. The performance of the model was evaluated by the Area Under the Receiver Operating Characteristic (ROC) Curve (AUC).

Results

Based on the model selection, clinical features gender, and age in the LAC group and TBG group showed a significant difference in both datasets (P < 0.05). CT appearance lobulated shape was also significantly different in the LAC group and TBG group (Training dataset, P = 0.034; Testing dataset, P = 0.030). AUC were 0.8344 (95% CI [0.7712-0.8872]) and 0.751 (95% CI [0.6382-0.8531]) in training and testing dataset, respectively.

Conclusion

With the capacity to detect differences between TBG and LAC based on their clinical features, radiomics models with a combined of clinical features may function as the potential non-invasive tool for distinguishing TBG and LAC in small pulmonary nodules.

Delta radiomics model for the prediction of progression-free survival time in advanced non-small-cell lung cancer patients after immunotherapy

Objective

To assess the validity of pre- and posttreatment computed tomography (CT)-based radiomics signatures and delta radiomics signatures for predicting progression-free survival (PFS) in stage III-IV non-small-cell lung cancer (NSCLC) patients after immune checkpoint inhibitor (ICI) therapy.

Methods

Quantitative image features of the largest primary lung tumours were extracted on CT-enhanced imaging at baseline (time point 0, TP0) and after the 2^nd-3^rd immunotherapy cycles (time point 1, TP1). The critical features were selected to construct TP0, TP1 and delta radiomics signatures for the risk stratification of patient survival after ICI treatment. In addition, a prediction model integrating the clinicopathologic risk characteristics and phenotypic signature was developed for the prediction of PFS.

Results

The C-index of TP0, TP1 and delta radiomics models in the training and validation cohort were 0.64, 0.75, 0.80, and 0.61, 0.68, 0.78, respectively. The delta radiomics score exhibited good accuracy for distinguishing patients with slow and rapid progression to ICI treatment. The predictive accuracy of the combined prediction model was higher than that of the clinical prediction model in both training and validation sets (P<0.05), with a C-index of 0.83 and 0.70, respectively. Additionally, the delta radiomics model (C-index of 0.86) had a higher predictive accuracy compared to PD-L1 expression (C-index of 0.50) (P<0.0001).

Conclusions

The combined prediction model including clinicopathologic characteristics (tumour anatomical classification and brain metastasis) and the delta radiomics signature could achieve the individualized prediction of PFS in ICIs-treated NSCLC patients.

A transformer-based deep-learning approach for classifying brain metastases into primary organ sites using clinical whole-brain MRI images

Treatment decisions for brain metastatic disease rely on knowledge of the primary organ site and are currently made with biopsy and histology. Here, we develop a deep-learning approach for accurate non-invasive digital histology with whole-brain magnetic resonance imaging (MRI) data. Contrast-enhanced T1-weighted and fast spoiled gradient echo brain MRI exams (n = 1,582) were preprocessed and input to the proposed deep-learning workflow for tumor segmentation, modality transfer, and primary site classification into one of five classes. Tenfold cross-validation generated an overall area under the receiver operating characteristic curve (AUC) of 0.878 (95% confidence interval [CI]: 0.873,0.883). These data establish that whole-brain imaging features are discriminative enough to allow accurate diagnosis of the primary organ site of malignancy. Our end-to-end deep radiomic approach has great potential for classifying metastatic tumor types from whole-brain MRI images. Further refinement may offer an invaluable clinical tool to expedite primary cancer site identification for precision treatment and improved outcomes.

Inferring FDG-PET-positivity of lymph node metastases in proven lung cancer from contrast-enhanced CT using radiomics and machine learning

Background

We evaluated the role of radiomics applied to contrast-enhanced computed tomography (CT) in the detection of lymph node (LN) metastases in patients with known lung cancer compared to ¹⁸F-fluorodeoxyglucose positron emission tomography (PET)/CT as a reference.

Methods

This retrospective analysis included 381 patients with 1,799 lymph nodes (450 malignant, 1,349 negative). The data set was divided into a training and validation set. A radiomics analysis with 4 filters and 6 algorithms resulting in 24 different radiomics signatures and a bootstrap algorithm (Bagging) with 30 bootstrap iterations was performed. A decision curve analysis was applied to generate a net benefit to compare the radiomics signature to two expert radiologists as one-by-one and as a prescreening tool in combination with the respective radiologist and only the radiologists.

Results

All 24 modeling methods showed good and reliable discrimination for malignant/benign LNs (area under the curve 0.75−0.87). The decision curve analysis showed a net benefit for the least absolute shrinkage and selection operator (LASSO) classifier for the entire probability range and outperformed the expert radiologists except for the high probability range. Using the radiomics signature as a prescreening tool for the radiologists did not improve net benefit.

Conclusions

Radiomics showed good discrimination power irrespective of the modeling technique in detecting LN metastases in patients with known lung cancer. The LASSO classifier was a suitable diagnostic tool and even outperformed the expert radiologists, except for high probabilities. Radiomics failed to improve clinical benefit as a prescreening tool.

Supplementary Information

The online version contains supplementary material available at 10.1186/s41747-022-00296-8.

The value of combined PET/MRI, CT and clinical metabolic parameters in differentiating lung adenocarcinoma from squamous cell carcinoma

Objective

This study aimed to study the diagnostic efficacy of positron emission tomography (PET)/magnetic resonance imaging (MRI), computed tomography (CT) and clinical metabolic parameters in predicting the histological classification of lung adenocarcinoma (ADC) and squamous cell carcinoma (SCC).

Methods

PET/MRI, CT and clinical metabolic data of 80 patients with lung ADC or SCC were retrospectively collected. According to the pathological results from surgery or fiberscopy, the patients were diagnosed with lung ADC (47 cases) or SCC (33 cases). All 80 patients were divided into a training group (64 cases), an internal testing group (8 cases) and an external testing group (8 cases) in the ratio of 8:1:1. Nine models were constructed by integrating features from different modalities. The Gaussian classifier was used to differentiate ADC and SCC. The prediction ability was evaluated using the receiver operating characteristic curve. The area under the curve (AUC) of the models was compared using Delong’s test. Based on the best composite model, a nomogram was established and evaluated with a calibration curve, decision curve and clinical impact curve.

Results

The composite model (PET/MRI + CT + Clinical) owned the highest AUC values in the training, internal testing and external testing sets, respectively. In the training set, significant differences in the AUC were found between the composite model and other models except for the PET/MRI + CT model. The calibration curves showed good consistency between the predicted output and actual disease. The decision curve analysis and clinical impact curves demonstrated that the composite model increased the clinical net benefit for predicting lung cancer subtypes.

Conclusion

The composite prediction model of PET/MRI + CT + Clinical better distinguished ADC from SCC pathological subtypes preoperatively and achieved clinical benefits, thus providing an accurate clinical diagnosis.

Radiomics and deep learning methods for the prediction of 2-year overall survival in LUNG1 dataset

In this study, we tested and compared radiomics and deep learning-based approaches on the public LUNG1 dataset, for the prediction of 2-year overall survival (OS) in non-small cell lung cancer patients. Radiomic features were extracted from the gross tumor volume using Pyradiomics, while deep features were extracted from bi-dimensional tumor slices by convolutional autoencoder. Both radiomic and deep features were fed to 24 different pipelines formed by the combination of four feature selection/reduction methods and six classifiers. Direct classification through convolutional neural networks (CNNs) was also performed. Each approach was investigated with and without the inclusion of clinical parameters. The maximum area under the receiver operating characteristic on the test set improved from 0.59, obtained for the baseline clinical model, to 0.67 ± 0.03, 0.63 ± 0.03 and 0.67 ± 0.02 for models based on radiomic features, deep features, and their combination, and to 0.64 ± 0.04 for direct CNN classification. Despite the high number of pipelines and approaches tested, results were comparable and in line with previous works, hence confirming that it is challenging to extract further imaging-based information from the LUNG1 dataset for the prediction of 2-year OS.

Systematic review for lung cancer detection and lung nodule classification: Taxonomy, challenges, and recommendation future works

Nowadays, lung cancer is one of the most dangerous diseases that require early diagnosis. Artificial intelligence has played an essential role in the medical field in general and in analyzing medical images and diagnosing diseases in particular, as it can reduce human errors that can occur with the medical expert when analyzing medical image. In this research study, we have done a systematic survey of the research published during the last 5 years in the diagnosis of lung cancer classification of lung nodules in 4 reliable databases (Science Direct, Scopus, web of science, and IEEE), and we selected 50 research paper using systematic literature review. The goal of this review work is to provide a concise overview of recent advancements in lung cancer diagnosis issues by machine learning and deep learning algorithms. This article summarizes the present state of knowledge on the subject. Addressing the findings offered in recent research publications gives the researchers a better grasp of the topic. We checked all the characteristics, such as challenges, recommendations for future work were analyzed in detail, and the published datasets and their source were presented to facilitate the researchers’ access to them and use it to develop the results achieved previously.

Possible Bias in Supervised Deep Learning Algorithms for CT Lung Nodule Detection and Classification

Simple Summary

Artificial Intelligence (AI) algorithms can assist clinicians in their daily tasks by automatically detecting and/or classifying nodules in chest CT scans. Bias of such algorithms is one of the reasons why implementation of them in clinical practice is still not widely adopted. There is no published review on the bias that these algorithms may contain. This review aims to present different types of bias in such algorithms and present possible ways to mitigate them. Only then it would be possible to ensure that these algorithms work as intended under many different clinical settings.

Abstract

Artificial Intelligence (AI) algorithms for automatic lung nodule detection and classification can assist radiologists in their daily routine of chest CT evaluation. Even though many AI algorithms for these tasks have already been developed, their implementation in the clinical workflow is still largely lacking. Apart from the significant number of false-positive findings, one of the reasons for that is the bias that these algorithms may contain. In this review, different types of biases that may exist in chest CT AI nodule detection and classification algorithms are listed and discussed. Examples from the literature in which each type of bias occurs are presented, along with ways to mitigate these biases. Different types of biases can occur in chest CT AI algorithms for lung nodule detection and classification. Mitigation of them can be very difficult, if not impossible to achieve completely.

Immunotherapy in non-small cell lung cancer: Past, present, and future directions

Many decades in the making, immunotherapy has demonstrated its ability to produce durable responses in several cancer types. In the last decade, immunotherapy has shown itself to be a viable therapeutic approach for non-small cell lung cancer (NSCLC). Several clinical trials have established the efficacy of immune checkpoint blockade (ICB), particularly in the form of anti-programmed death 1 (PD-1) antibodies, anti-cytotoxic T-lymphocyte-associated protein 4 (CTLA-4) antibodies and anti-programmed death 1 ligand (PD-L1) antibodies. Many trials have shown progression free survival (PFS) and overall survival (OS) benefit with either ICB alone or in combination with chemotherapy when compared to chemotherapy alone. The identification of biomarkers to predict response to immunotherapy continues to be evaluated. The future of immunotherapy in lung cancer continues to hold promise with the development of combination therapies, cytokine modulating therapies and cellular therapies. Lastly, we expect that innovative advances in technology, such as artificial intelligence (AI) and machine learning, will begin to play a role in the future care of patients with lung cancer.

Prediction of lung malignancy progression and survival with machine learning based on pre-treatment FDG-PET/CT

Background

Pre-treatment FDG-PET/CT scans were analyzed with machine learning to predict progression of lung malignancies and overall survival (OS).

Methods

A retrospective review across three institutions identified patients with a pre-procedure FDG-PET/CT and an associated malignancy diagnosis. Lesions were manually and automatically segmented, and convolutional neural networks (CNNs) were trained using FDG-PET/CT inputs to predict malignancy progression. Performance was evaluated using area under the receiver operating characteristic curve (AUC), accuracy, sensitivity, and specificity. Image features were extracted from CNNs and by radiomics feature extraction, and random survival forests (RSF) were constructed to predict OS. Concordance index (C-index) and integrated brier score (IBS) were used to evaluate OS prediction.

Findings

1168 nodules (n=965 patients) were identified. 792 nodules had progression and 376 were progression-free. The most common malignancies were adenocarcinoma (n=740) and squamous cell carcinoma (n=179). For progression risk, the PET+CT ensemble model with manual segmentation (accuracy=0.790, AUC=0.876) performed similarly to the CT only (accuracy=0.723, AUC=0.888) and better compared to the PET only (accuracy=0.664, AUC=0.669) models. For OS prediction with deep learning features, the PET+CT+clinical RSF ensemble model (C-index=0.737) performed similarly to the CT only (C-index=0.730) and better than the PET only (C-index=0.595), and clinical only (C-index=0.595) models. RSF models constructed with radiomics features had comparable performance to those with CNN features.

Interpretation

CNNs trained using pre-treatment FDG-PET/CT and extracted performed well in predicting lung malignancy progression and OS. OS prediction performance with CNN features was comparable to a radiomics approach. The prognostic models could inform treatment options and improve patient care.

Funding

NIH NHLBI training grant (5T35HL094308-12, John Sollee).

Prediction of 5-year progression-free survival in advanced nasopharyngeal carcinoma with pretreatment PET/CT using multi-modality deep learning-based radiomics

Objective

Deep learning-based radiomics (DLR) has achieved great success in medical image analysis and has been considered a replacement for conventional radiomics that relies on handcrafted features. In this study, we aimed to explore the capability of DLR for the prediction of 5-year progression-free survival (PFS) in advanced nasopharyngeal carcinoma (NPC) using pretreatment PET/CT images.

Methods

A total of 257 patients (170/87 patients in internal/external cohorts) with advanced NPC (TNM stage III or IVa) were enrolled. We developed an end-to-end multi-modality DLR model, in which a 3D convolutional neural network was optimized to extract deep features from pretreatment PET/CT images and predict the probability of 5-year PFS. The TNM stage, as a high-level clinical feature, could be integrated into our DLR model to further improve the prognostic performance. For a comparison between conventional radiomics and DLR, 1,456 handcrafted features were extracted, and optimal conventional radiomics methods were selected from 54 cross-combinations of six feature selection methods and nine classification methods. In addition, risk group stratification was performed with clinical signature, conventional radiomics signature, and DLR signature.

Results

Our multi-modality DLR model using both PET and CT achieved higher prognostic performance (area under the receiver operating characteristic curve (AUC) = 0.842 ± 0.034 and 0.823 ± 0.012 for the internal and external cohorts) than the optimal conventional radiomics method (AUC = 0.796 ± 0.033 and 0.782 ± 0.012). Furthermore, the multi-modality DLR model outperformed single-modality DLR models using only PET (AUC = 0.818 ± 0.029 and 0.796 ± 0.009) or only CT (AUC = 0.657 ± 0.055 and 0.645 ± 0.021). For risk group stratification, the conventional radiomics signature and DLR signature enabled significant difference between the high- and low-risk patient groups in both the internal and external cohorts (p < 0.001), while the clinical signature failed in the external cohort (p = 0.177).

Conclusion

Our study identified potential prognostic tools for survival prediction in advanced NPC, which suggests that DLR could provide complementary values to the current TNM staging.

Prognostic and predictive value of radiomic signature in stage I lung adenocarcinomas following complete lobectomy

BACKGROUND

The overall survival (OS) of stage I operable lung cancer is relatively low, and not all patients can benefit from adjuvant chemotherapy. This study aimed to develop and validate a radiomic signature (RS) for prediction of OS and adjuvant chemotherapy candidates in stage I lung adenocarcinoma.

METHODS

A total of 474 patients from 2 centers were divided into 1 training (n = 287), 1 internal validation (n = 122), and 1 external validation (n = 65) cohorts. We extracted 1218 radiomic features from preoperative CT images and constructed RS. We further investigated the prognostic value of the RS in survival analysis. Interaction between treatment and RS was assessed to evaluate its predictive value. Propensity score matching (PSM) was conducted.

RESULTS

Overall, 474 eligible patients with stage I lung adenocarcinoma (214 men [45.1%]; median age, 60 years) were identified. The RS was significantly associated with OS in the training and two validation cohorts (hazard ratios [HRs]  >  = 3.22). In multivariable analysis, the RS remained an independent prognostic factor adjusting for clinicopathologic variables (adjusted HRs >  = 2.63). The prognostic value of RS was also confirmed in PSM analysis. In stage I patients, the interaction between RS status and adjuvant chemotherapy was significant (interaction P = 0.020). Within the stratified analysis, good chemotherapy efficacy was only observed for patients with stage IB disease (interaction P < 0.001).

CONCLUSIONS

Our results suggested that the radiomic signature was associated with overall survival in patients with stage I lung adenocarcinoma and might predict adjuvant chemotherapy benefit, especially in stage IB patients. The potential of radiomic signature as a noninvasive predictor needed to be confirmed in future studies.

The impact of inter-observer variation in delineation on robustness of radiomics features in non-small cell lung cancer

Artificial intelligence and radiomics have the potential to revolutionise cancer prognostication and personalised treatment. Manual outlining of the tumour volume for extraction of radiomics features (RF) is a subjective process. This study investigates robustness of RF to inter-observer variation (IOV) in contouring in lung cancer. We utilised two public imaging datasets: ‘NSCLC-Radiomics’ and ‘NSCLC-Radiomics-Interobserver1’ (‘Interobserver’). For ‘NSCLC-Radiomics’, we created an additional set of manual contours for 92 patients, and for ‘Interobserver’, there were five manual and five semi-automated contours available for 20 patients. Dice coefficients (DC) were calculated for contours. 1113 RF were extracted including shape, first order and texture features. Intraclass correlation coefficient (ICC) was computed to assess robustness of RF to IOV. Cox regression analysis for overall survival (OS) was performed with a previously published radiomics signature. The median DC ranged from 0.81 (‘NSCLC-Radiomics’) to 0.85 (‘Interobserver’—semi-automated). The median ICC for the ‘NSCLC-Radiomics’, ‘Interobserver’ (manual) and ‘Interobserver’ (semi-automated) were 0.90, 0.88 and 0.93 respectively. The ICC varied by feature type and was lower for first order and gray level co-occurrence matrix (GLCM) features. Shape features had a lower median ICC in the ‘NSCLC-Radiomics’ dataset compared to the ‘Interobserver’ dataset. Survival analysis showed similar separation of curves for three of four RF apart from ‘original_shape_Compactness2’, a feature with low ICC (0.61). The majority of RF are robust to IOV, with first order, GLCM and shape features being the least robust. Semi-automated contouring improves feature stability. Decreased robustness of a feature is significant as it may impact upon the features’ prognostic capability.

Diagnostic Accuracy of Deep Learning and Radiomics in Lung Cancer Staging: A Systematic Review and Meta-Analysis

Background

Artificial intelligence has far surpassed previous related technologies in image recognition and is increasingly used in medical image analysis. We aimed to explore the diagnostic accuracy of the models based on deep learning or radiomics for lung cancer staging.

Methods

Studies were systematically reviewed using literature searches from PubMed, EMBASE, Web of Science, and Wanfang Database, according to PRISMA guidelines. Studies about the diagnostic accuracy of radiomics and deep learning, including the identifications of lung cancer, tumor types, malignant lung nodules and lymph node metastase, were included. After identifying the articles, the methodological quality was assessed using the QUADAS-2 checklist. We extracted the characteristic of each study; the sensitivity, specificity, and AUROC for lung cancer diagnosis were summarized for subgroup analysis.

Results

The systematic review identified 19 eligible studies, of which 14 used radiomics models and 5 used deep learning models. The pooled AUROC of 7 studies to determine whether patients had lung cancer was 0.83 (95% CI 0.78–0.88). The pooled AUROC of 9 studies to determine whether patients had NSCLC was 0.78 (95% CI 0.73–0.83). The pooled AUROC of the 6 studies that determined patients had malignant lung nodules was 0.79 (95% CI 0.77–0.82). The pooled AUROC of the other 6 studies that determined whether patients had lymph node metastases was 0.74 (95% CI 0.66–0.82).

Conclusion

The models based on deep learning or radiomics have the potential to improve diagnostic accuracy for lung cancer staging.

Systematic Review Registration

https://inplasy.com/inplasy-2022-3-0167/, identifier: INPLASY202230167.

Barriers to Use Artificial Intelligence Methodologies in Health Technology Assessment in Central and East European Countries

The aim of this paper is to identify the barriers that are specifically relevant to the use of Artificial Intelligence (AI)-based evidence in Central and Eastern European (CEE) Health Technology Assessment (HTA) systems. The study relied on two main parallel sources to identify barriers to use AI methodologies in HTA in CEE, including a scoping literature review and iterative focus group meetings with HTx team members. Most of the other selected articles discussed AI from a clinical perspective (n = 25), and the rest are from regulatory perspective (n = 13), and transfer of knowledge point of view (n = 3). Clinical areas studied are quite diverse—from pediatric, diabetes, diagnostic radiology, gynecology, oncology, surgery, psychiatry, cardiology, infection diseases, and oncology. Out of all 38 articles, 25 (66%) describe the AI method and the rest are more focused on the utilization barriers of different health care services and programs. The potential barriers could be classified as data related, methodological, technological, regulatory and policy related, and human factor related. Some of the barriers are quite similar, especially concerning the technologies. Studies focusing on the AI usage for HTA decision making are scarce. AI and augmented decision making tools are a novel science, and we are in the process of adapting it to existing needs. HTA as a process requires multiple steps, multiple evaluations which rely on heterogenous data. Therefore, the observed range of barriers come as a no surprise, and experts in the field need to give their opinion on the most important barriers in order to develop recommendations to overcome them and to disseminate the practical application of these tools.

Multi-scale segmentation squeeze-and-excitation UNet with conditional random field for segmenting lung tumor from CT images

BACKGROUND AND OBJECTIVE

Lung cancer counts among diseases with the highest global morbidity and mortality rates. The automatic segmentation of lung tumors from CT images is of vast significance. However, the segmentation faces several challenges, including variable shapes and different sizes, as well as complicated surrounding tissues.

METHODS

We propose a multi-scale segmentation squeeze-and-excitation UNet with a conditional random field (M-SegSEUNet-CRF) to automatically segment lung tumors from CT images. M-SegSEUNet-CRF employs a multi-scale strategy to solve the problem of variable tumor size. Through the spatially adaptive attention mechanism, the segmentation SE blocks embedded in 3D UNet are utilized to highlight tumor regions. The dense connected CRF framework is further added to delineate tumor boundaries at a detailed level. In total, 759 CT scans of patients with lung cancer were used to train and evaluate the M-SegSEUNet-CRF model (456 for training, 152 for validation, and 151 for test). Meanwhile, the public NSCLC-Radiomics and LIDC datasets have been utilized to validate the generalization of the proposed method. The role of different modules in the M-SegSEUNet-CRF model is analyzed by the ablation experiments, and the performance is compared with that of UNet, its variants and other state-of-the-art models.

RESULTS

M-SegSEUNet-CRF can achieve a Dice coefficient of 0.851 ± 0.071, intersection over union (IoU) of 0.747 ± 0.102, sensitivity of 0.827 ± 0.108, and positive predictive value (PPV) of 0.900 ± 0.107. Without a multi-scale strategy, the Dice coefficient drops to 0.820 ± 0.115; without CRF, it drops to 0.842 ± 0.082, and without both, it drops to 0.806 ± 0.120. M-SegSEUNet-CRF presented a higher Dice coefficient than 3D UNet (0.782 ± 0.115) and its variants (ResUNet, 0.797 ± 0.132; DenseUNet, 0.792 ± 0.111, and UNETR, 0.794 ± 0.130). Although the performance slightly declines with the decrease in tumor volume, M-SegSEUNet-CRF exhibits more obvious advantages than the other comparative models.

CONCLUSIONS

Our M-SegSEUNet-CRF model improves the segmentation ability of UNet through the multi-scale strategy and spatially adaptive attention mechanism. The CRF enables a more precise delineation of tumor boundaries. The M-SegSEUNet-CRF model integrates these characteristics and demonstrates outstanding performance in the task of lung tumor segmentation. It can furthermore be extended to deal with other segmentation problems in the medical imaging field.

A novel pyroptosis-related lncRNA prognostic signature associated with the immune microenvironment in lung squamous cell carcinoma

Background

A growing body of evidence suggests that pyroptosis-related lncRNAs (PRncRNAs) are associated with the prognoses of tumor patients and their tumor immune microenvironments. However, the function of PRlncRNAs in lung squamous cell carcinoma (LUSC) remains unclear.

Methods

We downloaded the transcriptome and clinical information of 551 LUSC samples from the The Cancer Genome Atlas (TCGA) database and randomly separated patients with complete information into two cohorts. Based on the training cohort, we developed a pyroptosis-related signature. We then examined the signature in the test cohort and all retained patients. We also clustered two risk groups in each cohort according to the signature and performed survival analysis, functional analysis, tumor immune microenvironment analysis and drug sensitivity analysis.

Results

A prognostic signature containing five PRlncRNAs (AP001189.1, PICART1, LINC02555, AC010422.4, and AL606469.1) was developed. A principal component analysis (PCA) indicated better differentiation between patients with different risk scores. Kaplan–Meier (K–M) analysis demonstrated poorer survival among patients with higher risk scores (P < 0.001). A receiver operating characteristic (ROC) curve analysis provided evidence confirming the accuracy of the signature, and univariate (p = 0.005) and multivariate (p = 0.008) Cox regression analyses confirmed the independent value of the risk score in prognoses. Clinical subgroup validation indicated that the signature was more suitable for patients with early-stage LUSC. We also created a nomogram to increase the accuracy of the prediction. Moreover, functional analysis revealed that pathways related to tumor development and pyroptosis were enriched in the high-risk group. Furthermore, the prognostic signature was proven to be a predictor of sensitivity to immunotherapy and chemotherapy.

Conclusions

We developed a novel pyroptosis-associated signature with independent value for the prognosis of LUSC patients. PRlncRNAs are closely associated with the tumor immune microenvironment in LUSC and might offer new directions for immunotherapy.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12885-022-09790-z.

A Nomogram Combined Radiomics and Clinical Features as Imaging Biomarkers for Prediction of Visceral Pleural Invasion in Lung Adenocarcinoma

Objectives

To develop and validate a nomogram model based on radiomics features for preoperative prediction of visceral pleural invasion (VPI) in patients with lung adenocarcinoma.

Methods

A total of 659 patients with surgically pathologically confirmed lung adenocarcinoma underwent CT examination. All cases were divided into a training cohort (n = 466) and a validation cohort (n = 193). CT features were analyzed by two chest radiologists. CT radiomics features were extracted from CT images. LASSO regression analysis was applied to determine the most useful radiomics features and construct radiomics score (radscore). A nomogram model was developed by combining the optimal clinical and CT features and the radscore. The model performance was evaluated using ROC analysis, calibration curve and decision curve analysis (DCA).

Results

A total of 1316 radiomics features were extracted. A radiomics signature model with a selection of the six optimal features was developed to identify patients with or without VPI. There was a significant difference in the radscore between the two groups of patients. Five clinical features were retained and contributed as clinical feature models. The nomogram combining clinical features and radiomics features showed improved accuracy, specificity, positive predictive value, and AUC for predicting VPI, compared to the radiomics model alone (specificity: training cohort: 0.89, validation cohort: 0.88, accuracy: training cohort: 0.84, validation cohort: 0.83, AUC: training cohort: 0.89, validation cohort: 0.89). The calibration curve and decision curve analyses suggested that the nomogram with clinical features is beyond the traditional clinical and radiomics features.

Conclusion

A nomogram model combining radiomics and clinical features is effective in non-invasively prediction of VPI in patients with lung adenocarcinoma.

A deep learning model combining multimodal radiomics, clinical and imaging features for differentiating ocular adnexal lymphoma from idiopathic orbital inflammation

OBJECTIVES

To evaluate the value of deep learning (DL) combining multimodal radiomics and clinical and imaging features for differentiating ocular adnexal lymphoma (OAL) from idiopathic orbital inflammation (IOI).

METHODS

Eighty-nine patients with histopathologically confirmed OAL (n = 39) and IOI (n = 50) were divided into training and validation groups. Convolutional neural networks and multimodal fusion layers were used to extract multimodal radiomics features from the T1-weighted image (T1WI), T2-weighted image, and contrast-enhanced T1WI. These multimodal radiomics features were then combined with clinical and imaging features and used together to differentiate between OAL and IOI. The area under the curve (AUC) was used to evaluate DL models with different features under five-fold cross-validation. The Student t-test, chi-squared, or Fisher exact test was used for comparison of different groups.

RESULTS

In the validation group, the diagnostic AUC of the DL model using combined features was 0.953 (95% CI, 0.895-1.000), higher than that of the DL model using multimodal radiomics features (0.843, 95% CI, 0.786-0.898, p < 0.01) or clinical and imaging features only (0.882, 95% CI, 0.782-0.982, p = 0.13). The DL model built on multimodal radiomics features outperformed those built on most bimodalities and unimodalities (p < 0.05). In addition, the DL-based analysis with the orbital cone area (covering both the orbital mass and surrounding tissues) was superior to that with the region of interest (ROI) covering only the mass area, although the difference was not significant (p = 0.33).

CONCLUSIONS

DL-based analysis that combines multimodal radiomics features with clinical and imaging features may help to differentiate between OAL and IOI.

KEY POINTS

• It is difficult to differentiate OAL from IOI due to the overlap in clinical and imaging manifestations. • Radiomics has shown potential for noninvasive diagnosis of different orbital lymphoproliferative disorders. • DL-based analysis combining radiomics and imaging and clinical features may help the differentiation between OAL and IOI.

Spatial interplay patterns of cancer nuclei and tumor-infiltrating lymphocytes (TILs) predict clinical benefit for immune checkpoint inhibitors

Immune checkpoint inhibitors (ICIs) show prominent clinical activity across multiple advanced tumors. However, less than half of patients respond even after molecule-based selection. Thus, improved biomarkers are required. In this study, we use an image analysis to capture morphologic attributes relating to the spatial interaction and architecture of tumor cells and tumor-infiltrating lymphocytes (TILs) from digitized H&E images. We evaluate the association of image features with progression-free (PFS) and overall survival in non-small cell lung cancer (NSCLC) (N = 187) and gynecological cancer (N = 39) patients treated with ICIs. We demonstrated that the classifier trained with NSCLC alone was associated with PFS in independent NSCLC cohorts and also in gynecological cancer. The classifier was also associated with clinical outcome independent of clinical factors. Moreover, the classifier was associated with PFS even with low PD-L1 expression. These findings suggest that image analysis can be used to predict clinical end points in patients receiving ICI.

Deep learning for locally advanced nasopharyngeal carcinoma prognostication based on pre- and post-treatment MRI

PURPOSE

We aimed to predict the prognosis of advanced nasopharyngeal carcinoma (stage Ⅲ-Ⅳa) using Pre- and Post-treatment MR images based on deep learning (DL).

METHODS

A total of 206 patients with primary nasopharyngeal carcinoma who were diagnosed and treated at the Renmin Hospital of Wuhan University between June 2012 and January 2018 were retrospectively selected. A rectangular region of interest (ROI), which included the tumor area, surrounding tissues and organs, was delineated on each Pre- and Post-treatment MR image. Two Inception-Resnet-V2 based transfer learning models, named Pre-model and Post-model, were trained with the Pre-treatment images and the Post-treatment images, respectively. In addition, an ensemble learning model based on the Pre-model and Post-models was established. The three established models were evaluated by receiver operating characteristic curve (ROC), confusion matrix, and Harrell's concordance indices (C-index). High-risk-related gradient-weighted class activation mapping (Grad-CAM) images were developed according to the DL models.

RESULTS

The Pre-model, Post-model, and ensemble model displayed a C-index of 0.717 (95% CI: 0.639 to 0.795), 0.811 (95% CI: 0.745-0.877), 0.830 (95% CI: 0.767-0.893), and AUC of 0.741 (95% CI: 0.584-0.900), 0.806 (95% CI: 0.670-0.942), and 0.842 (95% CI: 0.718-0.967) for the test cohort, respectively. In comparison with the models, the performance of Post-model was better than the performance of Pre-model, which indicated the importance of Post-treatment images for prognosis prediction. All three DL models performed better than the TNM staging system (0.723, 95% CI: 0.567-0.879). The captured features presented on Grad-CAM images suggested that the areas around the tumor and lymph nodes were related to the prognosis of the tumor.

CONCLUSIONS

The three established DL models based on Pre- and Post-treatment MR images have a better performance than TNM staging. Post-treatment MR images are of great significance for prognosis prediction and could contribute to clinical decision-making.

A narrative review of deep learning applications in lung cancer research: from screening to prognostication

Background and Objective

Deep learning (DL) algorithms have been developed for various tasks, including lung nodule detection on chest radiographs or lung cancer computed tomography screening, potential candidate selection in lung cancer screening, malignancy prediction for indeterminate pulmonary nodules, lung cancer staging, treatment response prediction, prognostication, and prediction of genetic mutations in lung cancer. Furthermore, these DL algorithms have been applied in various clinical settings in order for them to be generalized in real-world clinical practice. Multiple DL algorithms have been corroborated to be on par with experts or current clinical prediction models for several specific tasks. However, no article has yet comprehensively reviewed DL algorithms dedicated to lung cancer research. This narrative review presents an overview of the literature dealing with DL techniques applied in lung cancer research and briefly summarizes the results according to the DL algorithms’ clinical use cases.

Methods

we performed a narrative review by searching the Embase and OVID-MEDLINE databases for articles published in English from October, 2016 until September, 2021 and reviewing the bibliographies of key references to identify important literature related to DL in lung cancer research. The background, development, results, and clinical implications of each DL algorithm are briefly discussed. Lastly, we end this review article by highlighting future directions in lung cancer research using DL techniques.

Key Content and Findings

DL algorithms have been introduced to show comparable or higher performance than human experts in various clinical settings. Specifically, they have been actively applied to detect lung nodules in chest radiographs or computed tomography (CT) examinations, optimize candidate selection for lung cancer screening (LCS), predict the malignancy of lung nodules, stage lung cancer, and predict treatment response, patients’ prognoses, and genetic mutations in lung cancers.

Conclusions

DL algorithms have corroborated their potential value for various tasks, ranging from lung cancer screening to prognostication of lung cancer patients. Future research is warranted for the clinical application of these algorithms in daily clinical practice and verification of their real-world clinical usefulness.

Diagnosis of endometrium hyperplasia and screening of endometrial intraepithelial neoplasia in histopathological images using a global-to-local multi-scale convolutional neural network

BACKGROUND AND OBJECTIVE

Endometrial hyperplasia (EH), a uterine pathology characterized by an increased gland-to-stroma ratio compared to normal endometrium (NE), may precede the development of endometrial cancer (EC). Particularly, atypical EH also known as endometrial intraepithelial neoplasia (EIN), has been proven to be a precursor of EC. Thus, diagnosing different EH (EIN, hyperplasia without atypia (HwA) and NE) and screening EIN from non-EIN are crucial for the health of female reproductive system. Computer-aided-diagnosis (CAD) was used to diagnose endometrial histological images based on machine learning and deep learning. However, these studies perform single-scale image analysis and thus can only characterize partial endometrial features. Empirically, both global (cytological changes relative to background) and local features (gland-to-stromal ratio and lesion dimension) are helpful in identifying endometrial lesions.

METHODS

We proposed a global-to-local multi-scale convolutional neural network (G2LNet) to diagnose different EH and to screen EIN in endometrial histological images stained by hematoxylin and eosin (H&E). The G2LNet first used a supervised model in the global part to extract contextual features of endometrial lesions, and simultaneously deployed multi-instance learning in the local part to obtain textural features from multiple image patches. The contextual and textural features were used together to diagnose different endometrial lesions after fusion by a convolutional block attention module. In addition, we visualized the salient regions on both the global image and local images to investigate the interpretability of the model in endometrial diagnosis.

RESULTS

In the five-fold cross validation on 7812 H&E images from 467 endometrial specimens, G2LNet achieved an accuracy of 97.01% for EH diagnosis and an area-under-the-curve (AUC) of 0.9902 for EIN screening, significantly higher than state-of-the-arts. In external validation on 1631 H&E images from 135 specimens, G2LNet achieved an accuracy of 95.34% for EH diagnosis, which was comparable to that of a mid-level pathologist (95.71%). Specifically, G2LNet had advantages in diagnosing EIN, while humans performed better in identifying NE and HwA.

CONCLUSIONS

The developed G2LNet that integrated both the global (contextual) and local (textural) features may help pathologists diagnose endometrial lesions in clinical practices, especially to improve the accuracy and efficiency of screening for precancerous lesions.

Role of radiomics in predicting immunotherapy response

Immunotherapies have revolutionised cancer management. Despite their success, durable responses are limited to a subset of patients. Prediction of immunotherapy response in patients has proven to be difficult due to a lack of robust biomarkers. Routinely collected imaging may offer an additional information source to personalise patient treatment, with advantages over tissue-based biomarkers. Quantitative image analysis or radiomics, which involves the high-throughput extraction of imaging features, has the potential to non-invasively predict cancer histology, outcomes and prognosis. This review evaluates the value of radiomics in patients undergoing immunotherapy, with a summary provided of the performance of radiomics models in predicting immunotherapy response and toxicity, as well as immune correlates. Much of the literature focussed on clinical endpoints and correlates to tissue biomarkers, particularly in lung cancer, while few studies investigated association with immune-related adverse events. Strengths of the studies included more frequent use of clinical trial datasets, homogenous patient cohorts and high-quality diagnostic scans. Limitations of the studies include heterogeneity in study methodology, lack of well-defined homogenous imaging datasets, limited open publishing of imaging datasets, coding and parameters used for radiomics signature development and limited use of external validation datasets. Future research should address the above limitations, as well as further explore the relationship between radiomics and immune-related adverse effects and less well-studied biological correlates such tumour mutational burden, and incorporate known clinical prognostic scores into radiomics models.

Decoding the dopamine transporter imaging for the differential diagnosis of parkinsonism using deep learning

Purpose

This work attempts to decode the discriminative information in dopamine transporter (DAT) imaging using deep learning for the differential diagnosis of parkinsonism.

Methods

This study involved 1017 subjects who underwent DAT PET imaging ([¹¹C]CFT) including 43 healthy subjects and 974 parkinsonian patients with idiopathic Parkinson’s disease (IPD), multiple system atrophy (MSA) or progressive supranuclear palsy (PSP). We developed a 3D deep convolutional neural network to learn distinguishable DAT features for the differential diagnosis of parkinsonism. A full-gradient saliency map approach was employed to investigate the functional basis related to the decision mechanism of the network. Furthermore, deep-learning-guided radiomics features and quantitative analysis were compared with their conventional counterparts to further interpret the performance of deep learning.

Results

The proposed network achieved area under the curve of 0.953 (sensitivity 87.7%, specificity 93.2%), 0.948 (sensitivity 93.7%, specificity 97.5%), and 0.900 (sensitivity 81.5%, specificity 93.7%) in the cross-validation, together with sensitivity of 90.7%, 84.1%, 78.6% and specificity of 88.4%, 97.5% 93.3% in the blind test for the differential diagnosis of IPD, MSA and PSP, respectively. The saliency map demonstrated the most contributed areas determining the diagnosis located at parkinsonism-related regions, e.g., putamen, caudate and midbrain. The deep-learning-guided binding ratios showed significant differences among IPD, MSA and PSP groups (P < 0.001), while the conventional putamen and caudate binding ratios had no significant difference between IPD and MSA (P = 0.24 and P = 0.30). Furthermore, compared to conventional radiomics features, there existed average above 78.1% more deep-learning-guided radiomics features that had significant differences among IPD, MSA and PSP.

Conclusion

This study suggested the developed deep neural network can decode in-depth information from DAT and showed potential to assist the differential diagnosis of parkinsonism. The functional regions supporting the diagnosis decision were generally consistent with known parkinsonian pathology but provided more specific guidance for feature selection and quantitative analysis.

Supplementary Information

The online version contains supplementary material available at 10.1007/s00259-022-05804-x.