Comparison and Fusion of Deep Learning and Radiomics Features of Ground-Glass Nodules to Predict the Invasiveness Risk of Stage-I Lung Adenocarcinomas in CT Scan

Quantitative analysis of lung lesions using unenhanced chest computed tomography images

INTRODUCTION

Chest radiograph and computed tomography (CT) scans can accidentally reveal pulmonary nodules. Malignant and benign pulmonary nodules can be difficult to distinguish without specific imaging features, such as calcification, necrosis, and contrast enhancement. However, these lesions may exhibit different image texture characteristics which cannot be assessed visually. Thus, a computer-assisted quantitative method like histogram analysis (HA) of Hounsfield unit (HU) values can improve diagnostic accuracy, reducing the need for invasive biopsy.

METHODS

In this exploratory control study, nonenhanced chest CT images of 20 patients with benign (10) and cancerous (10) lesion were selected retrospectively. The appearances of benign and malignant lesions were very similar in chest CT images, and only pathology report was used to discriminate them. Free hand region of interest (ROI) was inserted inside the lesion for all slices of each lesion. Mean, minimum, maximum, and standard deviations of HU values were recorded and used to make HA.

RESULTS

HA showed that the most malignant lesions have a mean HU value between 30 and 50, a maximum HU less than 150, and a minimum HU between -30 and 20. Lesions outside these ranges were mostly benign.

CONCLUSION

Quantitative CT analysis may differentiate malignant from benign lesions without specific malignancy patterns on unenhanced chest CT image.

Deep learning or radiomics based on CT for predicting the response of gastric cancer to neoadjuvant chemotherapy: a meta-analysis and systematic review

Background

Artificial intelligence (AI) models, clinical models (CM), and the integrated model (IM) are utilized to evaluate the response to neoadjuvant chemotherapy (NACT) in patients diagnosed with gastric cancer.

Objective

The objective is to identify the diagnostic test of the AI model and to compare the accuracy of AI, CM, and IM through a comprehensive summary of head-to-head comparative studies.

Methods

PubMed, Web of Science, Cochrane Library, and Embase were systematically searched until September 5, 2023, to compile English language studies without regional restrictions. The quality of the included studies was evaluated using the Quality Assessment of Diagnostic Accuracy Studies-2 (QUADAS-2) criteria. Forest plots were utilized to illustrate the findings of diagnostic accuracy, while Hierarchical Summary Receiver Operating Characteristic curves were generated to estimate sensitivity (SEN) and specificity (SPE). Meta-regression was applied to analyze heterogeneity across the studies. To assess the presence of publication bias, Deeks' funnel plot and an asymmetry test were employed.

Results

A total of 9 studies, comprising 3313 patients, were included for the AI model, with 7 head-to-head comparative studies involving 2699 patients. Across the 9 studies, the pooled SEN for the AI model was 0.75 (95% confidence interval (CI): 0.66, 0.82), and SPE was 0.77 (95% CI: 0.69, 0.84). Meta-regression was conducted, revealing that the cut-off value, approach to predicting response, and gold standard might be sources of heterogeneity. In the head-to-head comparative studies, the pooled SEN for AI was 0.77 (95% CI: 0.69, 0.84) with SPE at 0.79 (95% CI: 0.70, 0.85). For CM, the pooled SEN was 0.67 (95% CI: 0.57, 0.77) with SPE at 0.59 (95% CI: 0.54, 0.64), while for IM, the pooled SEN was 0.83 (95% CI: 0.79, 0.86) with SPE at 0.69 (95% CI: 0.56, 0.79). Notably, there was no statistical difference, except that IM exhibited higher SEN than AI, while maintaining a similar level of SPE in pairwise comparisons. In the Receiver Operating Characteristic analysis subgroup, the CT-based Deep Learning (DL) subgroup, and the National Comprehensive Cancer Network (NCCN) guideline subgroup, the AI model exhibited higher SEN but lower SPE compared to the IM. Conversely, in the training cohort subgroup and the internal validation cohort subgroup, the AI model demonstrated lower SEN but higher SPE than the IM. The subgroup analysis underscored that factors such as the number of cohorts, cohort type, cut-off value, approach to predicting response, and choice of gold standard could impact the reliability and robustness of the results.

Conclusion

AI has demonstrated its viability as a tool for predicting the response of GC patients to NACT Furthermore, CT-based DL model in AI was sensitive to extract tumor features and predict the response. The results of subgroup analysis also supported the above conclusions. Large-scale rigorously designed diagnostic accuracy studies and head-to-head comparative studies are anticipated.

Systematic review registration

PROSPERO, CRD42022377030.

AI/ML advances in non-small cell lung cancer biomarker discovery

Lung cancer is the leading cause of cancer deaths among both men and women, representing approximately 25% of cancer fatalities each year. The treatment landscape for non-small cell lung cancer (NSCLC) is rapidly evolving due to the progress made in biomarker-driven targeted therapies. While advancements in targeted treatments have improved survival rates for NSCLC patients with actionable biomarkers, long-term survival remains low, with an overall 5-year relative survival rate below 20%. Artificial intelligence/machine learning (AI/ML) algorithms have shown promise in biomarker discovery, yet NSCLC-specific studies capturing the clinical challenges targeted and emerging patterns identified using AI/ML approaches are lacking. Here, we employed a text-mining approach and identified 215 studies that reported potential biomarkers of NSCLC using AI/ML algorithms. We catalogued these studies with respect to BEST (Biomarkers, EndpointS, and other Tools) biomarker sub-types and summarized emerging patterns and trends in AI/ML-driven NSCLC biomarker discovery. We anticipate that our comprehensive review will contribute to the current understanding of AI/ML advances in NSCLC biomarker research and provide an important catalogue that may facilitate clinical adoption of AI/ML-derived biomarkers.

A graph neural network model for the diagnosis of lung adenocarcinoma based on multimodal features and an edge-generation network

Background

Lung cancer is a global disease with high lethality, with early screening being considerably helpful for improving the 5-year survival rate. Multimodality features in early screening imaging are an important part of the prediction for lung adenocarcinoma, and establishing a model for adenocarcinoma diagnosis based on multimodal features is an obvious clinical need. Through our practice and investigation, we found that graph neural networks (GNNs) are excellent platforms for multimodal feature fusion, and the data can be completed using the edge-generation network. Therefore, we propose a new lung adenocarcinoma multiclassification model based on multimodal features and an edge-generation network.

Methods

According to a ratio of 80% to 20%, respectively, the dataset of 338 cases was divided into the training set and the test set through 5-fold cross-validation, and the distribution of the 2 sets was the same. First, the regions of interest (ROIs) cropped from computed tomography (CT) images were separately fed into convolutional neural networks (CNNs) and radiomics processing platforms. The results of the 2 parts were then input into a graph embedding representation network to obtain the fused feature vectors. Subsequently, a graph database based on the clinical and semantic features was established, and the data were supplemented by an edge-generation network, with the fused feature vectors being used as the input of the nodes. This enabled us to clearly understand where the information transmission of the GNN takes place and improves the interpretability of the model. Finally, the nodes were classified using GNNs.

Results

On our dataset, the proposed method presented in this paper achieved superior results compared to traditional methods and showed some comparability with state-of-the-art methods for lung nodule classification. The results of our method are as follows: accuracy (ACC) =66.26% (±4.46%), area under the curve (AUC) =75.86% (±1.79%), F1-score =64.00% (±3.65%), and Matthews correlation coefficient (MCC) =48.40% (±5.07%). The model with the edge-generating network consistently outperformed the model without it in all aspects.

Conclusions

The experiments demonstrate that with appropriate data=construction methods GNNs can outperform traditional image processing methods in the field of CT-based medical image classification. Additionally, our model has higher interpretability, as it employs subjective clinical and semantic features as the data construction approach. This will help doctors better leverage human-computer interactions.

An ensemble deep learning model for risk stratification of invasive lung adenocarcinoma using thin-slice CT

Lung cancer screening using computed tomography (CT) has increased the detection rate of small pulmonary nodules and early-stage lung adenocarcinoma. It would be clinically meaningful to accurate assessment of the nodule histology by CT scans with advanced deep learning algorithms. However, recent studies mainly focus on predicting benign and malignant nodules, lacking of model for the risk stratification of invasive adenocarcinoma. We propose an ensemble multi-view 3D convolutional neural network (EMV-3D-CNN) model to study the risk stratification of lung adenocarcinoma. We include 1075 lung nodules (≤30 mm and ≥4 mm) with preoperative thin-section CT scans and definite pathology confirmed by surgery. Our model achieves a state-of-art performance of 91.3% and 92.9% AUC for diagnosis of benign/malignant and pre-invasive/invasive nodules, respectively. Importantly, our model outperforms senior doctors in risk stratification of invasive adenocarcinoma with 77.6% accuracy [i.e., Grades 1, 2, 3]). It provides detailed predictive histological information for the surgical management of pulmonary nodules. Finally, for user-friendly access, the proposed model is implemented as a web-based system ( https://seeyourlung.com.cn ).

Comparison and fusion prediction model for lung adenocarcinoma with micropapillary and solid pattern using clinicoradiographic, radiomics and deep learning features

To investigate whether the combination scheme of deep learning score (DL-score) and radiomics can improve preoperative diagnosis in the presence of micropapillary/solid (MPP/SOL) patterns in lung adenocarcinoma (ADC). A retrospective cohort of 514 confirmed pathologically lung ADC in 512 patients after surgery was enrolled. The clinicoradiographic model (model 1) and radiomics model (model 2) were developed with logistic regression. The deep learning model (model 3) was constructed based on the deep learning score (DL-score). The combine model (model 4) was based on DL-score and R-score and clinicoradiographic variables. The performance of these models was evaluated with area under the receiver operating characteristic curve (AUC) and compared using DeLong's test internally and externally. The prediction nomogram was plotted, and clinical utility depicted with decision curve. The performance of model 1, model 2, model 3 and model 4 was supported by AUCs of 0.848, 0.896, 0.906, 0.921 in the Internal validation set, that of 0.700, 0.801, 0.730, 0.827 in external validation set, respectively. These models existed statistical significance in internal validation (model 4 vs model 3, P = 0.016; model 4 vs model 1, P = 0.009, respectively) and external validation (model 4 vs model 2, P = 0.036; model 4 vs model 3, P = 0.047; model 4 vs model 1, P = 0.016, respectively). The decision curve analysis (DCA) demonstrated that model 4 predicting the lung ADC with MPP/SOL structure would be more beneficial than the model 1and model 3 but comparable with the model 2. The combined model can improve preoperative diagnosis in the presence of MPP/SOL pattern in lung ADC in clinical practice.

Machine learning combined with radiomics and deep learning features extracted from CT images: a novel AI model to distinguish benign from malignant ovarian tumors

BACKGROUND

To develop an artificial intelligence (AI) model with radiomics and deep learning (DL) features extracted from CT images to distinguish benign from malignant ovarian tumors.

METHODS

We enrolled 149 patients with pathologically confirmed ovarian tumors. A total of 185 tumors were included and divided into training and testing sets in a 7:3 ratio. All tumors were manually segmented from preoperative contrast-enhanced CT images. CT image features were extracted using radiomics and DL. Five models with different combinations of feature sets were built. Benign and malignant tumors were classified using machine learning (ML) classifiers. The model performance was compared with five radiologists on the testing set.

RESULTS

 Among the five models, the best performing model is the ensemble model with a combination of radiomics, DL, and clinical feature sets. The model achieved an accuracy of 82%, specificity of 89% and sensitivity of 68%. Compared with junior radiologists averaged results, the model had a higher accuracy (82% vs 66%) and specificity (89% vs 65%) with comparable sensitivity (68% vs 67%). With the assistance of the model, the junior radiologists achieved a higher average accuracy (81% vs 66%), specificity (80% vs 65%), and sensitivity (82% vs 67%), approaching to the performance of senior radiologists.

CONCLUSIONS

 We developed a CT-based AI model that can differentiate benign and malignant ovarian tumors with high accuracy and specificity. This model significantly improved the performance of less-experienced radiologists in ovarian tumor assessment, and may potentially guide gynecologists to provide better therapeutic strategies for these patients.

Clinical radiomics-based machine learning versus three-dimension convolutional neural network analysis for differentiation of thymic epithelial tumors from other prevascular mediastinal tumors on chest computed tomography scan

Purpose

To compare the diagnostic performance of radiomic analysis with machine learning (ML) model with a convolutional neural network (CNN) in differentiating thymic epithelial tumors (TETs) from other prevascular mediastinal tumors (PMTs).

Methods

A retrospective study was performed in patients with PMTs and undergoing surgical resection or biopsy in National Cheng Kung University Hospital, Tainan, Taiwan, E-Da Hospital, Kaohsiung, Taiwan, and Kaohsiung Veterans General Hospital, Kaohsiung, Taiwan between January 2010 and December 2019. Clinical data including age, sex, myasthenia gravis (MG) symptoms and pathologic diagnosis were collected. The datasets were divided into UECT (unenhanced computed tomography) and CECT (enhanced computed tomography) for analysis and modelling. Radiomics model and 3D CNN model were used to differentiate TETs from non-TET PMTs (including cyst, malignant germ cell tumor, lymphoma and teratoma). The macro F1-score and receiver operating characteristic (ROC) analysis were performed to evaluate the prediction models.

Result

In the UECT dataset, there were 297 patients with TETs and 79 patients with other PMTs. The performance of radiomic analysis with machine learning model using LightGBM with Extra Tree (macro F1-Score = 83.95%, ROC-AUC = 0.9117) had better performance than the 3D CNN model (macro F1-score = 75.54%, ROC-AUC = 0.9015). In the CECT dataset, there were 296 patients with TETs and 77 patients with other PMTs. The performance of radiomic analysis with machine learning model using LightGBM with Extra Tree (macro F1-Score = 85.65%, ROC-AUC = 0.9464) had better performance than the 3D CNN model (macro F1-score = 81.01%, ROC-AUC = 0.9275).

Conclusion

Our study revealed that the individualized prediction model integrating clinical information and radiomic features using machine learning demonstrated better predictive performance in the differentiation of TETs from other PMTs at chest CT scan than 3D CNN model.

Feature selection methods and predictive models in CT lung cancer radiomics

Radiomics is a technique that extracts quantitative features from medical images using data-characterization algorithms. Radiomic features can be used to identify tissue characteristics and radiologic phenotyping that is not observable by clinicians. A typical workflow for a radiomics study includes cohort selection, radiomic feature extraction, feature and predictive model selection, and model training and validation. While there has been increasing attention given to radiomic feature extraction, standardization, and reproducibility, currently, there is a lack of rigorous evaluation of feature selection methods and predictive models. Herein, we review the published radiomics investigations in CT lung cancer and provide an overview of the commonly used radiomic feature selection methods and predictive models. We also compare limitations of various methods in clinical applications and present sources of uncertainty associated with those methods. This review is expected to help raise awareness of the impact of radiomic feature and model selection methods on the integrity of radiomics studies.

A retrospective diagnostic test study on circulating tumor cells and artificial intelligence imaging in patients with lung adenocarcinoma

Background

Either tumor volume or folate-receptor-positive circulating tumor cells (FR⁺CTC) has been proven effective in predicting tumor cell invasion. However, it has yet to be documented to use FR⁺CTC along with artificial intelligence (AI) tumor volume to differentiate between pathological subtypes of lung adenocarcinoma (LUAD). Therefore, this study is aimed to evaluate the accuracy of FR⁺CTC and AI tumor volume for classifying the invasiveness of LUAD.

Methods

A total of 226 patients who were diagnosed with LUAD were enrolled. The inclusion criteria were: (I) FR⁺CTC detection and AI imaging before anticancer therapy, and (II) definite histopathologic diagnosis, which is the gold diagnosis of LUAD and its subtypes. Use the CytoploRare^® Detection Kit to quantify FR⁺CTC and the AI-assisted diagnosis system, ScrynPro, to measure tumor volume. The clinical data were used to construct univariate and multivariate logistic regression models. A nomogram was drawn based on the multivariate logistic regression model. The validity is evaluated by the calibration curve and Hosmer-Lemeshow goodness-of-fit test.

Results

The mean age of 146 patients (96 males, 49 females and 1 gender missing) retrospectively enrolled was 56.6. In the cohort, 41 and 105 patients were assigned to adenocarcinoma in situ (AIS) + minimally invasive adenocarcinoma (MIA) and invasive pulmonary adenocarcinoma (IPA), respectively. There was no significant difference between the sex distribution and smoking history of the two groups (P=0.155 and P=0.442, respectively). In univariate analysis, the nodules type, maximum density, tumor volume and FR⁺CTC level were statistically significant with the invasiveness of LUAD (P<0.05). The multivariate analysis showed significant differences in FR⁺CTC and AI tumor volume (P<0.001). The area under the curves (AUCs) of FR⁺CTC and AI tumor volume in diagnosing tumor invasiveness were 0.659 and 0.698, respectively. A predictive model combining FR⁺CTC with AI tumor volume showed a sensitivity of 86.89% and a specificity of 70.94%, and the AUC was 0.841. The nomogram had good agreement with actual observation, and the Hosmer-Lemeshow test yielded non-significant goodness-of-fit.

Conclusions

FR⁺CTC and/or AI tumor volume are independent indicators of the invasiveness of LUAD, and the nomogram based on them can be used for the preoperative screening of patients.

3D deep learning versus the current methods for predicting tumor invasiveness of lung adenocarcinoma based on high-resolution computed tomography images

Background

Different pathological subtypes of lung adenocarcinoma lead to different treatment decisions and prognoses, and it is clinically important to distinguish invasive lung adenocarcinoma from preinvasive adenocarcinoma (adenocarcinoma in situ and minimally invasive adenocarcinoma). This study aims to investigate the performance of the deep learning approach based on high-resolution computed tomography (HRCT) images in the classification of tumor invasiveness and compare it with the performances of currently available approaches.

Methods

In this study, we used a deep learning approach based on 3D conventional networks to automatically predict the invasiveness of pulmonary nodules. A total of 901 early-stage non-small cell lung cancer patients who underwent surgical treatment at Shanghai Chest Hospital between November 2015 and March 2017 were retrospectively included and randomly assigned to a training set (n=814) or testing set 1 (n=87). We subsequently included 116 patients who underwent surgical treatment and intraoperative frozen section between April 2019 and January 2020 to form testing set 2. We compared the performance of our deep learning approach in predicting tumor invasiveness with that of intraoperative frozen section analysis and human experts (radiologists and surgeons).

Results

The deep learning approach yielded an area under the receiver operating characteristic curve (AUC) of 0.946 for distinguishing preinvasive adenocarcinoma from invasive lung adenocarcinoma in the testing set 1, which is significantly higher than the AUCs of human experts (P<0.05). In testing set 2, the deep learning approach distinguished invasive adenocarcinoma from preinvasive adenocarcinoma with an AUC of 0.862, which is higher than that of frozen section analysis (0.755, P=0.043), senior thoracic surgeons (0.720, P=0.006), radiologists (0.766, P>0.05) and junior thoracic surgeons (0.768, P>0.05).

Conclusions

We developed a deep learning model that achieved comparable performance to intraoperative frozen section analysis in determining tumor invasiveness. The proposed method may contribute to clinical decisions related to the extent of surgical resection.

Fusion of CT images and clinical variables based on deep learning for predicting invasiveness risk of stage I lung adenocarcinoma

PURPOSE

To develop a novel multimodal data fusion model by incorporating computed tomography (CT) images and clinical variables based on deep learning for predicting the invasiveness risk of stage I lung adenocarcinoma that manifests as ground-glass nodules (GGNs), and compare the diagnostic performance of it with that of radiologists.

METHODS

A total of 1946 patients with solitary and histopathologically confirmed GGNs with maximum diameter less than 3 cm were retrospectively enrolled. The training dataset containing 1704 GGNs was augmented by resampling, scaling, random cropping, etc., to generate new training data. A multimodal data fusion model based on residual learning architecture and two multilayer perceptron with attention mechanism combining CT images with patient general data and serum tumor markers was built. The distance-based confidence scores (DCS) were calculated and compared among multimodal data models with different combinations. An observer study was conducted and the prediction performance of the fusion algorithms was compared with that of the two radiologists by an independent testing dataset with 242 GGNs.

RESULTS

Among the whole GGNs, 606 GGNs are confirmed as invasive adenocarcinoma (IA) and 1340 are non-IA. The proposed novel multimodal data fusion model combining CT images, patient general data and serum tumor markers achieved the highest accuracy (88.5%), Area under a ROC curve (AUC) (0.957), F1 (81.5%), F1_weighted (81.9%) and Matthews correlation coefficient (MCC) (73.2%) for classifying between IA and non-IA GGNs, which was even better than the senior radiologist's performance (accuracy, 86.1%). In addition, the DCSs for multimodal data suggested that CT image had a stronger influence (0.9540) quantitatively than general data (0.6726) or tumor marker (0.6971).

CONCLUSION

This study demonstrated that the feasibility of integrating different types of data including CT images and clinical variables, and the multimodal data fusion model yielded higher performance for distinguishing IA from non-IA GGNs. This article is protected by copyright. All rights reserved.

Clinical Variables, Deep Learning and Radiomics Features Help Predict the Prognosis of Adult Anti-N-methyl-D-aspartate Receptor Encephalitis Early: A Two-Center Study in Southwest China

Objective

To develop a fusion model combining clinical variables, deep learning (DL), and radiomics features to predict the functional outcomes early in patients with adult anti-N-methyl-D-aspartate receptor (NMDAR) encephalitis in Southwest China.

Methods

From January 2012, a two-center study of anti-NMDAR encephalitis was initiated to collect clinical and MRI data from acute patients in Southwest China. Two experienced neurologists independently assessed the patients’ prognosis at 24 moths based on the modified Rankin Scale (mRS) (good outcome defined as mRS 0–2; bad outcome defined as mRS 3-6). Risk factors influencing the prognosis of patients with acute anti-NMDAR encephalitis were investigated using clinical data. Five DL and radiomics models trained with four single or combined four MRI sequences (T1-weighted imaging, T2-weighted imaging, fluid-attenuated inversion recovery imaging and diffusion weighted imaging) and a clinical model were developed to predict the prognosis of anti-NMDAR encephalitis. A fusion model combing a clinical model and two machine learning-based models was built. The performances of the fusion model, clinical model, DL-based models and radiomics-based models were compared using the area under the receiver operating characteristic curve (AUC) and accuracy and then assessed by paired t-tests (P < 0.05 was considered significant).

Results

The fusion model achieved the significantly greatest predictive performance in the internal test dataset with an AUC of 0.963 [95% CI: (0.874-0.999)], and also significantly exhibited an equally good performance in the external validation dataset, with an AUC of 0.927 [95% CI: (0.688-0.975)]. The radiomics_combined model (AUC: 0.889; accuracy: 0.857) provided significantly superior predictive performance than the DL_combined (AUC: 0.845; accuracy: 0.857) and clinical models (AUC: 0.840; accuracy: 0.905), whereas the clinical model showed significantly higher accuracy. Compared with all single-sequence models, the DL_combined model and the radiomics_combined model had significantly greater AUCs and accuracies.

Conclusions

The fusion model combining clinical variables and machine learning-based models may have early predictive value for poor outcomes associated with anti-NMDAR encephalitis.

Towards a safe and efficient clinical implementation of machine learning in radiation oncology by exploring model interpretability, explainability and data-model dependency

The interest in machine learning (ML) has grown tremendously in recent years, partly due to the performance leap that occurred with new techniques of deep learning, convolutional neural networks for images, increased computational power, and wider availability of large datasets. Most fields of medicine follow that popular trend and, notably, radiation oncology is one of those that are at the forefront, with already a long tradition in using digital images and fully computerized workflows. ML models are driven by data, and in contrast with many statistical or physical models, they can be very large and complex, with countless generic parameters. This inevitably raises two questions, namely, the tight dependence between the models and the datasets that feed them, and the interpretability of the models, which scales with its complexity. Any problems in the data used to train the model will be later reflected in their performance. This, together with the low interpretability of ML models, makes their implementation into the clinical workflow particularly difficult. Building tools for risk assessment and quality assurance of ML models must involve then two main points: interpretability and data-model dependency. After a joint introduction of both radiation oncology and ML, this paper reviews the main risks and current solutions when applying the latter to workflows in the former. Risks associated with data and models, as well as their interaction, are detailed. Next, the core concepts of interpretability, explainability, and data-model dependency are formally defined and illustrated with examples. Afterwards, a broad discussion goes through key applications of ML in workflows of radiation oncology as well as vendors' perspectives for the clinical implementation of ML.

Special issue "The advance of solid tumor research in China": Prognosis prediction for stage II colorectal cancer by fusing CT radiomics and deep-learning features of primary lesions and peripheral lymph nodes

Currently, the prognosis assessment of stage II colorectal cancer (CRC) remains a difficult clinical problem; therefore, more accurate prognostic predictors must be developed. In this study, we developed a prognostic prediction model for stage II CRC by fusing radiomics and deep-learning (DL) features of primary lesions and peripheral lymph nodes (LNs) in computed tomography (CT) scans. First, two CT radiomics models were built using primary lesion and LN image features. Subsequently, an information fusion method was used to build a fusion radiomics model by combining the tumor and LN image features. Furthermore, a transfer learning method was applied to build a deep convolutional neural network (CNN) model. Finally, the prediction scores generated by the radiomics and CNN models were fused to improve the prognosis prediction performance. The disease-free survival (DFS) and overall survival (OS) prediction areas under the curves (AUCs) generated by the fusion model improved to 0.76±0.08 and 0.91±0.05, respectively. These were significantly higher than the AUCs generated by the models using the individual CT radiomics and deep image features. Applying the survival analysis method, the DFS and OS fusion models yielded concordance index (C-index) values of 0.73 and 0.9, respectively. Hence, the combined model exhibited good predictive efficacy; therefore, it could be used for the accurate assessment of the prognosis of stage II CRC patients. Moreover, it could be used to screen out high-risk patients with poor prognoses, and assist in the formulation of clinical treatment decisions in a timely manner to achieve precision medicine. This article is protected by copyright. All rights reserved.

Application of Artificial Intelligence in Discovery and Development of Anticancer and Antidiabetic Therapeutic Agents

Spectacular developments in molecular and cellular biology have led to important discoveries in cancer research. Despite cancer is one of the major causes of morbidity and mortality globally, diabetes is one of the most leading sources of group of disorders. Artificial intelligence (AI) has been considered the fourth industrial revolution machine. The most major hurdles in drug discovery and development are the time and expenditures required to sustain the drug research pipeline. Large amounts of data can be explored and generated by AI, which can then be converted into useful knowledge. Because of this, the world's largest drug companies have already begun to use AI in their drug development research. In the present era, AI has a huge amount of potential for the rapid discovery and development of new anticancer drugs. Clinical studies, electronic medical records, high-resolution medical imaging, and genomic assessments are just a few of the tools that could aid drug development. Large data sets are available to researchers in the pharmaceutical and medical fields, which can be analyzed by advanced AI systems. This review looked at how computational biology and AI technologies may be utilized in cancer precision drug development by combining knowledge of cancer medicines, drug resistance, and structural biology. This review also highlighted a realistic assessment of the potential for AI in understanding and managing diabetes.

How AI Can Help in the Diagnostic Dilemma of Pulmonary Nodules

Simple Summary

Pulmonary nodules are considered a sign of bronchogenic carcinoma, detecting them early will reduce their progression and can save lives. Lung cancer is the second most common type of cancer in both men and women. This manuscript discusses the current applications of artificial intelligence (AI) in lung segmentation as well as pulmonary nodule segmentation and classification using computed tomography (CT) scans, published in the last two decades, in addition to the limitations and future prospects in the field of AI.

Abstract

Pulmonary nodules are the precursors of bronchogenic carcinoma, its early detection facilitates early treatment which save a lot of lives. Unfortunately, pulmonary nodule detection and classification are liable to subjective variations with high rate of missing small cancerous lesions which opens the way for implementation of artificial intelligence (AI) and computer aided diagnosis (CAD) systems. The field of deep learning and neural networks is expanding every day with new models designed to overcome diagnostic problems and provide more applicable and simply used models. We aim in this review to briefly discuss the current applications of AI in lung segmentation, pulmonary nodule detection and classification.

Application of Artificial Intelligence in Diagnosis of Craniopharyngioma

Craniopharyngioma is a congenital brain tumor with clinical characteristics of hypothalamic-pituitary dysfunction, increased intracranial pressure, and visual field disorder, among other injuries. Its clinical diagnosis mainly depends on radiological examinations (such as Computed Tomography, Magnetic Resonance Imaging). However, assessing numerous radiological images manually is a challenging task, and the experience of doctors has a great influence on the diagnosis result. The development of artificial intelligence has brought about a great transformation in the clinical diagnosis of craniopharyngioma. This study reviewed the application of artificial intelligence technology in the clinical diagnosis of craniopharyngioma from the aspects of differential classification, prediction of tissue invasion and gene mutation, prognosis prediction, and so on. Based on the reviews, the technical route of intelligent diagnosis based on the traditional machine learning model and deep learning model were further proposed. Additionally, in terms of the limitations and possibilities of the development of artificial intelligence in craniopharyngioma diagnosis, this study discussed the attentions required in future research, including few-shot learning, imbalanced data set, semi-supervised models, and multi-omics fusion.

Development of a deep learning-based method to diagnose pulmonary ground-glass nodules by sequential computed tomography imaging

Background

Early identification of the malignant propensity of pulmonary ground‐glass nodules (GGNs) can relieve the pressure from tracking lesions and personalized treatment adaptation. The purpose of this study was to develop a deep learning‐based method using sequential computed tomography (CT) imaging for diagnosing pulmonary GGNs.

Methods

This diagnostic study retrospectively enrolled 762 patients with GGNs from West China Hospital of Sichuan University between July 2009 and March 2019. All patients underwent surgical resection and at least two consecutive time‐point CT scans. We developed a deep learning‐based method to identify GGNs using sequential CT imaging on a training set consisting of 1524 CT sections from 508 patients and then evaluated 256 patients in the testing set. Afterwards, an observer study was conducted to compare the diagnostic performance between the deep learning model and two trained radiologists in the testing set. We further performed stratified analysis to further relieve the impact of histological types, nodule size, time interval between two CTs, and the component of GGNs. Receiver operating characteristic (ROC) analysis was used to assess the performance of all models.

Results

The deep learning model that used integrated DL‐features from initial and follow‐up CT images yielded the best diagnostic performance, with an area under the curve of 0.841. The observer study showed that the accuracies for the deep learning model, junior radiologist, and senior radiologist were 77.17%, 66.89%, and 77.03%, respectively. Stratified analyses showed that the deep learning model and radiologists exhibited higher performance in the subgroup of nodule sizes larger than 10 mm. With a longer time interval between two CTs, the deep learning model yielded higher diagnostic accuracy, but no general rules were yielded for radiologists. Different densities of components did not affect the performance of the deep learning model. In contrast, the radiologists were affected by the nodule component.

Conclusions

Deep learning can achieve diagnostic performance on par with or better than radiologists in identifying pulmonary GGNs.

Volumetric segmentation of ground glass nodule based on 3D attentional cascaded residual U-net and conditional radom field

BACKGROUND

Ground glass nodule (GGN) segmentation is one of the important and challenging tasks in diagnosing early-stage lung adenocarcinomas. Manually delineation of 3D GGN in computed tomography (CT) image is a subjective, laborious and tedious task, which presents poor repeatability.

PURPOSE

To reduce the annotation burden and improve the segmentation performance, this study proposes a 3D deep learning-based volumetric segmentation model to segment the GGN in CT images.

METHODS

A total of 379 GGNs were retrospectively collected from the public database, Shanghai Pulmonary Hospital (SHPH) and Fudan University Shanghai Cancer Center (FUSCC). First, a series of image pre-processing techniques involving image resampling, intensity normalization, 3D nodule patch cropping, and data augmentation, were adopted to generate the input images for the deep learning model by using the CT scans. Then, a 3D attentional cascaded residual network (ACRU-Net) was proposed to develop the deep learning-based segmentation model by using residual network and atrous spatial pyramid pooling module. To improve the model performance, a voxel-based conditional random field (CRF) method was used to optimize the segmentation results. Finally, a balanced cross-entropy and Dice combined loss function was applied to train and build the segmentation model.

RESULTS

Testing on SHPH and FUSCC datasets, the proposed method generates the Dice coefficients of 0.721±0.167 and 0.733±0.100, respectively, which are higher than that of 3D residual U-Net and ACRU-Net without CRF optimization.

CONCLUSIONS

The results demonstrated that combining 3D ACRU-Net and CRF effectively improved the GGN segmentation performance. The proposed segmentation model may provide a potential tool to help the radiologist in the segmentation and diagnosis of 3D GGN. This article is protected by copyright. All rights reserved.

Convolutional Neural Network-Based Diagnostic Model for a Solid, Indeterminate Solitary Pulmonary Nodule or Mass on Computed Tomography

PURPOSE

To establish a non-invasive diagnostic model based on convolutional neural networks (CNNs) to distinguish benign from malignant lesions manifesting as a solid, indeterminate solitary pulmonary nodule (SPN) or mass (SPM) on computed tomography (CT).

METHOD

A total of 459 patients with solid indeterminate SPNs/SPMs on CT were ultimately included in this retrospective study and assigned to the train (n=366), validation (n=46), and test (n=47) sets. Histopathologic analysis was available for each patient. An end-to-end CNN model was proposed to predict the natural history of solid indeterminate SPN/SPMs on CT. Receiver operating characteristic curves were plotted to evaluate the predictive performance of the proposed CNN model. The accuracy, sensitivity, and specificity of diagnoses by radiologists alone were compared with those of diagnoses by radiologists by using the CNN model to assess its clinical utility.

RESULTS

For the CNN model, the AUC was 91% (95% confidence interval [CI]: 0.83-0.99) in the test set. The diagnostic accuracy of radiologists with the CNN model was significantly higher than that without the model (89 vs. 66%, P<0.01; 87 vs. 61%, P<0.01; 85 vs. 66%, P=0.03, in the train, validation, and test sets, respectively). In addition, while there was a slight increase in sensitivity, the specificity improved significantly by an average of 42% (the corresponding improvements in the three sets ranged from 43, 33, and 42% to 82, 78, and 84%, respectively; P<0.01 for all).

CONCLUSION

The CNN model could be a valuable tool in non-invasively differentiating benign from malignant lesions manifesting as solid, indeterminate SPNs/SPMs on CT.

Classification of Clinically Significant Prostate Cancer on Multi-Parametric MRI: A Validation Study Comparing Deep Learning and Radiomics

The computer-aided analysis of prostate multiparametric MRI (mpMRI) could improve significant-prostate-cancer (PCa) detection. Various deep-learning- and radiomics-based methods for significant-PCa segmentation or classification have been reported in the literature. To be able to assess the generalizability of the performance of these methods, using various external data sets is crucial. While both deep-learning and radiomics approaches have been compared based on the same data set of one center, the comparison of the performances of both approaches on various data sets from different centers and different scanners is lacking. The goal of this study was to compare the performance of a deep-learning model with the performance of a radiomics model for the significant-PCa diagnosis of the cohorts of various patients. We included the data from two consecutive patient cohorts from our own center (n = 371 patients), and two external sets of which one was a publicly available patient cohort (n = 195 patients) and the other contained data from patients from two hospitals (n = 79 patients). Using multiparametric MRI (mpMRI), the radiologist tumor delineations and pathology reports were collected for all patients. During training, one of our patient cohorts (n = 271 patients) was used for both the deep-learning- and radiomics-model development, and the three remaining cohorts (n = 374 patients) were kept as unseen test sets. The performances of the models were assessed in terms of their area under the receiver-operating-characteristic curve (AUC). Whereas the internal cross-validation showed a higher AUC for the deep-learning approach, the radiomics model obtained AUCs of 0.88, 0.91 and 0.65 on the independent test sets compared to AUCs of 0.70, 0.73 and 0.44 for the deep-learning model. Our radiomics model that was based on delineated regions resulted in a more accurate tool for significant-PCa classification in the three unseen test sets when compared to a fully automated deep-learning model.

IMAL-Net: Interpretable multi-task attention learning network for invasive lung adenocarcinoma screening in CT images

PURPOSE

Feature maps created from deep convolutional neural networks (DCNNs) have been widely used for visual explanation of DCNN-based classification tasks. However, many clinical applications such as benign-malignant classification of lung nodules normally require quantitative and objective interpretability, rather than just visualization. In this paper, we propose a novel interpretable multi-task attention learning network named IMAL-Net for early invasive adenocarcinoma screening in chest computed tomography images, which takes advantage of segmentation prior to assist interpretable classification.

METHODS

Two sub-ResNets are firstly integrated together via a prior-attention mechanism for simultaneous nodule segmentation and invasiveness classification. Then, numerous radiomic features from the segmentation results are concatenated with high-level semantic features from the classification subnetwork by FC layers to achieve superior performance. Meanwhile, an end-to-end feature selection mechanism (named FSM) is designed to quantify crucial radiomic features greatly affecting the prediction of each sample, and thus it can provide clinically applicable interpretability to the prediction result.

RESULTS

Nodule samples from a total of 1626 patients were collected from two grade-A hospitals for large-scale verification. Five-fold cross validation demonstrated that the proposed IMAL-Net can achieve an AUC score of 93.8% ± 1.1% and a recall score of 93.8% ± 2.8% for identification of invasive lung adenocarcinoma.

CONCLUSIONS

It can be concluded that fusing semantic features and radiomic features can achieve obvious improvements in the invasiveness classification task. Moreover, by learning more fine-grained semantic features and highlighting the most important radiomics features, the proposed attention and FSM mechanisms not only can further improve the performance but also can be used for both visual explanations and objective analysis of the classification results.

CT-based radiomics for differentiating invasive adenocarcinomas from indolent lung adenocarcinomas appearing as ground-glass nodules: Asystematic review

PURPOSE

To provide an overview of the available studies investigating the use of computer tomography (CT) radiomics features for differentiating invasive adenocarcinomas (IAC) from indolent lung adenocarcinomas presenting as ground-glass nodules (GGNs), to identify the bias of the studies and to propose directions for future research.

METHOD

PubMed, Embase, Web of Science Core Collection were searched for relevant studies. The studies differentiating IAC from indolent lung adenocarcinomas appearing as GGNs based on CT radiomics features were included. Basic information, patient information, CT-scanner information, technique information and performance information were extracted for each included study. The quality of each study was assessed using the Radiomic Quality Score (RQS) and the Prediction model Risk of Bias Assessment Tool (PROBAST).

RESULTS

Twenty-eight studies were included with patients ranging from 34 to 794. All of them were retrospective. Patients in three studies were from multiple centers. Most studies segmented regions of interest manually. Pyradiomics and AK software were the most frequently used for features extraction. The number of radiomics features extracted varied from 7 to 10329. Logistic regression was the most frequently chosen model. Entropy was identified as radiomics signature in seven studies. The AUC of included studies ranged from 0.77 to 0.98 in 15 validation sets. The percentage RQS ranged from 3% to 50%. According to PROBAST, the overall risk of bias (ROB) was high in 89.3% (25/28) of included studies, unclear in 7.1% (2/28) of included studies, and low in 3.6% (1/28) of included studies. All studies were low concern regarding the applicability of primary studies to the review question.

CONCLUSION

CT radiomics-based model is promising and encouraging in differentiating IAC from indolent lung adenocarcinomas, though they require methodological rigor. Well-designed studies are necessary to demonstrate their validity and standardization of methods and results can prompt their use in daily clinical practice.

Deep Learning Application for Analyzing of Constituents and Their Correlations in the Interpretations of Medical Images

The need for time and attention, given by the doctor to the patient, due to the increased volume of medical data to be interpreted and filtered for diagnostic and therapeutic purposes has encouraged the development of the option to support, constructively and effectively, deep learning models. Deep learning (DL) has experienced an exponential development in recent years, with a major impact on interpretations of the medical image. This has influenced the development, diversification and increase of the quality of scientific data, the development of knowledge construction methods and the improvement of DL models used in medical applications. All research papers focus on description, highlighting, classification of one of the constituent elements of deep learning models (DL), used in the interpretation of medical images and do not provide a unified picture of the importance and impact of each constituent in the performance of DL models. The novelty in our paper consists primarily in the unitary approach, of the constituent elements of DL models, namely, data, tools used by DL architectures or specifically constructed DL architecture combinations and highlighting their "key" features, for completion of tasks in current applications in the interpretation of medical images. The use of "key" characteristics specific to each constituent of DL models and the correct determination of their correlations, may be the subject of future research, with the aim of increasing the performance of DL models in the interpretation of medical images.

Deep Learning-Based Stage-Wise Risk Stratification for Early Lung Adenocarcinoma in CT Images: A Multi-Center Study

Simple Summary

Prediction of the malignancy and invasiveness of ground glass nodules (GGNs) from computed tomography images is a crucial task for radiologists in risk stratification of early-stage lung adenocarcinoma. In order to solve this challenge, a two-stage deep neural network (DNN) was developed based on the images collected from four centers. A multi-reader multi-case observer study was conducted to evaluate the model capability. The performance of our model was comparable or even more accurate than that of senior radiologists, with average area under the curve values of 0.76 and 0.95 for two tasks, respectively. Findings suggest (1) a positive trend between the diagnostic performance and radiologist’s experience, (2) DNN yielded equivalent or even higher performance in comparison with senior radiologists, and (3) low image resolution reduced the model performance in predicting the risks of GGNs.

Abstract

This study aims to develop a deep neural network (DNN)-based two-stage risk stratification model for early lung adenocarcinomas in CT images, and investigate the performance compared with practicing radiologists. A total of 2393 GGNs were retrospectively collected from 2105 patients in four centers. All the pathologic results of GGNs were obtained from surgically resected specimens. A two-stage deep neural network was developed based on the 3D residual network and atrous convolution module to diagnose benign and malignant GGNs (Task1) and classify between invasive adenocarcinoma (IA) and non-IA for these malignant GGNs (Task2). A multi-reader multi-case observer study with six board-certified radiologists’ (average experience 11 years, range 2–28 years) participation was conducted to evaluate the model capability. DNN yielded area under the receiver operating characteristic curve (AUC) values of 0.76 ± 0.03 (95% confidence interval (CI): (0.69, 0.82)) and 0.96 ± 0.02 (95% CI: (0.92, 0.98)) for Task1 and Task2, which were equivalent to or higher than radiologists in the senior group with average AUC values of 0.76 and 0.95, respectively (p > 0.05). With the CT image slice thickness increasing from 1.15 mm ± 0.36 to 1.73 mm ± 0.64, DNN performance decreased 0.08 and 0.22 for the two tasks. The results demonstrated (1) a positive trend between the diagnostic performance and radiologist’s experience, (2) the DNN yielded equivalent or even higher performance in comparison with senior radiologists, and (3) low image resolution decreased model performance in predicting the risks of GGNs. Once tested prospectively in clinical practice, the DNN could have the potential to assist doctors in precision diagnosis and treatment of early lung adenocarcinoma.

On the performance of lung nodule detection, segmentation and classification

Computed tomography (CT) screening is an effective way for early detection of lung cancer in order to improve the survival rate of such a deadly disease. For more than two decades, image processing techniques such as nodule detection, segmentation, and classification have been extensively studied to assist physicians in identifying nodules from hundreds of CT slices to measure shapes and HU distributions of nodules automatically and to distinguish their malignancy. Thanks to new parallel computation, multi-layer convolution, nonlinear pooling operation, and the big data learning strategy, recent development of deep-learning algorithms has shown great progress in lung nodule screening and computer-assisted diagnosis (CADx) applications due to their high sensitivity and low false positive rates. This paper presents a survey of state-of-the-art deep-learning-based lung nodule screening and analysis techniques focusing on their performance and clinical applications, aiming to help better understand the current performance, the limitation, and the future trends of lung nodule analysis.

Structural and functional radiomics for lung cancer

INTRODUCTION

Lung cancer ranks second in new cancer cases and first in cancer-related deaths worldwide. Precision medicine is working on altering treatment approaches and improving outcomes in this patient population. Radiological images are a powerful non-invasive tool in the screening and diagnosis of early-stage lung cancer, treatment strategy support, prognosis assessment, and follow-up for advanced-stage lung cancer. Recently, radiological features have evolved from solely semantic to include (handcrafted and deep) radiomic features. Radiomics entails the extraction and analysis of quantitative features from medical images using mathematical and machine learning methods to explore possible ties with biology and clinical outcomes.

METHODS

Here, we outline the latest applications of both structural and functional radiomics in detection, diagnosis, and prediction of pathology, gene mutation, treatment strategy, follow-up, treatment response evaluation, and prognosis in the field of lung cancer.

CONCLUSION

The major drawbacks of radiomics are the lack of large datasets with high-quality data, standardization of methodology, the black-box nature of deep learning, and reproducibility. The prerequisite for the clinical implementation of radiomics is that these limitations are addressed. Future directions include a safer and more efficient model-training mode, merge multi-modality images, and combined multi-discipline or multi-omics to form "Medomics."

Computer-aided diagnosis of ground glass pulmonary nodule by fusing deep learning and radiomics features

OBJECTIVES

This study aims to develop a computer-aided diagnosis (CADx) scheme to classify between benign and malignant ground glass nodules (GGNs), and fuse deep leaning and radiomics imaging features to improve the classification performance.

METHODS

We first retrospectively collected 513 surgery histopathology confirmed GGNs from two centers. Among these GGNs, 100 were benign and 413 were malignant. All malignant tumors were stage I lung adenocarcinoma. To segment GGNs, we applied a deep convolutional neural network and residual architecture to train and build a 3D U-Net. Then, based on the pre-trained U-Net, we used a transfer learning approach to build a deep neural network (DNN) to classify between benign and malignant GGNs. With the GGN segmentation results generated by 3D U-Net, we also developed a CT radiomics model by adopting a series of image processing techniques, i.e. radiomics feature extraction, feature selection, synthetic minority over-sampling technique, and support vector machine classifier training/testing, etc. Finally, we applied an information fusion method to fuse the prediction scores generated by DNN based CADx model and CT-radiomics based model. To evaluate the proposed model performance, we conducted a comparison experiment by testing on an independent testing dataset.

RESULTS

Comparing with DNN model and radiomics model, our fusion model yielded a significant higher area under a receiver operating characteristic curve (AUC) value of 0.73 ± 0.06 (P < 0.01). The fusion model generated an accuracy of 75.6%, F1 score of 84.6%, weighted average F1 score of 70.3%, and Matthews correlation coefficient of 43.6%, which were higher than the DNN model and radiomics model individually.

CONCLUSIONS

Our experimental results demonstrated that (1) applying a CADx scheme was feasible to diagnosis of early-stage lung adenocarcinoma, (2) deep image features and radiomics features provided complementary information in classifying benign and malignant GGNs, and (3) it was an effective way to build DNN model with limited dataset by using transfer learning. Thus, to build a robust image analysis based CADx model, one can combine different types of image features to decode the imaging phenotypes of GGN.

Predicting benign, preinvasive, and invasive lung nodules on computed tomography scans using machine learning

OBJECTIVE

The study objective was to investigate if machine learning algorithms can predict whether a lung nodule is benign, adenocarcinoma, or its preinvasive subtype from computed tomography images alone.

METHODS

A dataset of chest computed tomography scans containing lung nodules was collected with their pathologic diagnosis from several sources. The dataset was split randomly into training (70%), internal validation (15%), and independent test sets (15%) at the patient level. Two machine learning algorithms were developed, trained, and validated. The first algorithm used the support vector machine model, and the second used deep learning technology: a convolutional neural network. Receiver operating characteristic analysis was used to evaluate the performance of the classification on the test dataset.

RESULTS

The support vector machine/convolutional neural network-based models classified nodules into 6 categories resulting in an area under the curve of 0.59/0.65 when differentiating atypical adenomatous hyperplasia versus adenocarcinoma in situ, 0.87/0.86 with minimally invasive adenocarcinoma versus invasive adenocarcinoma, 0.76/0.72 atypical adenomatous hyperplasia + adenocarcinoma in situ versus minimally invasive adenocarcinoma, 0.89/0.87 atypical adenomatous hyperplasia + adenocarcinoma in situ versus minimally invasive adenocarcinoma + invasive adenocarcinoma, and 0.93/0.92 atypical adenomatous hyperplasia + adenocarcinoma in situ + minimally invasive adenocarcinoma versus invasive adenocarcinoma. Classifying benign versus atypical adenomatous hyperplasia + adenocarcinoma in situ + minimally invasive adenocarcinoma versus invasive adenocarcinoma resulted in a micro-average area under the curve of 0.93/0.94 for the support vector machine/convolutional neural network models, respectively. The convolutional neural network-based methods had higher sensitivities than the support vector machine-based methods but lower specificities and accuracies.

CONCLUSIONS

The machine learning algorithms demonstrated reasonable performance in differentiating benign versus preinvasive versus invasive adenocarcinoma from computed tomography images alone. However, the prediction accuracy varies across its subtypes. This holds the potential for improved diagnostic capabilities with less-invasive means.

A survey on incorporating domain knowledge into deep learning for medical image analysis

Although deep learning models like CNNs have achieved great success in medical image analysis, the small size of medical datasets remains a major bottleneck in this area. To address this problem, researchers have started looking for external information beyond current available medical datasets. Traditional approaches generally leverage the information from natural images via transfer learning. More recent works utilize the domain knowledge from medical doctors, to create networks that resemble how medical doctors are trained, mimic their diagnostic patterns, or focus on the features or areas they pay particular attention to. In this survey, we summarize the current progress on integrating medical domain knowledge into deep learning models for various tasks, such as disease diagnosis, lesion, organ and abnormality detection, lesion and organ segmentation. For each task, we systematically categorize different kinds of medical domain knowledge that have been utilized and their corresponding integrating methods. We also provide current challenges and directions for future research.

Machine and Deep Learning Based Radiomics Models for Preoperative Prediction of Benign and Malignant Sacral Tumors

Purpose

To assess the performance of deep neural network (DNN) and machine learning based radiomics on 3D computed tomography (CT) and clinical characteristics to predict benign or malignant sacral tumors.

Materials and methods

This single-center retrospective analysis included 459 patients with pathologically proven sacral tumors. After semi-automatic segmentation, 1,316 hand-crafted radiomics features of each patient were extracted. All models were built on training set (321 patients) and tested on validation set (138 patients). A DNN model and four machine learning classifiers (logistic regression [LR], random forest [RF], support vector machine [SVM] and k-nearest neighbor [KNN]) based on CT features and clinical characteristics were built, respectively. The area under the receiver operating characteristic curve (AUC) and accuracy (ACC) were used to evaluate different models.

Results

In total, 459 patients (255 males, 204 females; mean age of 42.1 ± 17.8 years, range 4–82 years) were enrolled in this study, including 206 cases of benign tumor and 253 cases of malignant tumor. The sex, age and tumor size had significant differences between the benign tumors and malignant tumors (χ²_sex = 10.854, Z_age = −6.616, Z_size = 2.843, P < 0.05). The radscore, sex, and age were important indicators for differentiating benign and malignant sacral tumors (odds ratio [OR]1 = 2.492, OR2 = 2.236, OR3 = 1.037, P < 0.01). Among the four clinical-radiomics models (RMs), clinical-LR had the best performance in the validation set (AUC = 0.84, ACC = 0.81). The clinical-DNN model also achieved a high performance (an AUC of 0.83 and an ACC of 0.76 in the validation set) in identifying benign and malignant sacral tumors.

Conclusions

Both the clinical-LR and clinical-DNN models would have a high impact on assisting radiologists in their clinical diagnosis of sacral tumors.