Classifier performance prediction for computer-aided diagnosis using a limited dataset

Evaluation of the performance of both machine learning models using PET and CT radiomics for predicting recurrence following lung stereotactic body radiation therapy: A single-institutional study

PURPOSE

Predicting recurrence following stereotactic body radiotherapy (SBRT) for non-small cell lung cancer provides important information for the feasibility of the individualized radiotherapy and allows to select the appropriate treatment strategy based on the risk of recurrence. In this study, we evaluated the performance of both machine learning models using positron emission tomography (PET) and computed tomography (CT) radiomic features for predicting recurrence after SBRT.

METHODS

Planning CT and PET images of 82 non-small cell lung cancer patients who performed SBRT at our hospital were used. First, tumors were delineated on each CT and PET of each patient, and 111 unique radiomic features were extracted, respectively. Next, the 10 features were selected using three different feature selection algorithms, respectively. Recurrence prediction models based on the selected features and four different machine learning algorithms were developed, respectively. Finally, we compared the predictive performance of each model for each recurrence pattern using the mean area under the curve (AUC) calculated following the 0.632+ bootstrap method.

RESULTS

The highest performance for local recurrence, regional lymph node metastasis, and distant metastasis were observed in models using Support vector machine with PET features (mean AUC = 0.646), Naive Bayes with PET features (mean AUC = 0.611), and Support vector machine with CT features (mean AUC = 0.645), respectively.

CONCLUSIONS

We comprehensively evaluated the performance of prediction model developed for recurrence following SBRT. The model in this study would provide information to predict the recurrence pattern and assist in making treatment strategies.

Role of sureness in evaluating AI/CADx: Lesion-based repeatability of machine learning classification performance on breast MRI

BACKGROUND

Artificial intelligence/computer-aided diagnosis (AI/CADx) and its use of radiomics have shown potential in diagnosis and prognosis of breast cancer. Performance metrics such as the area under the receiver operating characteristic (ROC) curve (AUC) are frequently used as figures of merit for the evaluation of CADx. Methods for evaluating lesion-based measures of performance may enhance the assessment of AI/CADx pipelines, particularly in the situation of comparing performances by classifier.

PURPOSE

The purpose of this study was to investigate the use case of two standard classifiers to (1) compare overall classification performance of the classifiers in the task of distinguishing between benign and malignant breast lesions using radiomic features extracted from dynamic contrast-enhanced magnetic resonance (DCE-MR) images, (2) define a new repeatability metric (termed sureness), and (3) use sureness to examine if one classifier provides an advantage in AI diagnostic performance by lesion when using radiomic features.

METHODS

Images of 1052 breast lesions (201 benign, 851 cancers) had been retrospectively collected under HIPAA/IRB compliance. The lesions had been segmented automatically using a fuzzy c-means method and thirty-two radiomic features had been extracted. Classification was investigated for the task of malignant lesions (81% of the dataset) versus benign lesions (19%). Two classifiers (linear discriminant analysis, LDA and support vector machines, SVM) were trained and tested within 0.632 bootstrap analyses (2000 iterations). Whole-set classification performance was evaluated at two levels: (1) the 0.632+ bias-corrected area under the ROC curve (AUC) and (2) performance metric curves which give variability in operating sensitivity and specificity at a target operating point (95% target sensitivity). Sureness was defined as 1-95% confidence interval of the classifier output for each lesion for each classifier. Lesion-based repeatability was evaluated at two levels: (1) repeatability profiles, which represent the distribution of sureness across the decision threshold and (2) sureness of each lesion. The latter was used to identify lesions with better sureness with one classifier over another while maintaining lesion-based performance across the bootstrap iterations.

RESULTS

In classification performance assessment, the median and 95% CI of difference in AUC between the two classifiers did not show evidence of difference (ΔAUC = -0.003 [-0.031, 0.018]). Both classifiers achieved the target sensitivity. Sureness was more consistent across the classifier output range for the SVM classifier than the LDA classifier. The SVM resulted in a net gain of 33 benign lesions and 307 cancers with higher sureness and maintained lesion-based performance. However, with the LDA there was a notable percentage of benign lesions (42%) with better sureness but lower lesion-based performance.

CONCLUSIONS

When there is no evidence for difference in performance between classifiers using AUC or other performance summary measures, a lesion-based sureness metric may provide additional insight into AI pipeline design. These findings present and emphasize the utility of lesion-based repeatability via sureness in AI/CADx as a complementary enhancement to other evaluation measures.

AAPM task group report 273: Recommendations on best practices for AI and machine learning for computer-aided diagnosis in medical imaging

Rapid advances in artificial intelligence (AI) and machine learning, and specifically in deep learning (DL) techniques, have enabled broad application of these methods in health care. The promise of the DL approach has spurred further interest in computer-aided diagnosis (CAD) development and applications using both "traditional" machine learning methods and newer DL-based methods. We use the term CAD-AI to refer to this expanded clinical decision support environment that uses traditional and DL-based AI methods. Numerous studies have been published to date on the development of machine learning tools for computer-aided, or AI-assisted, clinical tasks. However, most of these machine learning models are not ready for clinical deployment. It is of paramount importance to ensure that a clinical decision support tool undergoes proper training and rigorous validation of its generalizability and robustness before adoption for patient care in the clinic. To address these important issues, the American Association of Physicists in Medicine (AAPM) Computer-Aided Image Analysis Subcommittee (CADSC) is charged, in part, to develop recommendations on practices and standards for the development and performance assessment of computer-aided decision support systems. The committee has previously published two opinion papers on the evaluation of CAD systems and issues associated with user training and quality assurance of these systems in the clinic. With machine learning techniques continuing to evolve and CAD applications expanding to new stages of the patient care process, the current task group report considers the broader issues common to the development of most, if not all, CAD-AI applications and their translation from the bench to the clinic. The goal is to bring attention to the proper training and validation of machine learning algorithms that may improve their generalizability and reliability and accelerate the adoption of CAD-AI systems for clinical decision support.

Pattern Classification for Ovarian Tumors by Integration of Radiomics and Deep Learning Features

BACKGROUND

Ovarian tumor is a common female genital tumor, among which malignant tumors have a poor prognosis. The survival rate of 70% of patients with ovarian cancer is less than 5 years, while benign ovarian tumor is better, so the early diagnosis of ovarian cancer is important for the treatment and prognosis of patients.

OBJECTIVES

Our aim is to establish a classification model for ovarian tumors.

METHODS

We extracted radiomics and deep learning features from patients'CT images. The four-step feature selection algorithm proposed in this paper was used to obtain the optimal combination of features, then, a classification model was developed by combining those selected features and support vector machine. The receiver operating characteristic curve and an area under the curve (AUC) analysis were used to evaluate the performance of the classification model in both the training and test cohort.

RESULTS

The classification model, which combined radiomics features with deep learning features, demonstrated better classification performance with respect to the radiomics features model alone in training cohort (AUC 0.9289 vs. 0.8804, P < 0.0001, accuracy 0.8970 vs. 0.7993, P < 0.0001), and significantly improve the performance in the test cohort (AUC 0.9089 vs. 0.8446, P = 0.001, accuracy 0.8296 vs. 0.7259, P < 0.0001).

CONCLUSION

The experiments showed that deep learning features play an active role in the construction of classification model, and the proposed classification model achieved excellent classification performance, which can potentially become a new auxiliary diagnostic tool.

A deep learning-based radiomics approach to predict head and neck tumor regression for adaptive radiotherapy

Early regression—the regression in tumor volume during the initial phase of radiotherapy (approximately 2 weeks after treatment initiation)—is a common occurrence during radiotherapy. This rapid radiation-induced tumor regression may alter target coordinates, necessitating adaptive radiotherapy (ART). We developed a deep learning-based radiomics (DLR) approach to predict early head and neck tumor regression and thereby facilitate ART. Primary gross tumor volume (GTVp) was monitored in 96 patients and nodal GTV (GTVn) in 79 patients during treatment. All patients underwent two computed tomography (CT) scans: one before the start of radiotherapy for initial planning and one during radiotherapy for boost planning. Patients were assigned to regression and nonregression groups according to their median tumor regression rate (ΔGTV/treatment day from initial to boost CT scan). We input a GTV image into the convolutional neural network model, which was pretrained using natural image datasets, via transfer learning. The deep features were extracted from the last fully connected layer. To clarify the prognostic power of the deep features, machine learning models were trained. The models then predicted the regression and nonregression of GTVp and GTVn and evaluated the predictive performance by 0.632 + bootstrap area under the curve (AUC). Predictive performance for GTVp regression was highest using the InceptionResNetv2 model (mean AUC = 0.75) and that for GTVn was highest using NASNetLarge (mean AUC = 0.73). Both models outperformed the handcrafted radiomics features (mean AUC = 0.63 for GTVp and 0.61 for GTVn) or clinical factors (0.64 and 0.67, respectively). DLR may facilitate ART for improved radiation side-effects and target coverage.

Performance metric curve analysis framework to assess impact of the decision variable threshold, disease prevalence, and dataset variability in two-class classification

Purpose: The aim of this study is to (1) demonstrate a graphical method and interpretation framework to extend performance evaluation beyond receiver operating characteristic curve analysis and (2) assess the impact of disease prevalence and variability in training and testing sets, particularly when a specific operating point is used. Approach: The proposed performance metric curves (PMCs) simultaneously assess sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV), and the 95% confidence intervals thereof, as a function of the threshold for the decision variable. We investigated the utility of PMCs using six example operating points associated with commonly used methods to select operating points (including the Youden index and maximum mutual information). As an example, we applied PMCs to the task of distinguishing between malignant and benign breast lesions using human-engineered radiomic features extracted from dynamic contrast-enhanced magnetic resonance images. The dataset had 1885 lesions, with the images acquired in 2015 and 2016 serving as the training set (1450 lesions) and those acquired in 2017 as the test set (435 lesions). Our study used this dataset in two ways: (1) the clinical dataset itself and (2) simulated datasets with features based on the clinical set but with five different disease prevalences. The median and 95% CI of the number of type I (false positive) and type II (false negative) errors were determined for each operating point of interest. Results: PMCs from both the clinical and simulated datasets demonstrated that PMCs could support interpretation of the impact of decision threshold choice on type I and type II errors of classification, particularly relevant to prevalence. Conclusion: PMCs allow simultaneous evaluation of the four performance metrics of sensitivity, specificity, PPV, and NPV as a function of the decision threshold. This may create a better understanding of two-class classifier performance in machine learning.

MRI-Based Radiomic Features Help Identify Lesions and Predict Histopathological Grade of Hepatocellular Carcinoma

Hepatocellular carcinoma (HCC) is the most common form of liver cancer. Radiomics is a promising tool that may increase the value of magnetic resonance imaging (MRI) in the management of HCC. The purpose of our study is to develop an MRI-based radiomics approach to preoperatively detect HCC and predict its histological grade. Thirty-eight HCC patients at staging who underwent axial T2-weighted and dynamic contrast-enhanced MRI (DCE-MRI) were considered. Three-dimensional volumes of interest (VOIs) were manually placed on HCC lesions and normal hepatic tissue (HT) on arterial phase post-contrast images. Radiomic features from T2 images and arterial, portal and tardive post-contrast images from DCE-MRI were extracted by using Pyradiomics. Feature selection was performed using correlation filter, Wilcoxon-rank sum test and mutual information. Predictive models were constructed for HCC differentiation with respect to HT and HCC histopathologic grading used at each step an imbalance-adjusted bootstrap resampling (IABR) on 1000 samples. Promising results were obtained from radiomic prediction models, with best AUCs ranging from 71% to 96%. Radiomics MRI based on T2 and DCE-MRI revealed promising results concerning both HCC detection and grading. It may be a suitable tool for personalized treatment of HCC patients and could also be used to develop new prognostic biomarkers useful for HCC assessment without the need for invasive procedures.

Finite element derived femoral strength is a better predictor of hip fracture risk than aBMD in the AGES Reykjavik study cohort

Hip fractures associated with a high economic burden, loss of independence, and a high rate of post-fracture mortality, are a major health concern for modern societies. Areal bone mineral density is the current clinical metric of choice when assessing an individual's future risk of fracture. However, this metric has been shown to lack sensitivity and specificity in the targeted selection of individuals for preventive interventions. Although femoral strength derived from computed tomography based finite element models has been proposed as an alternative based on its superior femoral strength prediction ex vivo, such predictions have only shown marginal or no improvement for assessing hip fracture risk. This study compares finite element derived femoral strength to aBMD as a metric for hip fracture risk assessment in subjects (N = 601) from the AGES Reykjavik Study cohort and analyses the dependence of femoral strength predictions and classification accuracy on the material model and femoral loading alignment. We found hip fracture classification based on finite element derived femoral strength to be significantly improved compared to aBMD. Finite element models with non-linear material models performed better at classifying hip fractures compared to finite element models with linear material models and loading alignments with low internal rotation and adduction, which do not correspond to weak femur alignments, were found to be most suitable for hip fracture classification.

Impact of radiogenomics in esophageal cancer on clinical outcomes: A pilot study

BACKGROUND

Esophageal cancer (ESCA) is the sixth most common malignancy in the world, and its incidence is rapidly increasing. Recently, several microRNAs (miRNAs) and messenger RNA (mRNA) targets were evaluated as potential biomarkers and regulators of epigenetic mechanisms involved in early diagnosis. In addition, computed tomography (CT) radiomic studies on ESCA improved the early stage identification and the prediction of response to treatment. Radiogenomics provides clinically useful prognostic predictions by linking molecular characteristics such as gene mutations and gene expression patterns of malignant tumors with medical images and could provide more opportunities in the management of patients with ESCA.

AIM

To explore the combination of CT radiomic features and molecular targets associated with clinical outcomes for characterization of ESCA patients.

METHODS

Of 15 patients with diagnosed ESCA were included in this study and their CT imaging and transcriptomic data were extracted from The Cancer Imaging Archive and gene expression data from The Cancer Genome Atlas, respectively. Cancer stage, history of significant alcohol consumption and body mass index (BMI) were considered as clinical outcomes. Radiomic analysis was performed on CT images acquired after injection of contrast medium. In total, 1302 radiomics features were extracted from three-dimensional regions of interest by using PyRadiomics. Feature selection was performed using a correlation filter based on Spearman’s correlation (ρ) and Wilcoxon-rank sum test respect to clinical outcomes. Radiogenomic analysis involved ρ analysis between radiomic features associated with clinical outcomes and transcriptomic signatures consisting of eight N6-methyladenosine RNA methylation regulators and five up-regulated miRNA. The significance level was set at P < 0.05.

RESULTS

Of 25, five and 29 radiomic features survived after feature selection, considering stage, alcohol history and BMI as clinical outcomes, respectively. Radiogenomic analysis with stage as clinical outcome revealed that six of the eight mRNA regulators and two of the five up-regulated miRNA were significantly correlated with ten and three of the 25 selected radiomic features, respectively (-0.61 < ρ < -0.60 and 0.53 < ρ < 0.69, P < 0.05). Assuming alcohol history as clinical outcome, no correlation was found between the five selected radiomic features and mRNA regulators, while a significant correlation was found between one radiomic feature and three up-regulated miRNAs (ρ = -0.56, ρ = -0.64 and ρ = 0.61, P < 0.05). Radiogenomic analysis with BMI as clinical outcome revealed that four mRNA regulators and one up-regulated miRNA were significantly correlated with 10 and two radiomic features, respectively (-0.67 < ρ < -0.54 and 0.53 < ρ < 0.71, P < 0.05).

CONCLUSION

Our study revealed interesting relationships between the expression of eight N6-methyladenosine RNA regulators, as well as five up-regulated miRNAs, and CT radiomic features associated with clinical outcomes of ESCA patients.

Multiparametric Quantitative US Examination of Liver Fibrosis: A Feature-engineering and Machine-learning Based Analysis

Quantitative ultrasound (QUS), which is commonly used to extract quantitative features from the ultrasound radiofrequency (RF) data or the RF envelope signals for tissue characterization, is becoming a promising technique for noninvasive assessments of liver fibrosis. However, the number of feature variables examined and finally used in the existing QUS methods is typically small, to some extent limiting the diagnostic performance. Therefore, this paper devises a new multiparametric QUS (MP-QUS) method which enables the extraction of a large number of feature variables from US RF signals and allows for the use of feature-engineering and machinelearning based algorithms for liver fibrosis assessment. In the MP-QUS, eighty-four feature variables were extracted from multiple QUS parametric maps derived from the RF signals and the envelope data. Afterwards, feature reduction and selection were performed in turn to remove the feature redundancy and identify the best combination of features in the reduced feature set. Finally, a variety of machine-learning algorithms were tested for classifying liver fibrosis with the selected features, based on the results of which the optimal classifier was established and used for final classification. The performance of the proposed MPQUS method for staging liver fibrosis was evaluated on an animal model, with histologic examination as the reference standard. The mean accuracy, sensitivity, specificity and area under the receiver-operating-characteristic curve achieved by MP-QUS are respectively 83.38%, 86.04%, 80.82% and 0.891 for recognizing significant liver fibrosis, and 85.50%, 88.92%, 85.24% and 0.924 for diagnosing liver cirrhosis. The proposed MP-QUS method paves a way for its future extension to assess liver fibrosis in human subjects.

Prediction of chemotherapy response in breast cancer patients at pre-treatment using second derivative texture of CT images and machine learning

•

Textural and second derivative textural features of CT images can be used in conjunction with machine learning models to predict breast cancer response to chemotherapy prior to the start of treatment.

•

The proposed predictive model separates the patients at pre-treatment into two cohorts (responders/non-responders) with significantly different survival.

•

The proposed methodology is a step forward towards the precision oncology paradigm for breast cancer patients.

Although neoadjuvant chemotherapy (NAC) is a crucial component of treatment for locally advanced breast cancer (LABC), only about 70% of patients respond to it. Effective adjustment of NAC for individual patients can significantly improve survival rates of those resistant to standard regimens. Thus, the early prediction of NAC outcome is of great importance in facilitating a personalized paradigm for breast cancer therapeutics. In this study, quantitative computed tomography (qCT) parametric imaging in conjunction with machine learning techniques were investigated to predict LABC tumor response to NAC. Textural and second derivative textural (SDT) features of CT images of 72 patients diagnosed with LABC were analysed before the initiation of NAC to quantify intra-tumor heterogeneity. These quantitative features were processed through a correlation-based feature reduction followed by a sequential feature selection with a bootstrap 0.632+ area under the receiver operating characteristic (ROC) curve (AUC0.632+) criterion. The best feature subset consisted of a combination of one textural and three SDT features. Using these features, an AdaBoost decision tree could predict the patient response with a cross-validated AUC0.632+ accuracy, sensitivity and specificity of 0.88, 85%, 88% and 75%, respectively. This study demonstrates, for the first time, that a combination of textural and SDT features of CT images can be used to predict breast cancer response NAC prior to the start of treatment which can potentially facilitate early therapy adjustments.

Weakly unsupervised conditional generative adversarial network for image-based prognostic prediction for COVID-19 patients based on chest CT

Because of the rapid spread and wide range of the clinical manifestations of the coronavirus disease 2019 (COVID-19), fast and accurate estimation of the disease progression and mortality is vital for the management of the patients. Currently available image-based prognostic predictors for patients with COVID-19 are largely limited to semi-automated schemes with manually designed features and supervised learning, and the survival analysis is largely limited to logistic regression. We developed a weakly unsupervised conditional generative adversarial network, called pix2surv, which can be trained to estimate the time-to-event information for survival analysis directly from the chest computed tomography (CT) images of a patient. We show that the performance of pix2surv based on CT images significantly outperforms those of existing laboratory tests and image-based visual and quantitative predictors in estimating the disease progression and mortality of COVID-19 patients. Thus, pix2surv is a promising approach for performing image-based prognostic predictions.

Evaluation of a multiparametric MRI radiomic-based approach for stratification of equivocal PI-RADS 3 and upgraded PI-RADS 4 prostatic lesions

Despite the key-role of the Prostate Imaging and Reporting and Data System (PI-RADS) in the diagnosis and characterization of prostate cancer (PCa), this system remains to be affected by several limitations, primarily associated with the interpretation of equivocal PI-RADS 3 lesions and with the debated role of Dynamic Contrast Enhanced-Magnetic Resonance Imaging (DCE-MRI), which is only used to upgrade peripheral PI-RADS category 3 lesions to PI-RADS category 4 if enhancement is focal. We aimed at investigating the usefulness of radiomics for detection of PCa lesions (Gleason Score ≥ 6) in PI-RADS 3 lesions and in peripheral PI-RADS 3 upgraded to PI-RADS 4 lesions (upPI-RADS 4). Multiparametric MRI (mpMRI) data of patients who underwent prostatic mpMRI between April 2013 and September 2018 were retrospectively evaluated. Biopsy results were used as gold standard. PI-RADS 3 and PI-RADS 4 lesions were re-scored according to the PI-RADS v2.1 before and after DCE-MRI evaluation. Radiomic features were extracted from T2-weighted MRI (T2), Apparent diffusion Coefficient (ADC) map and DCE-MRI subtracted images using PyRadiomics. Feature selection was performed using Wilcoxon-ranksum test and Minimum Redundancy Maximum Relevance (mRMR). Predictive models were constructed for PCa detection in PI-RADS 3 and upPI-RADS 4 lesions using at each step an imbalance-adjusted bootstrap resampling (IABR) on 1000 samples. 41 PI-RADS 3 and 32 upPI-RADS 4 lesions were analyzed. Among 293 radiomic features, the top selected features derived from T2 and ADC. For PI-RADS 3 stratification, second order model showed higher performances (Area Under the Receiver Operating Characteristic Curve-AUC- = 80%), while for upPI-RADS 4 stratification, first order model showed higher performances respect to superior order models (AUC = 89%). Our results support the significant role of T2 and ADC radiomic features for PCa detection in lesions scored as PI-RADS 3 and upPI-RADS 4. Radiomics models showed high diagnostic efficacy in classify PI-RADS 3 and upPI-RADS 4 lesions, outperforming PI-RADS v2.1 performance.

Posaconazole and fluconazole prophylaxis during induction therapy for pediatric acute lymphoblastic leukemia

PURPOSE

To assess and compare the efficacy and safety of posaconazole with fluconazole for the prevention of invasive fungal infections in children who were undergoing induction therapy for acute lymphoblastic leukemia (ALL). To develop an approach to predict invasive fungal infections in ALL patients who accepted posaconazole prophylaxis.

METHODS

This was a single-center, retrospective cohort study of patients with newly diagnosed ALL, comparing invasive fungal infections in patients who received no prophylaxis, posaconazole prophylaxis, or fluconazole prophylaxis during induction therapy. A propensity score-weighted logistic regression model was used to adjust for confounders. Hepatotoxicity was assessed according to the Common Terminology Criteria for Adverse Events (CTCAE) criteria.

RESULTS

Out of the 155 ALL patients, 60 received no prophylaxis, 70 received posaconazole prophylaxis, and 25 received fluconazole prophylaxis. Posaconazole prophylaxis reduced the odds of invasive fungal infections by > 60%, prolonged infection-free survival significantly, and did not increase the risk of hepatotoxicity. Additionally, we found that the combination of age at diagnosis, clinically documented bacterial infection in the first 15 days of induction therapy, and absolute neutrophil count (ANC) curve enabled significant prediction of the susceptibility to infections after receiving posaconazole prophylaxis.

CONCLUSIONS

Our findings supported using targeted prophylaxis with posaconazole in ALL children undergoing induction chemotherapy. Age, clinically documented bacterial infection and ANC are important predictors of invasive fungal infections in patients with posaconazole prophylaxis.

A priori prediction of tumour response to neoadjuvant chemotherapy in breast cancer patients using quantitative CT and machine learning

Response to Neoadjuvant chemotherapy (NAC) has demonstrated a high correlation to survival in locally advanced breast cancer (LABC) patients. An early prediction of responsiveness to NAC could facilitate treatment adjustments on an individual patient basis that would be expected to improve treatment outcomes and patient survival. This study investigated, for the first time, the efficacy of quantitative computed tomography (qCT) parametric imaging to characterize intra-tumour heterogeneity and its application in predicting tumour response to NAC in LABC patients. Textural analyses were performed on CT images acquired from 72 patients before the start of chemotherapy to determine quantitative features of intra-tumour heterogeneity. The best feature subset for response prediction was selected through a sequential feature selection with bootstrap 0.632 + area under the receiver operating characteristic (ROC) curve (\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${\mathrm{A}\mathrm{U}\mathrm{C}}_{0.632+}$$\end{document}AUC0.632+) as a performance criterion. Several classifiers were evaluated for response prediction using the selected feature subset. Amongst the applied classifiers an Adaboost decision tree provided the best results with cross-validated \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${\mathrm{A}\mathrm{U}\mathrm{C}}_{0.632+}$$\end{document}AUC0.632+, accuracy, sensitivity and specificity of 0.89, 84%, 80% and 88%, respectively. The promising results obtained in this study demonstrate the potential of the proposed biomarkers to be used as predictors of LABC tumour response to NAC prior to the start of treatment.

Machine Learning-Based A Priori Chemotherapy Response Prediction in Breast Cancer Patients using Textural CT Biomarkers. ANNUAL INTERNATIONAL CONFERENCE OF THE IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. ANNUAL INTERNATIONAL CONFERENCE 2020;2020:1250-1253. [PMID: 33018214 DOI: 10.1109/embc44109.2020.9176099] [Citation(s) in RCA: 0] [Impact Index Per Article: 0]

Early prediction of cancer response to neoadjuvant chemotherapy (NAC) could permit personalized treatment adjustments for patients, which would improve treatment outcomes and patient survival. For the first time, the efficiency of quantitative computed tomography (qCT) textural and second derivative of textural (SDT) features were investigated and compared in this study. It was demonstrated that intra-tumour heterogeneity can be probed through these biomarkers and used as chemotherapy tumour response predictors in breast cancer patients prior to the start of treatment. These features were used to develop a machine learning approach which provided promising results with cross-validated AUC0.632+, accuracy, sensitivity and specificity of 0.86, 81%, 74% and 88%, respectively.Clinical Relevance- The results obtained in this study demonstrate the potential of textural CT biomarkers as response predictors of standard NAC before treatment initiation.

Generalization error analysis for deep convolutional neural network with transfer learning in breast cancer diagnosis

Deep convolutional neural network (DCNN), now popularly called artificial intelligence (AI), has shown the potential to improve over previous computer-assisted tools in medical imaging developed in the past decades. A DCNN has millions of free parameters that need to be trained, but the training sample set is limited in size for most medical imaging tasks so that transfer learning is typically used. Automatic data mining may be an efficient way to enlarge the collected data set but the data can be noisy such as incorrect labels or even a wrong type of image. In this work we studied the generalization error of DCNN with transfer learning in medical imaging for the task of classifying malignant and benign masses on mammograms. With a finite available data set, we simulated a training set containing corrupted data or noisy labels. The balance between learning and memorization of the DCNN was manipulated by varying the proportion of corrupted data in the training set. The generalization error of DCNN was analyzed by the area under the receiver operating characteristic curve for the training and test sets and the weight changes after transfer learning. The study demonstrates that the transfer learning strategy of DCNN for such tasks needs to be designed properly, taking into consideration the constraints of the available training set having limited size and quality for the classification task at hand, to minimize memorization and improve generalizability.

An MR Radiomics Framework for Predicting the Outcome of Stereotactic Radiation Therapy in Brain Metastasis

Despite recent advances in cancer treatment, patients with brain metastasis still suffer from poor overall survival (OS) after standard treatment. Predicting the treatment outcome before or early after the treatment can potentially assist the physicians in improving the therapy outcome by adjusting a standard treatment on an individual patient basis. In this study, a data-driven computational framework was proposed and investigated to predict the local control/failure (LC/LF) outcome in patients with brain metastasis treated with hypo-fractionated stereotactic radiation therapy (SRT). The framework extracted several geometrical and textural features from the magnetic resonance (MR) images of the tumour and edema regions acquired for 38 patients. Subsequent to a multi-step feature reduction/selection, a quantitative MR biomarker consisting of two features was constructed. A support vector machine classifier was used for outcome prediction using the constructed MR biomarker. The bootstrap .632+ and leave-one-patient-out cross-validation methods were used to assess the model's performance. The results indicated that the outcome of LF after SRT could be predicted with an area under the curve of 0.80 and a cross-validated accuracy of 82%. The results obtained implied a good potential of the proposed framework for local outcome prediction in patients with brain metastasis treated with SRT and encourage further investigations on a larger cohort of patients.

Assessment of the invariance and discriminant power of morphological features under geometric transformations for breast tumor classification

BACKGROUND AND OBJECTIVES

Computer-aided diagnosis (CAD) systems are intended to assist specialists in the interpretation of images aiming to support clinical conduct. In breast tumor classification, CAD systems involve a feature extraction stage, in which morphological features are used to describe the tumor shape. Such features are expected to satisfy at least two conditions: (1) discriminant to distinguish between benign and malignant tumors, and (2) invariant to geometric transformations. Herein, 39 morphological features were evaluated in terms of invariance and discriminant power for breast tumor classification.

METHODS

Morphological features were divided into region-based features, for describing the irregularity of the tumor shape, and boundary-based features, for measuring the anfractuosity of the tumor margin. Also, two datasets were considered in the experiments: 2054 breast ultrasound images and 892 mammographies. From both datasets, synthetic data augmentation was performed to obtain distinct combinations of rotation and scaling of breast tumors, from which morphological features were calculated. The linear discriminant analysis was used to classify breast tumors in benign and malignant classes. The area under the ROC curve (AUC) quantified the discriminant power of every morphological feature, whereas the relative difference (RD) between AUC values measured the invariance to geometric transformations. For indicating adequate performance, AUC and RD should tend toward unity and zero, respectively.

RESULTS

For both datasets, the convexity was the most discriminant feature that reached AUC > 0.81 with RD<1×10^-2, while the most invariant feature was the roundness that attained RD<1×10^-3 with AUC < 0.72. Additionally, for each dataset, the most discriminant and invariant features were combined for performing tumor classification. For mammography, it was achieved accuracy (ACC) of 0.76, sensitivity (SEN) of 0.76, and specificity (SPE) of 0.84, whereas for breast ultrasound the results were ACC=0.88,SEN=0.81, and SPE=0.91.

CONCLUSIONS

In general, region-based features are more discriminant and invariant than boundary-based features. Moreover, it was observed that an invariant feature is not necessarily a discriminant feature; hence, a balance between invariance and discriminant power should be attained for breast tumor classification.

Multiparametric MRI for Prostate Cancer Detection: New Insights into the Combined Use of a Radiomic Approach with Advanced Acquisition Protocol

Prostate cancer (PCa) is a disease affecting an increasing number of men worldwide. Several efforts have been made to identify imaging biomarkers to non-invasively detect and characterize PCa, with substantial improvements thanks to multiparametric Magnetic Resonance Imaging (mpMRI). In recent years, diffusion kurtosis imaging (DKI) was proposed to be directly related to tissue physiological and pathological characteristic, while the radiomic approach was proven to be a key method to study cancer imaging phenotypes. Our aim was to compare a standard radiomic model for PCa detection, built using T2-weighted (T2W) and Apparent Diffusion Coefficient (ADC), with an advanced one, including DKI and quantitative Dynamic Contrast Enhanced (DCE), while also evaluating differences in prediction performance when using 2D or 3D lesion segmentation. The obtained results in terms of diagnostic accuracy were high for all of the performed comparisons, reaching values up to 0.99 for the area under a receiver operating characteristic curve (AUC), and 0.98 for both sensitivity and specificity. In comparison, the radiomic model based on standard features led to prediction performances higher than those of the advanced model, while greater accuracy was achieved by the model extracted from 3D segmentation. These results provide new insights into active topics of discussion, such as choosing the most convenient acquisition protocol and the most appropriate postprocessing pipeline to accurately detect and characterize PCa.

Magnetic Resonance Imaging Texture Analysis Predicts Recurrence in Patients with Nasopharyngeal Carcinoma

Background

The personalization of oncologic treatment using radiomic signatures is mounting in nasopharyngeal carcinoma (NPC). We ascertain the predictive ability of 3D volumetric magnetic resonance imaging (MRI) texture features on NPC disease recurrence.

Methods

A retrospective study of 58 patients with NPC undergoing primary curative-intent treatment was performed. Forty-two image texture features were extracted from pre-treatment MRI in addition to clinical factors. A multivariate logistic regression was used to model the texture features. A receiver operating characteristic curve on 100 bootstrap samples was used to maximize generalizability to out-of-sample data. A Cox proportional model was used to predict disease recurrence in the final model.

Results

A total of 58 patients were included in the study. MRI texture features predicted disease recurrence with an area under the curve (AUC), sensitivity, and specificity of 0.79, 0.73, and 0.71, respectively. Loco-regional recurrence was predicted with AUC, sensitivity, and specificity of 0.82, 0.73 and 0.74 respectively while prediction for distant metastasis had an AUC, sensitivity, and specificity of 0.92, 0.79 and 0.84, respectively. Texture features on MRI had a hazard ratio of 4.37 (95% confidence interval 1.72–20.2) for disease recurrence when adjusting for age, sex, smoking, and TNM staging.

Conclusion

Texture features on MRI are independent predictors of NPC recurrence in patients undergoing curative-intent treatment.

The effects of volume of interest delineation on MRI-based radiomics analysis: evaluation with two disease groups

BACKGROUND

Manual delineation of volume of interest (VOI) is widely used in current radiomics analysis, suffering from high variability. The tolerance of delineation differences and possible influence on each step of radiomics analysis are not clear, requiring quantitative assessment. The purpose of our study was to investigate the effects of delineation of VOIs on radiomics analysis for the preoperative prediction of metastasis in nasopharyngeal carcinoma (NPC) and sentinel lymph node (SLN) metastasis in breast cancer.

METHODS

This study retrospectively enrolled two datasets (NPC group: 238 cases; SLN group: 146 cases). Three operations, namely, erosion, smoothing, and dilation, were implemented on the VOIs accurately delineated by radiologists to generate diverse VOI variations. Then, we extracted 2068 radiomics features and evaluated the effects of VOI differences on feature values by the intra-class correlation coefficient (ICC). Feature selection was conducted by Maximum Relevance Minimum Redundancy combined with 0.632+ bootstrap algorithms. The prediction performance of radiomics models with random forest classifier were tested on an independent validation cohort by the area under the receive operating characteristic curve (AUC).

RESULTS

The larger the VOIs changed, the fewer features with high ICCs. Under any variation, SLN group showed fewer features with ICC ≥ 0.9 compared with NPC group. Not more than 15% top-predictive features identical to the accurate VOIs were observed across feature selection. The differences of AUCs of models derived from VOIs across smoothing or dilation with 3 pixels were not statistically significant compared with the accurate VOIs (p > 0.05) except for T2-weighted fat suppression images (smoothing: 0.845 vs. 0.725, p = 0.001; dilation: 0.800 vs. 0.725, p = 0.042). Dilation with 5 and 7 pixels contributed to remarkable AUCs in SLN group but the opposite in NPC group. The radiomics models did not perform well when tested by data from other delineations.

CONCLUSIONS

Differences in delineation of VOIs affected radiomics analysis, related to specific disease and MRI sequences. Differences from smooth delineation or expansion with 3 pixels width around the tumors or lesions were acceptable. The delineation for radiomics analysis should follow a predefined and unified standard.

Classification of Soccer and Basketball Players' Jumping Performance Characteristics: A Logistic Regression Approach

This study aimed to examine countermovement jump (CMJ) kinetic data using logistic regression, in order to distinguish sports-related mechanical profiles. Eighty-one professional basketball and soccer athletes participated, each performing three CMJs on a force platform. Inferential parametric and nonparametric statistics were performed to explore group differences. Binary logistic regression was used to model the response variable (soccer or not soccer). Statistical significance (p < 0.05) was reached for differences between groups in maximum braking rate of force development (RFD_Dmax, U₇₉ = 1035), mean braking rate of force development (RFD_Davg, U₇₉ = 1038), propulsive impulse (IMP_U, t₇₉ = 2.375), minimum value of vertical displacement for center of mass (S_BCMmin, t₇₉ = 3.135), and time difference (% of impulse time; Δ_Τ) between the peak value of maximum force value (F_Umax) and S_BCMmin (U₇₉ = 1188). Logistic regression showed that RFD_Davg, impulse during the downward phase (IMP_D), IMP_U, and Δ_Τ were all significant predictors. The model showed that soccer group membership could be strongly related to IMP_U, with the odds ratio being 6.48 times higher from the basketball group, whereas RFD_Davg, IMP_D, and Δ_Τ were related to basketball group. The results imply that soccer players execute CMJ differently compared to basketball players, exhibiting increased countermovement depth and impulse generation during the propulsive phase.

Breast MRI radiomics for the pretreatment prediction of response to neoadjuvant chemotherapy in node-positive breast cancer patients

The purpose of this study was to evaluate breast MRI radiomics in predicting, prior to any treatment, the response to neoadjuvant chemotherapy (NAC) in patients with invasive lymph node (LN)-positive breast cancer for two tasks: (1) prediction of pathologic complete response and (2) prediction of post-NAC LN status. Our study included 158 patients, with 19 showing post-NAC complete pathologic response (pathologic TNM stage T0,N0,MX) and 139 showing incomplete response. Forty-two patients were post-NAC LN-negative, and 116 were post-NAC LN-positive. We further analyzed prediction of response by hormone receptor subtype of the primary cancer (77 hormone receptor-positive, 39 HER2-enriched, 38 triple negative, and 4 cancers with unknown receptor status). Only pre-NAC MRIs underwent computer analysis, initialized by an expert breast radiologist indicating index cancers and metastatic axillary sentinel LNs on DCE-MRI images. Forty-nine computer-extracted radiomics features were obtained, both for the primary cancers and for the metastatic sentinel LNs. Since the dataset contained MRIs acquired at 1.5 T and at 3.0 T, we eliminated features affected by magnet strength using the Mann-Whitney U-test with the null-hypothesis that 1.5 T and 3.0 T samples were selected from populations having the same distribution. Bootstrapping and ROC analysis were used to assess performance of individual features in the two classification tasks. Eighteen features appeared unaffected by magnet strength. Pre-NAC tumor features generally appeared uninformative in predicting response to therapy. In contrast, some pre-NAC LN features were able to predict response: two pre-NAC LN features were able to predict pathologic complete response (area under the ROC curve (AUC) up to 0.82 [0.70; 0.88]), and another two were able to predict post-NAC LN-status (AUC up to 0.72 [0.62; 0.77]), respectively. In the analysis by a hormone receptor subtype, several potentially useful features were identified for predicting response to therapy in the hormone receptor-positive and HER2-enriched cancers.

MRI features predict survival and molecular markers in diffuse lower-grade gliomas

Background

Previous studies have shown that MR imaging features can be used to predict survival and molecular profile of glioblastoma. However, no study of a similar type has been performed on lower-grade gliomas (LGGs).

Methods

Presurgical MRIs of 165 patients with diffuse low- and intermediate-grade gliomas (histological grades II and III) were scored according to the Visually Accessible Rembrandt Images (VASARI) annotations. Radiomic models using automated texture analysis and VASARI features were built to predict isocitrate dehydrogenase 1 (IDH1) mutation, 1p/19q codeletion status, histological grade, and tumor progression.

Results

Interrater analysis showed significant agreement in all imaging features scored (k = 0.703-1.000). On multivariate Cox regression analysis, no enhancement and a smooth non-enhancing margin were associated with longer progression-free survival (PFS), while a smooth non-enhancing margin was associated with longer overall survival (OS) after taking into account age, grade, tumor location, histology, extent of resection, and IDH1 1p/19q subtype. Using logistic regression and bootstrap testing evaluations, texture models were found to possess higher prediction potential for IDH1 mutation, 1p/19q codeletion status, histological grade, and progression of LGGs than VASARI features, with areas under the receiver-operating characteristic curves of 0.86 ± 0.01, 0.96 ± 0.01, 0.86 ± 0.01, and 0.80 ± 0.01, respectively.

Conclusion

No enhancement and a smooth non-enhancing margin on MRI were predictive of longer PFS, while a smooth non-enhancing margin was a significant predictor of longer OS in LGGs. Textural analyses of MR imaging data predicted IDH1 mutation, 1p/19q codeletion, histological grade, and tumor progression with high accuracy.

[A radiomics-based model for differentiation between benign and malignant gastrointestinal stromal tumors]

OBJECTIVE

To establish a model for discrimination between benign and malignant gastrointestinal stromal tumors (GIST) by analyzing the texture features extracted from computed tomography (CT) images.

METHODS

The CT datasets were collected from 110 patients with GIST (including 80 as the training cohort and 30 as the validation cohort). Feature set reduction was executed with the 0.632 + bootstrap method in the initial feature set followed by stepwise forward feature selection in the feature subset, and the classification model was generated by logistic regression.

RESULTS

The 6-texture-featurebased classification model successfully discriminated between benign and malignant GIST in both the training and validation cohorts with AUCs of 0.93 and 0.91, sensitivity of 0.88 and 0.87, specificity of 0.85 and 0.86, and accuracy of 0.87 and 0.86 in the two cohorts, respectively.

CONCLUSIONS

This classification model established by radiomics analysis is capable of discrimination between benign and malignant GIST to provide assistance in preoperative diagnosis of GIST.

DCE-MRI Pharmacokinetic-Based Phenotyping of Invasive Ductal Carcinoma: A Radiomic Study for Prediction of Histological Outcomes

Breast cancer is a disease affecting an increasing number of women worldwide. Several efforts have been made in the last years to identify imaging biomarker and to develop noninvasive diagnostic tools for breast tumor characterization and monitoring, which could help in patients' stratification, outcome prediction, and treatment personalization. In particular, radiomic approaches have paved the way to the study of the cancer imaging phenotypes. In this work, a group of 49 patients with diagnosis of invasive ductal carcinoma was studied. The purpose of this study was to select radiomic features extracted from a DCE-MRI pharmacokinetic protocol, including quantitative maps of ktrans, kep, ve, iAUC, and R1 and to construct predictive models for the discrimination of molecular receptor status (ER+/ER-, PR+/PR-, and HER2+/HER2-), triple negative (TN)/non-triple negative (NTN), ki67 levels, and tumor grade. A total of 163 features were obtained and, after feature set reduction step, followed by feature selection and prediction performance estimations, the predictive model coefficients were computed for each classification task. The AUC values obtained were 0.826 ± 0.006 for ER+/ER-, 0.875 ± 0.009 for PR+/PR-, 0.838 ± 0.006 for HER2+/HER2-, 0.876 ± 0.007 for TN/NTN, 0.811 ± 0.005 for ki67+/ki67-, and 0.895 ± 0.006 for lowGrade/highGrade. In conclusion, DCE-MRI pharmacokinetic-based phenotyping shows promising for discrimination of the histological outcomes.

Prediction of Metal Ion Binding Sites in Proteins from Amino Acid Sequences by Using Simplified Amino Acid Alphabets and Random Forest Model

Metal binding proteins or metallo-proteins are important for the stability of the protein and also serve as co-factors in various functions like controlling metabolism, regulating signal transport, and metal homeostasis. In structural genomics, prediction of metal binding proteins help in the selection of suitable growth medium for overexpression's studies and also help in obtaining the functional protein. Computational prediction using machine learning approach has been widely used in various fields of bioinformatics based on the fact all the information contains in amino acid sequence. In this study, random forest machine learning prediction systems were deployed with simplified amino acid for prediction of individual major metal ion binding sites like copper, calcium, cobalt, iron, magnesium, manganese, nickel, and zinc.

Radiomics strategies for risk assessment of tumour failure in head-and-neck cancer

Quantitative extraction of high-dimensional mineable data from medical images is a process known as radiomics. Radiomics is foreseen as an essential prognostic tool for cancer risk assessment and the quantification of intratumoural heterogeneity. In this work, 1615 radiomic features (quantifying tumour image intensity, shape, texture) extracted from pre-treatment FDG-PET and CT images of 300 patients from four different cohorts were analyzed for the risk assessment of locoregional recurrences (LR) and distant metastases (DM) in head-and-neck cancer. Prediction models combining radiomic and clinical variables were constructed via random forests and imbalance-adjustment strategies using two of the four cohorts. Independent validation of the prediction and prognostic performance of the models was carried out on the other two cohorts (LR: AUC = 0.69 and CI = 0.67; DM: AUC = 0.86 and CI = 0.88). Furthermore, the results obtained via Kaplan-Meier analysis demonstrated the potential of radiomics for assessing the risk of specific tumour outcomes using multiple stratification groups. This could have important clinical impact, notably by allowing for a better personalization of chemo-radiation treatments for head-and-neck cancer patients from different risk groups.

A comparison of resampling schemes for estimating model observer performance with small ensembles

In objective assessment of image quality, an ensemble of images is used to compute the 1st and 2nd order statistics of the data. Often, only a finite number of images is available, leading to the issue of statistical variability in numerical observer performance. Resampling-based strategies can help overcome this issue. In this paper, we compared different combinations of resampling schemes (the leave-one-out (LOO) and the half-train/half-test (HT/HT)) and model observers (the conventional channelized Hotelling observer (CHO), channelized linear discriminant (CLD) and channelized quadratic discriminant). Observer performance was quantified by the area under the ROC curve (AUC). For a binary classification task and for each observer, the AUC value for an ensemble size of 2000 samples per class served as a gold standard for that observer. Results indicated that each observer yielded a different performance depending on the ensemble size and the resampling scheme. For a small ensemble size, the combination [CHO, HT/HT] had more accurate rankings than the combination [CHO, LOO]. Using the LOO scheme, the CLD and CHO had similar performance for large ensembles. However, the CLD outperformed the CHO and gave more accurate rankings for smaller ensembles. As the ensemble size decreased, the performance of the [CHO, LOO] combination seriously deteriorated as opposed to the [CLD, LOO] combination. Thus, it might be desirable to use the CLD with the LOO scheme when smaller ensemble size is available.

Incorporation of pre-therapy 18 F-FDG uptake data with CT texture features into a radiomics model for radiation pneumonitis diagnosis

PURPOSE

To determine whether the addition of standardized uptake value (SUV) from PET scans to CT lung texture features could improve a radiomics-based model of radiation pneumonitis (RP) diagnosis in patients undergoing radiotherapy.

METHODS AND MATERIALS

Anonymized data from 96 esophageal cancer patients (18 RP-positive cases of Grade ≥ 2) were collected including pre-therapy PET/CT scans, pre-/post-therapy diagnostic CT scans and RP status. Twenty texture features (first-order, fractal, Laws' filter and gray-level co-occurrence matrix) were calculated from diagnostic CT scans and compared in anatomically matched regions of the lung. Classifier performance (texture, SUV, or combination) was assessed by calculating the area under the receiver operating characteristic curve (AUC). For each texture feature, logistic regression classifiers consisting of the average change in texture feature value and the pre-therapy SUV standard deviation (SUV_SD ) were created and compared with the texture feature as a lone classifier using ANOVA with correction for multiple comparisons (P < 0.0025).

RESULTS

While clinical parameters (mean lung dose, smoking history, tumor location) were not significantly different among patients with and without symptomatic RP, SUV and texture parameters were significantly associated with RP status. AUC for single-texture feature classifiers alone ranged from 0.58 to 0.81 and 0.53 to 0.71 in high-dose (≥ 30 Gy) and low-dose (< 10 Gy) regions of the lungs, respectively. AUC for SUV_SD alone was 0.69 (95% confidence interval: 0.54-0.83). Adding SUV_SD into a logistic regression model significantly improved model fit for 18, 14 and 11 texture features and increased the mean AUC across features by 0.08, 0.06, and 0.04 in the low-, medium-, and high-dose regions, respectively.

CONCLUSIONS

Addition of SUV_SD to a single-texture feature improves classifier performance on average, but the improvement is smaller in magnitude when SUV_SD is added to an already effective classifier using texture alone. These findings demonstrate the potential for more accurate assessment of RP using information from multiple imaging modalities.

Classification of breast lesions using segmented quantitative ultrasound maps of homodyned K distribution parameters

PURPOSE

Statistical modeling of an ultrasound backscattered echo envelope is used for tissue characterization. However, in the presence of complex structures within the analyzed area, estimation of parameters is disturbed and unreliable, e.g., in the case of breast tumor classification. In order to improve the differentiation of breast lesions, the authors proposed a method based on the segmentation of homodyned K distribution parameter maps. Regions within lesions of different scattering properties were extracted and analyzed. In order to improve the classification, the best-performing features were selected from various regions and then combined.

METHODS

A radio-frequency data set consisting of 103 breast lesions was used in the authors' analysis. Maps of homodyned K distribution parameters were created using an algorithm based on signal-to-noise ratio, kurtosis, and skewness of fractional-order envelope moments. A Markov random field model was used to segment parametric maps. Features of different segments were extracted and evaluated based on bootstrapping and the receiver operating characteristic curve. To determine the best-performing feature subset, the authors applied the joint mutual information criterion.

RESULTS

It was found that there were individual features which performed better than the ones commonly used for lesion characterization, like the parameter obtained through averaging of values over the whole lesion. The authors selected and discussed the best-performing features. Properties of different extracted regions were important and improved the distinction between benign and malignant tumors. The best performance was obtained by combining four features with the area under the receiver operating curve of 0.84.

CONCLUSIONS

The study showed that the analysis of internal changes in lesion parametric maps leads to a better classification of breast tumors. The authors recommend combining multiple features for characterization, instead of using only one parameter, especially in the case of heterogeneous lesions.

Assessment of predictive performance in incomplete data by combining internal validation and multiple imputation

BACKGROUND

Missing values are a frequent issue in human studies. In many situations, multiple imputation (MI) is an appropriate missing data handling strategy, whereby missing values are imputed multiple times, the analysis is performed in every imputed data set, and the obtained estimates are pooled. If the aim is to estimate (added) predictive performance measures, such as (change in) the area under the receiver-operating characteristic curve (AUC), internal validation strategies become desirable in order to correct for optimism. It is not fully understood how internal validation should be combined with multiple imputation.

METHODS

In a comprehensive simulation study and in a real data set based on blood markers as predictors for mortality, we compare three combination strategies: Val-MI, internal validation followed by MI on the training and test parts separately, MI-Val, MI on the full data set followed by internal validation, and MI(-y)-Val, MI on the full data set omitting the outcome followed by internal validation. Different validation strategies, including bootstrap und cross-validation, different (added) performance measures, and various data characteristics are considered, and the strategies are evaluated with regard to bias and mean squared error of the obtained performance estimates. In addition, we elaborate on the number of resamples and imputations to be used, and adopt a strategy for confidence interval construction to incomplete data.

RESULTS

Internal validation is essential in order to avoid optimism, with the bootstrap 0.632+ estimate representing a reliable method to correct for optimism. While estimates obtained by MI-Val are optimistically biased, those obtained by MI(-y)-Val tend to be pessimistic in the presence of a true underlying effect. Val-MI provides largely unbiased estimates, with a slight pessimistic bias with increasing true effect size, number of covariates and decreasing sample size. In Val-MI, accuracy of the estimate is more strongly improved by increasing the number of bootstrap draws rather than the number of imputations. With a simple integrated approach, valid confidence intervals for performance estimates can be obtained.

CONCLUSIONS

When prognostic models are developed on incomplete data, Val-MI represents a valid strategy to obtain estimates of predictive performance measures.

Perspectives on making big data analytics work for oncology

Oncology, with its unique combination of clinical, physical, technological, and biological data provides an ideal case study for applying big data analytics to improve cancer treatment safety and outcomes. An oncology treatment course such as chemoradiotherapy can generate a large pool of information carrying the 5Vs hallmarks of big data. This data is comprised of a heterogeneous mixture of patient demographics, radiation/chemo dosimetry, multimodality imaging features, and biological markers generated over a treatment period that can span few days to several weeks. Efforts using commercial and in-house tools are underway to facilitate data aggregation, ontology creation, sharing, visualization and varying analytics in a secure environment. However, open questions related to proper data structure representation and effective analytics tools to support oncology decision-making need to be addressed. It is recognized that oncology data constitutes a mix of structured (tabulated) and unstructured (electronic documents) that need to be processed to facilitate searching and subsequent knowledge discovery from relational or NoSQL databases. In this context, methods based on advanced analytics and image feature extraction for oncology applications will be discussed. On the other hand, the classical p (variables)≫n (samples) inference problem of statistical learning is challenged in the Big data realm and this is particularly true for oncology applications where p-omics is witnessing exponential growth while the number of cancer incidences has generally plateaued over the past 5-years leading to a quasi-linear growth in samples per patient. Within the Big data paradigm, this kind of phenomenon may yield undesirable effects such as echo chamber anomalies, Yule-Simpson reversal paradox, or misleading ghost analytics. In this work, we will present these effects as they pertain to oncology and engage small thinking methodologies to counter these effects ranging from incorporating prior knowledge, using information-theoretic techniques to modern ensemble machine learning approaches or combination of these. We will particularly discuss the pros and cons of different approaches to improve mining of big data in oncology.

Bayesian network ensemble as a multivariate strategy to predict radiation pneumonitis risk

PURPOSE

Prediction of radiation pneumonitis (RP) has been shown to be challenging due to the involvement of a variety of factors including dose-volume metrics and radiosensitivity biomarkers. Some of these factors are highly correlated and might affect prediction results when combined. Bayesian network (BN) provides a probabilistic framework to represent variable dependencies in a directed acyclic graph. The aim of this study is to integrate the BN framework and a systems' biology approach to detect possible interactions among RP risk factors and exploit these relationships to enhance both the understanding and prediction of RP.

METHODS

The authors studied 54 nonsmall-cell lung cancer patients who received curative 3D-conformal radiotherapy. Nineteen RP events were observed (common toxicity criteria for adverse events grade 2 or higher). Serum concentration of the following four candidate biomarkers were measured at baseline and midtreatment: alpha-2-macroglobulin, angiotensin converting enzyme (ACE), transforming growth factor, interleukin-6. Dose-volumetric and clinical parameters were also included as covariates. Feature selection was performed using a Markov blanket approach based on the Koller-Sahami filter. The Markov chain Monte Carlo technique estimated the posterior distribution of BN graphs built from the observed data of the selected variables and causality constraints. RP probability was estimated using a limited number of high posterior graphs (ensemble) and was averaged for the final RP estimate using Bayes' rule. A resampling method based on bootstrapping was applied to model training and validation in order to control under- and overfit pitfalls.

RESULTS

RP prediction power of the BN ensemble approach reached its optimum at a size of 200. The optimized performance of the BN model recorded an area under the receiver operating characteristic curve (AUC) of 0.83, which was significantly higher than multivariate logistic regression (0.77), mean heart dose (0.69), and a pre-to-midtreatment change in ACE (0.66). When RP prediction was made only with pretreatment information, the AUC ranged from 0.76 to 0.81 depending on the ensemble size. Bootstrap validation of graph features in the ensemble quantified confidence of association between variables in the graphs where ten interactions were statistically significant.

CONCLUSIONS

The presented BN methodology provides the flexibility to model hierarchical interactions between RP covariates, which is applied to probabilistic inference on RP. The authors' preliminary results demonstrate that such framework combined with an ensemble method can possibly improve prediction of RP under real-life clinical circumstances such as missing data or treatment plan adaptation.

A radiomics model from joint FDG-PET and MRI texture features for the prediction of lung metastases in soft-tissue sarcomas of the extremities

This study aims at developing a joint FDG-PET and MRI texture-based model for the early evaluation of lung metastasis risk in soft-tissue sarcomas (STSs). We investigate if the creation of new composite textures from the combination of FDG-PET and MR imaging information could better identify aggressive tumours. Towards this goal, a cohort of 51 patients with histologically proven STSs of the extremities was retrospectively evaluated. All patients had pre-treatment FDG-PET and MRI scans comprised of T1-weighted and T2-weighted fat-suppression sequences (T2FS). Nine non-texture features (SUV metrics and shape features) and forty-one texture features were extracted from the tumour region of separate (FDG-PET, T1 and T2FS) and fused (FDG-PET/T1 and FDG-PET/T2FS) scans. Volume fusion of the FDG-PET and MRI scans was implemented using the wavelet transform. The influence of six different extraction parameters on the predictive value of textures was investigated. The incorporation of features into multivariable models was performed using logistic regression. The multivariable modeling strategy involved imbalance-adjusted bootstrap resampling in the following four steps leading to final prediction model construction: (1) feature set reduction; (2) feature selection; (3) prediction performance estimation; and (4) computation of model coefficients. Univariate analysis showed that the isotropic voxel size at which texture features were extracted had the most impact on predictive value. In multivariable analysis, texture features extracted from fused scans significantly outperformed those from separate scans in terms of lung metastases prediction estimates. The best performance was obtained using a combination of four texture features extracted from FDG-PET/T1 and FDG-PET/T2FS scans. This model reached an area under the receiver-operating characteristic curve of 0.984 ± 0.002, a sensitivity of 0.955 ± 0.006, and a specificity of 0.926 ± 0.004 in bootstrapping evaluations. Ultimately, lung metastasis risk assessment at diagnosis of STSs could improve patient outcomes by allowing better treatment adaptation.

Robust phase-based texture descriptor for classification of breast ultrasound images

BACKGROUND

Classification of breast ultrasound (BUS) images is an important step in the computer-aided diagnosis (CAD) system for breast cancer. In this paper, a novel phase-based texture descriptor is proposed for efficient and robust classifiers to discriminate benign and malignant tumors in BUS images.

METHOD

The proposed descriptor, namely the phased congruency-based binary pattern (PCBP) is an oriented local texture descriptor that combines the phase congruency (PC) approach with the local binary pattern (LBP). The support vector machine (SVM) is further applied for the tumor classification. To verify the efficiency of the proposed PCBP texture descriptor, we compare the PCBP with other three state-of-art texture descriptors, and experiments are carried out on a BUS image database including 138 cases. The receiver operating characteristic (ROC) analysis is firstly performed and seven criteria are utilized to evaluate the classification performance using different texture descriptors. Then, in order to verify the robustness of the PCBP against illumination variations, we train the SVM classifier on texture features obtained from the original BUS images, and use this classifier to deal with the texture features extracted from BUS images with different illumination conditions (i.e., contrast-improved, gamma-corrected and histogram-equalized). The area under ROC curve (AUC) index is used as the figure of merit to evaluate the classification performances.

RESULTS AND CONCLUSIONS

The proposed PCBP texture descriptor achieves the highest values (i.e. 0.894) and the least variations in respect of the AUC index, regardless of the gray-scale variations. It's revealed in the experimental results that classifications of BUS images with the proposed PCBP texture descriptor are efficient and robust, which may be potentially useful for breast ultrasound CADs.

Support vector machine model for diagnosing pneumoconiosis based on wavelet texture features of digital chest radiographs

This study aims to explore the classification ability of decision trees (DTs) and support vector machines (SVMs) to discriminate between the digital chest radiographs (DRs) of pneumoconiosis patients and control subjects. Twenty-eight wavelet-based energy texture features were calculated at the lung fields on DRs of 85 healthy controls and 40 patients with stage I and stage II pneumoconiosis. DTs with algorithm C5.0 and SVMs with four different kernels were trained by samples with two combinations of the texture features to classify a DR as of a healthy subject or of a patient with pneumoconiosis. All of the models were developed with fivefold cross-validation, and the final performances of each model were compared by the area under receiver operating characteristic (ROC) curve. For both SVM (with a radial basis function kernel) and DT (with algorithm C5.0), areas under ROC curves (AUCs) were 0.94 ± 0.02 and 0.86 ± 0.04 (P = 0.02) when using the full feature set and 0.95 ± 0.02 and 0.88 ± 0.04 (P = 0.05) when using the selected feature set, respectively. When built on the selected texture features, the SVM with a polynomial kernel showed a higher diagnostic performance with an AUC value of 0.97 ± 0.02 than SVMs with a linear kernel, a radial basis function kernel and a sigmoid kernel with AUC values of 0.96 ± 0.02 (P = 0.37), 0.95 ± 0.02 (P = 0.24), and 0.90 ± 0.03 (P = 0.01), respectively. The SVM model with a polynomial kernel built on the selected feature set showed the highest diagnostic performance among all tested models when using either all the wavelet texture features or the selected ones. The model has a good potential in diagnosing pneumoconiosis based on digital chest radiographs.

The development and evaluation of a computerized diagnosis scheme for pneumoconiosis on digital chest radiographs

Purpose

To diagnose pneumoconiosis using a computer-aided diagnosis system based on digital chest radiographs.

Methods

Lung fields were first extracted by combining the traditional Otsu-threshold method with a morphological reconstruction on digital radiographs (DRs), and then subdivided into six non-overlapping regions (region (a-f)). Twenty-two wavelet-based energy texture features were calculated exclusively from each region and selected using a decision tree algorithm. A support vector machine (SVM) with a linear kernel was trained using samples with texture features to classify an individual region of a healthy subject or a pneumoconiosis patient. The final classification results were obtained by integrating these individual classifiers with the weighted voting method. All models were developed on a dataset of 85 healthy controls and 40 stage I or II pneumoconiosis patients and validated by using the bootstrap resampling with replacement method.

Results

The areas under receiver operating characteristic curves (AUCs) of regions (c) and (f) were 0.688 and 0.563, which were worse than those of the other four regions. Region (c) and (f) were both excluded from the individual classifiers that were going to be assembled further. When built on the selected texture features, each individual SVM showed a higher diagnostic performance for the training set and the test set. The classification performance after an ensemble was 0.997 and 0.961 of the AUC value for the training and test sets, respectively. The final results were 0.974 ± 0.018 for AUC value and 0.929 ± 0.018 for accuracy.

Conclusion

The integrated SVM model built on the selected feature set showed the highest diagnostic performance among all individual SVM models. The model has good potential in diagnosing pneumoconiosis based on digital chest radiographs.

Robust Texture Analysis Using Multi-Resolution Gray-Scale Invariant Features for Breast Sonographic Tumor Diagnosis

Computer-aided diagnosis (CAD) systems in gray-scale breast ultrasound images have the potential to reduce unnecessary biopsy of breast masses. The purpose of our study is to develop a robust CAD system based on the texture analysis. First, gray-scale invariant features are extracted from ultrasound images via multi-resolution ranklet transform. Thus, one can apply linear support vector machines (SVMs) on the resulting gray-level co-occurrence matrix (GLCM)-based texture features for discriminating the benign and malignant masses. To verify the effectiveness and robustness of the proposed texture analysis, breast ultrasound images obtained from three different platforms are evaluated based on cross-platform training/testing and leave-one-out cross-validation (LOO-CV) schemes. We compare our proposed features with those extracted by wavelet transform in terms of receiver operating characteristic (ROC) analysis. The AUC values derived from the area under the curve for the three databases via ranklet transform are 0.918 (95% confidence interval [CI], 0.848 to 0.961), 0.943 (95% CI, 0.906 to 0.968), and 0.934 (95% CI, 0.883 to 0.961), respectively, while those via wavelet transform are 0.847 (95% CI, 0.762 to 0.910), 0.922 (95% CI, 0.878 to 0.958), and 0.867 (95% CI, 0.798 to 0.914), respectively. Experiments with cross-platform training/testing scheme between each database reveal that the diagnostic performance of our texture analysis using ranklet transform is less sensitive to the sonographic ultrasound platforms. Also, we adopt several co-occurrence statistics in terms of quantization levels and orientations (i.e., descriptor settings) for computing the co-occurrence matrices with 0.632+ bootstrap estimators to verify the use of the proposed texture analysis. These experiments suggest that the texture analysis using multi-resolution gray-scale invariant features via ranklet transform is useful for designing a robust CAD system.

Computer-aided staging of lymphoma patients with FDG PET/CT imaging based on textural information

We have designed a computer-aided diagnosis system to discriminate between hypermetabolic cancer lesions and hypermetabolic inflammatory or physiological but noncancerous processes in FDG PET/CT exams of lymphoma patients. Detection performance of the support vector machine (SVM) classifier was assessed based on feature sets including 105 positron emission tomography (PET) and Computed tomography (CT) characteristics derived from the clinical practice and from more sophisticated texture analysis. An original feature selection method based on combining different filter methods was proposed. The evaluation database consisted of 156 lymphomatous and 32 suspicious but nonlymphomatous regions of interest. Different types of training databases including either the PET and CT features or the PET features only, with or without feature selection, were evaluated to assess the added value of multimodality and texture information on classification performance. An optimization study was conducted for each classifier separately to select the best combination of parameters. Promising classification performance was achieved by the SVM classifier combined with the 12 most discriminant PET and CT features with a value of the area under the receiver operating curve of 0.91.

On the assessment of the added value of new predictive biomarkers

BACKGROUND

The surge in biomarker development calls for research on statistical evaluation methodology to rigorously assess emerging biomarkers and classification models. Recently, several authors reported the puzzling observation that, in assessing the added value of new biomarkers to existing ones in a logistic regression model, statistical significance of new predictor variables does not necessarily translate into a statistically significant increase in the area under the ROC curve (AUC). Vickers et al. concluded that this inconsistency is because AUC "has vastly inferior statistical properties," i.e., it is extremely conservative. This statement is based on simulations that misuse the DeLong et al. method. Our purpose is to provide a fair comparison of the likelihood ratio (LR) test and the Wald test versus diagnostic accuracy (AUC) tests.

DISCUSSION

We present a test to compare ideal AUCs of nested linear discriminant functions via an F test. We compare it with the LR test and the Wald test for the logistic regression model. The null hypotheses of these three tests are equivalent; however, the F test is an exact test whereas the LR test and the Wald test are asymptotic tests. Our simulation shows that the F test has the nominal type I error even with a small sample size. Our results also indicate that the LR test and the Wald test have inflated type I errors when the sample size is small, while the type I error converges to the nominal value asymptotically with increasing sample size as expected. We further show that the DeLong et al. method tests a different hypothesis and has the nominal type I error when it is used within its designed scope. Finally, we summarize the pros and cons of all four methods we consider in this paper.

SUMMARY

We show that there is nothing inherently less powerful or disagreeable about ROC analysis for showing the usefulness of new biomarkers or characterizing the performance of classification models. Each statistical method for assessing biomarkers and classification models has its own strengths and weaknesses. Investigators need to choose methods based on the assessment purpose, the biomarker development phase at which the assessment is being performed, the available patient data, and the validity of assumptions behind the methodologies.

Machine Learning in Computer-aided Diagnosis of the Thorax and Colon in CT: A Survey

Computer-aided detection (CADe) and diagnosis (CAD) has been a rapidly growing, active area of research in medical imaging. Machine leaning (ML) plays an essential role in CAD, because objects such as lesions and organs may not be represented accurately by a simple equation; thus, medical pattern recognition essentially require "learning from examples." One of the most popular uses of ML is the classification of objects such as lesion candidates into certain classes (e.g., abnormal or normal, and lesions or non-lesions) based on input features (e.g., contrast and area) obtained from segmented lesion candidates. The task of ML is to determine "optimal" boundaries for separating classes in the multidimensional feature space which is formed by the input features. ML algorithms for classification include linear discriminant analysis (LDA), quadratic discriminant analysis (QDA), multilayer perceptrons, and support vector machines (SVM). Recently, pixel/voxel-based ML (PML) emerged in medical image processing/analysis, which uses pixel/voxel values in images directly, instead of features calculated from segmented lesions, as input information; thus, feature calculation or segmentation is not required. In this paper, ML techniques used in CAD schemes for detection and diagnosis of lung nodules in thoracic CT and for detection of polyps in CT colonography (CTC) are surveyed and reviewed.

Computer-aided diagnosis systems for lung cancer: challenges and methodologies

This paper overviews one of the most important, interesting, and challenging problems in oncology, the problem of lung cancer diagnosis. Developing an effective computer-aided diagnosis (CAD) system for lung cancer is of great clinical importance and can increase the patient's chance of survival. For this reason, CAD systems for lung cancer have been investigated in a huge number of research studies. A typical CAD system for lung cancer diagnosis is composed of four main processing steps: segmentation of the lung fields, detection of nodules inside the lung fields, segmentation of the detected nodules, and diagnosis of the nodules as benign or malignant. This paper overviews the current state-of-the-art techniques that have been developed to implement each of these CAD processing steps. For each technique, various aspects of technical issues, implemented methodologies, training and testing databases, and validation methods, as well as achieved performances, are described. In addition, the paper addresses several challenges that researchers face in each implementation step and outlines the strengths and drawbacks of the existing approaches for lung cancer CAD systems.

A review of computer-aided diagnosis in thoracic and colonic imaging

Medical imaging has been indispensable in medicine since the discovery of x-rays. Medical imaging offers useful information on patients' medical conditions and on the causes of their symptoms and diseases. As imaging technologies advance, a large number of medical images are produced which physicians/radiologists must interpret. Thus, computer aids are demanded and become indispensable in physicians' decision making based on medical images. Consequently, computer-aided detection and diagnosis (CAD) has been investigated and has been an active research area in medical imaging. CAD is defined as detection and/or diagnosis made by a radiologist/physician who takes into account the computer output as a "second opinion". In CAD research, detection and diagnosis of lung and colorectal cancer in thoracic and colonic imaging constitute major areas, because lung and colorectal cancers are the leading and second leading causes, respectively, of cancer deaths in the U.S. and also in other countries. In this review, CAD of the thorax and colon, including CAD for detection and diagnosis of lung nodules in thoracic CT, and that for detection of polyps in CT colonography, are reviewed.