Accounting for reconstruction kernel-induced variability in CT radiomic features using noise power spectra

Rank acquisition impact on radiomics estimation (AсquIRE) in chest CT imaging: A retrospective multi-site, multi-use-case study

BACKGROUND

Radiomics is a method within medical image analysis that involves the extraction of quantitative data from radiologic scans, often in conjunction with machine learning algorithms to phenotype disease appearance, prognosticate disease outcome, and predict treatment response. However, variance in CT scanner acquisition parameters, such as convolution kernels or pixel spacing, can impact radiomics texture feature values.

PURPOSE

The extent to which the parameters influence radiomics features continues to be an active area of investigation. In this study, we describe a novel approach, Acquisition Impact on Radiomics Estimation (AcquIRE), to rank the impact of CT acquisition parameters on radiomic texture features.

METHODS

In this work, we used three chest CT imaging datasets (n = 749 patients) from nine sites comprising: i) lung granulomas and adenocarcinomas (D1) (10 and 52 patients, respectively); ii) minimal and frank invasive adenocarcinoma (D2) (74 and 145 patients); and iii) early-stage NSCLC patients (D3) (315 patients). Datasets D2 and D3 were collected from four sites each, and D1 from a single site. For each patient, 744 texture features and nine acquisition parameters were extracted and utilized to evaluate which parameters impact radiomic features the most. The AcquIRE method establishes a relative assessment between acquisition parameters and radiomic texture featuresa through the creation of a classification model, which is then utilized to assess the rank of the acquisition parameters.

RESULTS

Across the use cases, CT software version and convolution kernel parameters were found to have the most variance. In D1, it was observed that the Haralick texture feature family was the least affected by variations in acquisition parameters, while the Gabor feature family was the most impacted. However, in datasets D2 and D3, the Gabor features were found to be the least affected. Our findings suggest that the impact on radiomic parameters is as much a function of the problem in question as it is acquisition parameters.

CONCLUSIONS

The software version and convolution kernel parameters impacted the radiomics feature the most.

The impact of inter-observer variation in delineation on robustness of radiomics features in non-small cell lung cancer

Artificial intelligence and radiomics have the potential to revolutionise cancer prognostication and personalised treatment. Manual outlining of the tumour volume for extraction of radiomics features (RF) is a subjective process. This study investigates robustness of RF to inter-observer variation (IOV) in contouring in lung cancer. We utilised two public imaging datasets: ‘NSCLC-Radiomics’ and ‘NSCLC-Radiomics-Interobserver1’ (‘Interobserver’). For ‘NSCLC-Radiomics’, we created an additional set of manual contours for 92 patients, and for ‘Interobserver’, there were five manual and five semi-automated contours available for 20 patients. Dice coefficients (DC) were calculated for contours. 1113 RF were extracted including shape, first order and texture features. Intraclass correlation coefficient (ICC) was computed to assess robustness of RF to IOV. Cox regression analysis for overall survival (OS) was performed with a previously published radiomics signature. The median DC ranged from 0.81 (‘NSCLC-Radiomics’) to 0.85 (‘Interobserver’—semi-automated). The median ICC for the ‘NSCLC-Radiomics’, ‘Interobserver’ (manual) and ‘Interobserver’ (semi-automated) were 0.90, 0.88 and 0.93 respectively. The ICC varied by feature type and was lower for first order and gray level co-occurrence matrix (GLCM) features. Shape features had a lower median ICC in the ‘NSCLC-Radiomics’ dataset compared to the ‘Interobserver’ dataset. Survival analysis showed similar separation of curves for three of four RF apart from ‘original_shape_Compactness2’, a feature with low ICC (0.61). The majority of RF are robust to IOV, with first order, GLCM and shape features being the least robust. Semi-automated contouring improves feature stability. Decreased robustness of a feature is significant as it may impact upon the features’ prognostic capability.

Radiomics of Patients with Locally Advanced Rectal Cancer: Effect of Preprocessing on Features Estimation from Computed Tomography Imaging

The purpose of this study was to investigate the effect of image preprocessing on radiomic features estimation from computed tomography (CT) imaging of locally advanced rectal cancer (LARC). CT images of 20 patients with LARC were used to estimate 105 radiomic features of 7 classes (shape, first-order, GLCM, GLDM, GLRLM, GLSZM, and NGTDM). Radiomic features were estimated for 6 different isotropic resampling voxel sizes, using 10 interpolation algorithms (at fixed bin width) and 6 different bin widths (at fixed interpolation algorithm). The intraclass correlation coefficient (ICC) and the coefficient of variation (CV) were calculated to assess the variability in radiomic features estimation due to preprocessing. A repeated measures correlation analysis was performed to assess any linear correlation between radiomic feature estimate and resampling voxel size or bin width. Reproducibility of radiomic feature estimate, when assessed through ICC analysis, was nominally excellent (ICC > 0.9) for shape features, good (0.75 < ICC ≤ 0.9) or moderate (0.5 < ICC ≤ 0.75) for first-order features, and moderate or poor (0 ≤ ICC ≤ 0.5) for textural features. A number of radiomic features characterized by good or excellent reproducibility in terms of ICC showed however median CV values greater than 15%. For most textural features, a significant (p < 0.05) correlation between their estimate and resampling voxel size or bin width was found. In CT imaging of patients with LARC, the estimate of textural features, as well as of first-order features to a lesser extent, is appreciably biased by preprocessing. Accordingly, this should be taken into account when planning clinical or research studies, as well as when comparing results from different studies and performing multicenter studies.

Deep learning for lung disease segmentation on CT: Which reconstruction kernel should be used?

PURPOSE

The purpose of this study was to determine whether a single reconstruction kernel or both high and low frequency kernels should be used for training deep learning models for the segmentation of diffuse lung disease on chest computed tomography (CT).

MATERIALS AND METHODS

Two annotated datasets of COVID-19 pneumonia (323,960 slices) and interstitial lung disease (ILD) (4,284 slices) were used. Annotated CT images were used to train a U-Net architecture to segment disease. All CT slices were reconstructed using both a lung kernel (LK) and a mediastinal kernel (MK). Three different trainings, resulting in three different models were compared for each disease: training on LK only, MK only or LK+MK images. Dice similarity scores (DSC) were compared using the Wilcoxon signed-rank test.

RESULTS

Models only trained on LK images performed better on LK images than on MK images (median DSC = 0.62 [interquartile range (IQR): 0.54, 0.69] vs. 0.60 [IQR: 0.50, 0.70], P < 0.001 for COVID-19 and median DSC = 0.62 [IQR: 0.56, 0.69] vs. 0.50 [IQR 0.43, 0.57], P < 0.001 for ILD). Similarly, models only trained on MK images performed better on MK images (median DSC = 0.62 [IQR: 0.53, 0.68] vs. 0.54 [IQR: 0.47, 0.63], P < 0.001 for COVID-19 and 0.69 [IQR: 0.61, 0.73] vs. 0.63 [IQR: 0.53, 0.70], P < 0.001 for ILD). Models trained on both kernels performed better or similarly than those trained on only one kernel. For COVID-19, median DSC was 0.67 (IQR: =0.59, 0.73) when applied on LK images and 0.67 (IQR: 0.60, 0.74) when applied on MK images (P < 0.001 for both). For ILD, median DSC was 0.69 (IQR: 0.63, 0.73) when applied on LK images (P = 0.006) and 0.68 (IQR: 0.62, 0.72) when applied on MK images (P > 0.99).

CONCLUSION

Reconstruction kernels impact the performance of deep learning-based models for lung disease segmentation. Training on both LK and MK images improves the performance.

Deep Feature Stability Analysis Using CT Images of a Physical Phantom Across Scanner Manufacturers, Cartridges, Pixel Sizes, and Slice Thickness

Image acquisition parameters for computed tomography scans such as slice thickness and field of view may vary depending on tumor size and site. Recent studies have shown that some radiomics features were dependent on voxel size (= pixel size × slice thickness), and with proper normalization, this voxel size dependency could be reduced. Deep features from a convolutional neural network (CNN) have shown great promise in characterizing cancers. However, how do these deep features vary with changes in imaging acquisition parameters? To analyze the variability of deep features, a physical radiomics phantom with 10 different material cartridges was scanned on 8 different scanners. We assessed scans from 3 different cartridges (rubber, dense cork, and normal cork). Deep features from the penultimate layer of the CNN before (pre-rectified linear unit) and after (post-rectified linear unit) applying the rectified linear unit activation function were extracted from a pre-trained CNN using transfer learning. We studied both the interscanner and intrascanner dependency of deep features and also the deep features' dependency over the 3 cartridges. We found some deep features were dependent on pixel size and that, with appropriate normalization, this dependency could be reduced. False discovery rate was applied for multiple comparisons, to mitigate potentially optimistic results. We also used stable deep features for prognostic analysis on 1 non-small cell lung cancer data set.

Radiogenomics in brain, breast, and lung cancer: opportunities and challenges

The field of radiogenomics largely focuses on developing imaging surrogates for genomic signatures and integrating imaging, genomic, and molecular data to develop combined personalized biomarkers for characterizing various diseases. Our study aims to highlight the current state-of-the-art and the role of radiogenomics in cancer research, focusing mainly on solid tumors, and is broadly divided into four sections. The first section reviews representative studies that establish the biologic basis of radiomic signatures using gene expression and molecular profiling information. The second section includes studies that aim to non-invasively predict molecular subtypes of tumors using radiomic signatures. The third section reviews studies that evaluate the potential to augment the performance of established prognostic signatures by combining complementary information encoded by radiomic and genomic signatures derived from cancer tumors. The fourth section includes studies that focus on ascertaining the biological significance of radiomic phenotypes. We conclude by discussing current challenges and opportunities in the field, such as the importance of coordination between imaging device manufacturers, regulatory organizations, health care providers, pharmaceutical companies, academic institutions, and physicians for the effective standardization of the results from radiogenomic signatures and for the potential use of these findings to improve precision care for cancer patients.

Reproducibility of lung nodule radiomic features: Multivariable and univariable investigations that account for interactions between CT acquisition and reconstruction parameters

PURPOSE

Recent studies have demonstrated a lack of reproducibility of radiomic features in response to variations in CT parameters. In addition, reproducibility of radiomic features has not been well established in clinical datasets. We aimed to investigate the effects of a wide range of CT acquisition and reconstruction parameters on radiomic features in a realistic setting using clinical low dose lung cancer screening cases. We performed univariable and multivariable explorations to consider the effects of individual parameters and the simultaneous interactions between three different acquisition/reconstruction parameters of radiation dose level, reconstructed slice thickness, and kernel.

METHOD

A cohort of 89 lung cancer screening patients were collected that each had a solid lung nodule >4mm diameter. A computational pipeline was used to perform a simulation of dose reduction of the raw projection data, collected from patient scans. This was followed by reconstruction of raw data with weighted filter back projection (wFBP) algorithm and automatic lung nodule detection and segmentation using a computer-aided detection tool. For each patient, 36 different image datasets were created corresponding to dose levels of 100%, 50%, 25%, and 10% of the original dose level, three slice thicknesses of 0.6 mm, 1 mm, and 2 mm, as well as three reconstruction kernels of smooth, medium, and sharp. For each nodule, 226 well-known radiomic features were calculated at each image condition. The reproducibility of radiomic features was first evaluated by measuring the intercondition agreement of the feature values among the 36 image conditions. Then in a series of univariable analyses, the impact of individual CT parameters was assessed by selecting subsets of conditions with one varying and two constant CT parameters. In each subset, intraparameter agreements were assessed. Overall concordance correlation coefficient (OCCC) served as the measure of agreement. An OCCC ≥ 0.9 implied strong agreement and reproducibility of radiomic features in intercondition or intraparameter comparisons. Furthermore, the interaction of CT parameters in impacting radiomic feature values was investigated via ANOVA.

RESULTS

All included radiomic features lacked intercondition reproducibility (OCCC < 0.9) among all the 36 conditions. Out of 226 radiomic features analyzed, only 17 and 18 features were considered reproducible (OCCC ≥ 0.9) to dose and kernel variation, respectively, within the corresponding condition subsets. Slice thickness demonstrated the largest impact on radiomic feature values where only one to five features were reproducible at a few condition subsets. ANOVA revealed significant interactions (P < 0.05) between CT parameters affecting the variability of >50% of radiomic features.

CONCLUSION

We systematically explored the multidimensional space of CT parameters in affecting lung nodule radiomic features. Univariable and multivariable analyses of this study not only showed the lack of reproducibility of the majority of radiomic features but also revealed existing interactions among CT parameters, meaning that the effect of individual CT parameters on radiomic features can be conditional upon other CT acquisition and reconstruction parameters. Our findings advise on careful radiomic feature selection and attention to the inclusion criteria for CT image acquisition protocols within the datasets of radiomic studies.

Prediction of Human Papillomavirus (HPV) Association of Oropharyngeal Cancer (OPC) Using Radiomics: The Impact of the Variation of CT Scanner

Simple Summary

Recent studies exploring the application of radiomics features in medicine have shown promising results. However, variation in imaging parameters may impact the robustness of these features. Feature robustness may then in turn affect the prediction performance of the machine learning models built upon these features. While numerous studies have tested feature robustness against a variety of imaging parameters, the extent to which feature robustness affects predictions remains unclear. A particularly notable application of radiomics in clinical oncology is the prediction of Human Papillomavirus (HPV) association in Oropharyngeal cancer. In this study we explore how CT scanner type affects the performance of radiomics features for HPV association prediction and highlight the need to implement precautionary approaches so as to minimize this effect.

Abstract

Studies have shown that radiomic features are sensitive to the variability of imaging parameters (e.g., scanner models), and one of the major challenges in these studies lies in improving the robustness of quantitative features against the variations in imaging datasets from multi-center studies. Here, we assess the impact of scanner choice on computed tomography (CT)-derived radiomic features to predict the association of oropharyngeal squamous cell carcinoma with human papillomavirus (HPV). This experiment was performed on CT image datasets acquired from two different scanner manufacturers. We demonstrate strong scanner dependency by developing a machine learning model to classify HPV status from radiological images. These experiments reveal the effect of scanner manufacturer on the robustness of radiomic features, and the extent of this dependency is reflected in the performance of HPV prediction models. The results of this study highlight the importance of implementing an appropriate approach to reducing the impact of imaging parameters on radiomic features and consequently on the machine learning models, without removing features which are deemed non-robust but may contain learning information.

Distinguishing granulomas from adenocarcinomas by integrating stable and discriminating radiomic features on non-contrast computed tomography scans

OBJECTIVE

To identify stable and discriminating radiomic features on non-contrast CT scans to develop more generalisable radiomic classifiers for distinguishing granulomas from adenocarcinomas.

METHODS

In total, 412 patients with adenocarcinomas and granulomas from three institutions were retrospectively included. Segmentations of the lung nodules were performed manually by an expert radiologist in a 2D axial view. Radiomic features were extracted from intra- and perinodular regions. A total of 145 patients were used as part of the training set (Str), whereas 205 patients were used as part of test set I (Ste1) and 62 patients were used as part of independent test set II (Ste2). To mitigate the variation of CT acquisition parameters, we defined 'stable' radiomic features as those for which the feature expression remains relatively unchanged between different sites, as assessed using a Wilcoxon rank-sum test. These stable features were used to develop more generalisable radiomic classifiers that were more resilient to variations in lung CT scans. Features were ranked based on two criteria, firstly based on discriminability (i.e. maximising AUC) alone and subsequently based on maximising both feature stability and discriminability. Different machine-learning classifiers (Linear discriminant analysis, Quadratic discriminant analysis, Support vector machines and random forest) were trained with features selected using the two different criteria and then compared on the two independent test sets for distinguishing granulomas from adenocarcinomas, in terms of area under the receiver operating characteristic curve.

RESULTS

In the test sets, classifiers constructed using the criteria involving maximising feature stability and discriminability simultaneously achieved higher AUC compared with the discriminating alone criteria (Ste1 [n = 205]: maximum AUCs of 0.85versus . 0.80; p-value = 0.047 and Ste2 [n = 62]: maximum AUCs of 0.87 versus. 0.79; p-value = 0.021). These differences held for features extracted from scans with <3 mm slice thickness (AUC = 0.88 versus. 0.80; p-value = 0.039, n = 100) and for the ≥3 mm cases (AUC = 0.81 versus. 0.76; p-value = 0.034, n = 105). In both experiments, shape and peritumoural texture features had a higher stability compared with intratumoural texture features.

CONCLUSIONS

Our study suggests that explicitly accounting for both stability and discriminability results in more generalisable radiomic classifiers to distinguish adenocarcinomas from granulomas on non-contrast CT scans. Our results also showed that peritumoural texture and shape features were less affected by the scanner parameters compared with intratumoural texture features; however, they were also less discriminating compared with intratumoural features.

Understanding Sources of Variation to Improve the Reproducibility of Radiomics

Radiomics is the method of choice for investigating the association between cancer imaging phenotype, cancer genotype and clinical outcome prediction in the era of precision medicine. The fast dispersal of this new methodology has benefited from the existing advances of the core technologies involved in radiomics workflow: image acquisition, tumor segmentation, feature extraction and machine learning. However, despite the rapidly increasing body of publications, there is no real clinical use of a developed radiomics signature so far. Reasons are multifaceted. One of the major challenges is the lack of reproducibility and generalizability of the reported radiomics signatures (features and models). Sources of variation exist in each step of the workflow; some are controllable or can be controlled to certain degrees, while others are uncontrollable or even unknown. Insufficient transparency in reporting radiomics studies further prevents translation of the developed radiomics signatures from the bench to the bedside. This review article first addresses sources of variation, which is illustrated using demonstrative examples. Then, it reviews a number of published studies and progresses made to date in the investigation and improvement of feature reproducibility and model performance. Lastly, it discusses potential strategies and practical considerations to reduce feature variability and improve the quality of radiomics study. This review focuses on CT image acquisition, tumor segmentation, quantitative feature extraction, and the disease of lung cancer.

Standardization of histogram- and GLCM-based radiomics in the presence of blur and noise

Radiomics have been extensively investigated as quantitative biomarkers that can enhance the utility of imaging studies and aid the clinical decision making process. A major challenge to the clinical translation of radiomics is their variability as a result of different imaging and reconstruction protocols. In this work, we present a novel radiomics standardization framework capable of modeling and recovering the underlying radiomic feature in images that have been corrupted by the effects of spatial resolution and noise. We focus on two classes of radiomics based on pixel value distributions - i.e., histograms and gray-level co-occurrence matrices. We developed a model that predicts these distributions in the presence of system blur and noise, and used that model to invert these physical effects and recover the underlying distributions. Specifically, the effect of blur on histogram and GLCM is highly image-dependent, while additive noise convolves the histogram/GLCM of the noiseless image with those of the noise. The recovery method therefore consists of two deconvolution operations: the first in the image domain to remove the effect of system blur, the second in the histogram/GLCM domain to remove the effect of noise. The performance of the proposed recovery strategy was investigated using a set of texture phantoms and an emulated CT imaging chain with a range of realistic blur and noise levels. The proposed method was able to obtain histogram and GLCM estimates that closely resemble the ground truth. The method performed well across imaging conditions and significantly lowered the variability associated with different imaging protocols. This improvement also translated to better classification accuracy, where recovered radiomic values results in greater separation of radiomic clusters for two different texture phantoms as compared to values derived from the original blurred and noisy images. In summary, the novel radiomics standardization framework demonstrates high potential for mitigating radiomic variability as a result of the imaging system and can potentially be integrated as a preprocessing step towards more robust and reproducible radiomic models.

Impact of CT convolution kernel on robustness of radiomic features for different lung diseases and tissue types

OBJECTIVES

In this study, we aimed to assess the impact of different CT reconstruction kernels on the stability of radiomic features and the transferability between different diseases and tissue types. Three lung diseases were evaluated, i.e. non-small cell lung cancer (NSCLC), malignant pleural mesothelioma (MPM) and interstitial lung disease related to systemic sclerosis (SSc-ILD) as well as four different tissue types, i.e. primary tumor, largest involved lymph node ipsilateral and contralateral lung.

METHODS

Pre-treatment non-contrast enhanced CT scans from 23 NSCLC, 10 MPM and 12 SSc-ILD patients were collected retrospectively. For each patient, CT scans were reconstructed using smooth and sharp kernel in filtered back projection. The regions of interest (ROIs) were contoured on the smooth kernel-based CT and transferred to the sharp kernel-based CT. The voxels were resized to the largest voxel dimension of each cohort. In total, 1386 features were analyzed. Feature stability was assessed using the intraclass correlation coefficient. Features above the stability threshold >0.9 were considered stable.

RESULTS

We observed a strong impact of the reconstruction method on stability of the features (at maximum 26% of the 1386 features were stable). Intensity features were the most stable followed by texture and wavelet features. The wavelet features showed a positive correlation between percentage of stable features and size of the ROI (R2 = 0.79, p = 0.005). Lymph node radiomics showed poorest stability (<10%) and lung radiomics the largest stability (26%). Robustness analysis done on the contralateral lung could to a large extent be transferred to the ipsilateral lung, and the overlap of stable lung features between different lung diseases was more than 50%. However, results of robustness studies cannot be transferred between tissue types, which was investigated in NSCLC and MPM patients; the overlap of stable features for lymph node and lung, as well as for primary tumor and lymph node was very small in both disease types.

CONCLUSION

The robustness of radiomic features is strongly affected by different reconstruction kernels. The effect is largely influenced by the tissue type and less by the disease type.

ADVANCES IN KNOWLEDGE

The study presents to our knowledge the most complete analysis on the impact of convolution kernel on the robustness of CT-based radiomics for four relevant tissue types in three different lung diseases. .

Radiomics in medical imaging-"how-to" guide and critical reflection

Radiomics is a quantitative approach to medical imaging, which aims at enhancing the existing data available to clinicians by means of advanced mathematical analysis. Through mathematical extraction of the spatial distribution of signal intensities and pixel interrelationships, radiomics quantifies textural information by using analysis methods from the field of artificial intelligence. Various studies from different fields in imaging have been published so far, highlighting the potential of radiomics to enhance clinical decision-making. However, the field faces several important challenges, which are mainly caused by the various technical factors influencing the extracted radiomic features.The aim of the present review is twofold: first, we present the typical workflow of a radiomics analysis and deliver a practical "how-to" guide for a typical radiomics analysis. Second, we discuss the current limitations of radiomics, suggest potential improvements, and summarize relevant literature on the subject.

Radiomics as a personalized medicine tool in lung cancer: Separating the hope from the hype

Radiomics has become a popular image analysis method in the last few years. Its key hypothesis is that medical images harbor biological, prognostic and predictive information that is not revealed upon visual inspection. In contrast to previous work with a priori defined imaging biomarkers, radiomics instead calculates image features at scale and uses statistical methods to identify those most strongly associated to outcome. This builds on years of research into computer aided diagnosis and pattern recognition. While the potential of radiomics to aid personalized medicine is widely recognized, several technical limitations exist which hinder biomarker translation. Aspects of the radiomic workflow lack repeatability or reproducibility under particular circumstances, which is a key requirement for the translation of imaging biomarkers into clinical practice. One of the most commonly studied uses of radiomics is for personalized medicine applications in Non-Small Cell Lung Cancer (NSCLC). In this review, we summarize reported methodological limitations in CT based radiomic analyses together with suggested solutions. We then evaluate the current NSCLC radiomics literature to assess the risk associated with accepting the published conclusions with respect to these limitations. We review different complementary scoring systems and initiatives that can be used to critically appraise data from radiomics studies. Wider awareness should improve the quality of ongoing and future radiomics studies and advance their potential as clinically relevant biomarkers for personalized medicine in patients with NSCLC.

Stable and discriminating radiomic predictor of recurrence in early stage non-small cell lung cancer: Multi-site study

OBJECTIVES

To evaluate whether combining stability and discriminability criteria in building radiomic classifiers will improve the prognosis of cancer recurrence in early stage non-small cell lung cancer on non-contrast computer tomography (CT).

MATERIALS AND METHODS

CT scans of 610 patients with early stage (IA, IB, IIA) NSCLC from four independent cohorts were evaluated. A total of 350 patients from Cleveland Clinic Foundation and University of Pennsylvania were divided into two equal sets for training (D₁) and validation set (D₂). 80 patients from The Cancer Genome Atlas Lung Adenocarcinoma and Squamous Cell Carcinoma and 195 patients from The Cancer Imaging Archive, were used as independent second (D₃) and third (D₄) validation sets. A linear discriminant analysis (LDA) classifier was built based on the most stable and discriminate features. In addition, a radiomic risk score (RRS) was generated by using least absolute shrinkage and selection operator, Cox regression model to predict time to progression (TTP) following surgery.

RESULTS

A feature selection strategy focusing on both feature discriminability and stability resulted in the classifier having a higher discriminability on validation datasets compared to the discriminability alone criteria in discriminating cancer recurrence (D₂, AUC of 0.75 vs. 0.65; D₃, 0.74 vs. 0.62; D₄, 0.76 vs. 0.63). The RRS generated by most stable-discriminating features was significantly associated with TTP compared to discriminating alone criteria (HR = 1.66, C-index of 0.72 vs. HR = 1.04, C-index of 0.62).

CONCLUSION

Accounting for both stability and discriminability yielded a more generalizable classifier for predicting cancer recurrence and TTP in early stage NSCLC.

Learning from scanners: Bias reduction and feature correction in radiomics

•

Typically encountered CT exposure variations result in radiomics values distortion.

•

Proposed correction method suppresses the non-tumor related variation in radiomics.

•

The clinical relevance was shown using a 221 lung cancer patient cohort.

Purpose

Radiomics are quantitative features extracted from medical images. Many radiomic features depend not only on tumor properties, but also on non-tumor related factors such as scanner signal-to-noise ratio (SNR), reconstruction kernel and other image acquisition settings. This causes undesirable value variations in the features and reduces the performance of prediction models. In this paper, we investigate whether we can use phantom measurements to characterize and correct for the scanner SNR dependence.

Methods

We used a phantom with 17 regions of interest (ROI) to investigate the influence of different SNR values. CT scans were acquired with 9 different exposure settings. We developed an additive correction model to reduce scanner SNR influence.

Results

Sixty-two of 92 radiomic features showed high variance due to the scanner SNR. Of these 62 features, 47 showed at least a factor 2 significant standard deviation reduction by using the additive correction model. We assessed the clinical relevance of radiomics instability by using a 221 NSCLC patient cohort measured with the same scanner.

Conclusions

Phantom measurements show that roughly two third of the radiomic features depend on the exposure setting of the scanner. The dependence can be modeled and corrected significantly reducing the variation in feature values with at least a factor of 2. More complex models will likely increase the correctability. Scanner SNR correction will result in more reliable radiomics predictions in NSCLC.

Systematic analysis of bias and variability of texture measurements in computed tomography

Texture is a key radiomics measurement for quantification of disease and disease progression. The sensitivity of the measurements to image acquisition, however, is uncertain. We assessed bias and variability of computed tomography (CT) texture feature measurements across many clinical image acquisition settings and reconstruction algorithms. Diverse, anatomically informed textures (texture A, B, and C) were simulated across 1188 clinically relevant CT imaging conditions representing four in-plane pixel sizes (0.4, 0.5, 0.7, and 0.9 mm), three slice thicknesses (0.625, 1.25, and 2.5 mm), three dose levels ( CTDI vol 1.90, 3.75, and 7.50 mGy), and 33 reconstruction kernels. Imaging conditions corresponded to noise and resolution properties representative of five commercial scanners (GE LightSpeed VCT, GE Discovery 750 HD, GE Revolution, Siemens Definition Flash, and Siemens Force) in filtered backprojection and iterative reconstruction. About 21 texture features were calculated and compared between the ground-truth phantom (i.e., preimaging) and its corresponding images. Each feature was measured with four unique volumes of interest (VOIs) sizes (244, 579, 1000, and 1953    mm 3 . To characterize the bias, the percentage relative difference [PRD(%)] in each feature was calculated between the imaged scenario and the ground truth for all VOI sizes. Feature variability was assessed in terms of (1)  σ PRD ( % ) indicating the variability between the ground truth and simulated image scenario based on the PRD(%), (2)  COV f indicating the simulation-based variability, and (3)  COV T indicating the natural variability present in the ground-truth phantom. The PRD ranged widely from - 97 % to 1220%, with an underlying variability ( σ ) of up to 241%. Features such as gray-level nonuniformity, texture entropy, sum average, and homogeneity exhibited low susceptibility to reconstruction kernel effects ( PRD < 3 % ) with relatively small σ PRD ( % ) ( ≤ 5 % ) across imaging conditions. The dynamic range of results indicates that image acquisition and reconstruction conditions of in-plane pixel sizes, slice thicknesses, dose levels, and reconstruction kernels can lead to significant bias and variability in feature measurements.

Matching and Homogenizing Convolution Kernels for Quantitative Studies in Computed Tomography

The sharpness of the kernels used for image reconstruction in computed tomography affects the values of the quantitative image features. We sought to identify the kernels that produce similar feature values to enable a more effective comparison of images produced using scanners from different manufactures. We also investigated a new image filter designed to change the kernel-related component of the frequency spectrum of a postreconstruction image from that of the initial kernel to that of a preferred kernel. A radiomics texture phantom was imaged using scanners from GE, Philips, Siemens, and Toshiba. Images were reconstructed multiple times, varying the kernel from smooth to sharp. The phantom comprised 10 cartridges of various textures. A semiautomated method was used to produce 8 × 2 × 2 cm regions of interest for each cartridge and for all scans. For each region of interest, 38 radiomics features from the categories intensity direct (n = 12), gray-level co-occurrence matrix (n = 21), and neighborhood gray-tone difference matrix (n = 5) were extracted. We then calculated the fractional differences of the features from those of the baseline kernel (GE Standard). To gauge the importance of the differences, we scaled them by the coefficient of variation of the same feature from a cohort of patients with non-small cell lung cancer. The noise power spectra for each kernel were estimated from the phantom's solid acrylic cartridge, and kernel-homogenization filters were developed from these estimates. The Philips C, Siemens B30f, and Toshiba FC24 kernels produced feature values most similar to GE Standard. The kernel homogenization filters reduced the median differences from baseline to less than 1 coefficient of variation in the patient population for all of the GE, Philips, and Siemens kernels except for GE Edge and Toshiba kernels. For prospective computed tomographic radiomics studies, the scanning protocol should specify kernels that have been shown to produce similar feature values. For retrospective studies, kernel homogenization filters can be designed and applied to reduce the kernel-related differences in the feature values.

Comprehensive Investigation on Controlling for CT Imaging Variabilities in Radiomics Studies

Radiomics has shown promise in improving models for predicting patient outcomes. However, to maximize the information gain of the radiomics features, especially in larger patient cohorts, the variability in radiomics features owing to differences between scanners and scanning protocols must be accounted for. To this aim, the imaging variability of radiomics feature values was evaluated on 100 computed tomography scanners at 35 clinics by imaging a radiomics phantom using a controlled protocol and the commonly used chest and head protocols of the local clinic. We used a linear mixed-effects model to determine the degree to which the manufacturer and individual scanners contribute to the overall variability. Using a controlled protocol reduced the overall variability by 57% and 52% compared to the local chest and head protocols respectively. The controlled protocol also reduced the relative contribution of the manufacturer to the total variability. For almost all variabilities (manufacturer, scanner, and residual with different preprocesssing), the controlled protocol scans had a significantly smaller variability than the local protocol scans did. For most radiomics features, the imaging variability was small relative to the inter-patient feature variability in non-small cell lung cancer and head and neck squamous cell carcinoma patient cohorts. From this study, we conclude that using controlled scans can reduce the variability in radiomics features, and our results demonstrate the importance of using controlled protocols in prospective radiomics studies.