Semi-automatic software increases CT measurement accuracy but not response classification of colorectal liver metastases after chemotherapy

Deep learning models for automatic tumor segmentation and total tumor volume assessment in patients with colorectal liver metastases

BACKGROUND

We developed models for tumor segmentation to automate the assessment of total tumor volume (TTV) in patients with colorectal liver metastases (CRLM).

METHODS

In this prospective cohort study, pre- and post-systemic treatment computed tomography (CT) scans of 259 patients with initially unresectable CRLM of the CAIRO5 trial (NCT02162563) were included. In total, 595 CT scans comprising 8,959 CRLM were divided into training (73%), validation (6.5%), and test sets (21%). Deep learning models were trained with ground truth segmentations of the liver and CRLM. TTV was calculated based on the CRLM segmentations. An external validation cohort was included, comprising 72 preoperative CT scans of patients with 112 resectable CRLM. Image segmentation evaluation metrics and intraclass correlation coefficient (ICC) were calculated.

RESULTS

In the test set (122 CT scans), the autosegmentation models showed a global Dice similarity coefficient (DSC) of 0.96 (liver) and 0.86 (CRLM). The corresponding median per-case DSC was 0.96 (interquartile range [IQR] 0.95-0.96) and 0.80 (IQR 0.67-0.87). For tumor segmentation, the intersection-over-union, precision, and recall were 0.75, 0.89, and 0.84, respectively. An excellent agreement was observed between the reference and automatically computed TTV for the test set (ICC 0.98) and external validation cohort (ICC 0.98). In the external validation, the global DSC was 0.82 and the median per-case DSC was 0.60 (IQR 0.29-0.76) for tumor segmentation.

CONCLUSIONS

Deep learning autosegmentation models were able to segment the liver and CRLM automatically and accurately in patients with initially unresectable CRLM, enabling automatic TTV assessment in such patients.

RELEVANCE STATEMENT

Automatic segmentation enables the assessment of total tumor volume in patients with colorectal liver metastases, with a high potential of decreasing radiologist's workload and increasing accuracy and consistency.

KEY POINTS

• Tumor response evaluation is time-consuming, manually performed, and ignores total tumor volume. • Automatic models can accurately segment tumors in patients with colorectal liver metastases. • Total tumor volume can be accurately calculated based on automatic segmentations.

The Prognostic Value of Total Tumor Volume Response Compared With RECIST1.1 in Patients With Initially Unresectable Colorectal Liver Metastases Undergoing Systemic Treatment

Objectives

Compare total tumor volume (TTV) response after systemic treatment to Response Evaluation Criteria in Solid Tumors (RECIST1.1) and assess the prognostic value of TTV change and RECIST1.1 for recurrence-free survival (RFS) in patients with colorectal liver-only metastases (CRLM).

Background

RECIST1.1 provides unidimensional criteria to evaluate tumor response to systemic therapy. Those criteria are accepted worldwide but are limited by interobserver variability and ignore potentially valuable information about TTV.

Methods

Patients with initially unresectable CRLM receiving systemic treatment from the randomized, controlled CAIRO5 trial (NCT02162563) were included. TTV response was assessed using software specifically developed together with SAS analytics. Baseline and follow-up computed tomography (CT) scans were used to calculate RECIST1.1 and TTV response to systemic therapy. Different thresholds (10%, 20%, 40%) were used to define response of TTV as no standard currently exists. RFS was assessed in a subgroup of patients with secondarily resectable CRLM after induction treatment.

Results

A total of 420 CT scans comprising 7820 CRLM in 210 patients were evaluated. In 30% to 50% (depending on chosen TTV threshold) of patients, discordance was observed between RECIST1.1 and TTV change. A TTV decrease of >40% was observed in 47 (22%) patients who had stable disease according to RECIST1.1. In 118 patients with secondarily resectable CRLM, RFS was shorter for patients with less than 10% TTV decrease compared with patients with more than 10% TTV decrease (P = 0.015), while RECIST1.1 was not prognostic (P = 0.821).

Conclusions

TTV response assessment shows prognostic potential in the evaluation of systemic therapy response in patients with CRLM.

Variability of quantitative measurements of metastatic liver lesions: a multi-radiation-dose-level and multi-reader comparison

PURPOSE

To evaluate the variability of quantitative measurements of metastatic liver lesions by using a multi-radiation-dose-level and multi-reader comparison.

METHODS

Twenty-three study subjects (mean age, 60 years) with 39 liver lesions who underwent a single-energy dual-source contrast-enhanced staging CT between June 2015 and December 2015 were included. CT data were reconstructed with seven different radiation dose levels (ranging from 25 to 100%) on the basis of a single CT acquisition. Four radiologists independently performed manual tumor measurements and two radiologists performed semi-automated tumor measurements. Interobserver, intraobserver, and interdose sources of variability for longest diameter and volumetric measurements were estimated and compared using Wilcoxon rank-sum tests and intraclass correlation coefficients.

RESULTS

Inter- and intraobserver variabilities for manual measurements of the longest diameter were higher compared to semi-automated measurements (p < 0.001 for overall). Inter- and intraobserver variabilities of volume measurements were higher compared to the longest diameter measurement (p < 0.001 for overall). Quantitative measurements were statistically different at < 50% radiation dose levels for semi-automated measurements of the longest diameter, and at 25% radiation dose level for volumetric measurements. The variability related to radiation dose was not significantly different from the inter- and intraobserver variability for the measurements of the longest diameter.

CONCLUSION

The variability related to radiation dose is comparable to the inter- and intraobserver variability for measurements of the longest diameter. Caution should be warranted in reducing radiation dose level below 50% of a conventional CT protocol due to the potentially detrimental impact on the assessment of lesion response in the liver.

Volumetric tumor response assessment is inefficient without overt clinical benefit compared to conventional, manual veterinary response assessment in canine nasal tumors

Accurate assessment of tumor response to therapy is critical in guiding management of veterinary oncology patients and is most commonly performed using response evaluation criteria in solid tumors criteria. This process can be time consuming and have high intra- and interobserver variability. The primary aim of this serial measurements, secondary analysis study was to compare manual linear tumor response assessment to semi-automated, contoured response assessment in canine nasal tumors. The secondary objective was to determine if tumor measurements or clinical characteristics, such as stage, would correlate to progression-free interval. Three investigators evaluated paired CT scans of skulls of 22 dogs with nasal tumors obtained prior to and following radiation therapy. The automatically generated tumor volumes were not useful for canine nasal tumors in this study, characterized by poor intraobserver agreement between automatically generated contours and hand-adjusted contours. The radiologist's manual linear method of determining response evaluation criteria in solid tumors categorization and tumor volume is significantly faster (P < .0001) but significantly underestimates nasal tumor volume (P < .05) when compared to a contour-based method. Interobserver agreement was greater for volume determination using the contour-based method when compared to response evaluation criteria in solid tumors categorization utilizing the same method. However, response evaluation criteria in solid tumors categorization and percentage volume change were strongly correlated, providing validity to response evaluation criteria in solid tumors as a rapid method of tumor response assessment for canine nasal tumors. No clinical characteristics or tumor measurements were significantly associated with progression-free interval.

Precision of manual two-dimensional segmentations of lung and liver metastases and its impact on tumour response assessment using RECIST 1.1

Background

Response evaluation criteria in solid tumours (RECIST) has significant limitations in terms of variability and reproducibility, which may not be independent. The aim of the study was to evaluate the precision of manual bi-dimensional segmentation of lung, liver metastases, and to quantify the uncertainty in tumour response assessment.

Methods

A total of 520 segmentations of metastases from six livers and seven lungs were independently performed by ten physicians and ten scientists on CT images, reflecting the variability encountered in clinical practice. Operators manually contoured the tumours, firstly independently according to the RECIST and secondly on a preselected slice. Diameters and areas were extracted from the segmentations. Mean standard deviations were used to build regression models and 95% confidence intervals (95% CI) were calculated for each tumour size and for limits of progressive disease (PD) and partial response (PR) derived from RECIST 1.1.

Results

Thirteen aberrant segmentations (2.5%) were observed without significant differences between the physicians and scientists; only the mean area of liver tumours (p = 0.034) and mean diameter of lung tumours (p = 0.021) differed significantly. No difference was observed between the methods. Inter-observer agreement was excellent (intra-class correlation >0.90) for all variables. In liver, overlaps of the 95% CI with the 95% CI of limits of PD or PR were observed for diameters above 22.7 and 37.9 mm, respectively. An overlap of 95% CIs was systematically observed for area. No overlaps were observed in lung.

Conclusions

Although the experience of readers might not affect the precision of segmentation in lung and liver, the results of manual segmentation performed for tumour response assessment remain uncertain for large liver metastases.

Electronic supplementary material

The online version of this article (doi:10.1186/s41747-017-0015-4) contains supplementary material, which is available to authorized users.

Semi-automatic 3D-volumetry of liver metastases from neuroendocrine tumors to improve combination therapy with 177Lu-DOTATOC and 90Y-DOTATOC

PURPOSE

Patients with neuroendocrine tumors (NET) often present with disseminated liver metastases and can be treated with a number of different nuclides or nuclide combinations in peptide receptor radionuclide therapy (PRRT) depending on tumor load and lesion diameter. For quantification of disseminated liver lesions, semi-automatic lesion detection is helpful to determine tumor burden and tumor diameter in a time efficient manner. Here, we aimed to evaluate semi-automated measurement of total metastatic burden for therapy stratification.

METHODS

Nineteen patients with liver metastasized NET underwent contrast-enhanced 1.5 T MRI using gadolinium-ethoxybenzyl diethylenetriaminepentaacetic acid. Liver metastases (n=1537) were segmented using Fraunhofer MEVIS Software for three-dimensional (3D) segmentation. All lesions were stratified according to longest 3D diameter >20 mm or ≤20 mm and relative contribution to tumor load was used for therapy stratification.

RESULTS

Mean count of lesions ≤20 mm was 67.5 and mean count of lesions >20 mm was 13.4. However, mean contribution to total tumor volume of lesions ≤20 mm was 24%, while contribution of lesions >20 mm was 76%.

CONCLUSION

Semi-automatic lesion analysis provides useful information about lesion distribution in predominantly liver metastasized NET patients prior to PRRT. As conventional manual lesion measurements are laborious, our study shows this new approach is more efficient and less operator-dependent and may prove to be useful in the decision making process selecting the best combination PRRT in each patient.

Observer variability in RECIST-based tumour burden measurements: a meta-analysis

BACKGROUND

Response Evaluation Criteria in Solid Tumours (RECIST)-based tumour burden measurements involve observer variability, the extent of which ought to be determined.

METHODS

A literature search identified studies on observer variability during manual measurements of tumour burdens via computed tomography according to the RECIST guideline. The 95% limit of agreement (LOA) values of relative measurement difference (RMD) were pooled using a random-effects model.

RESULTS

Twelve studies were included. Pooled 95% LOAs of RMD in measuring unidimensional longest diameters of single lesions ranged from -22.1% (95% confidence interval [CI], -30.3% to -14.0%) to 25.4% (95% CI, 17.2% to 33.5%) between observers and -17.8% (95% CI, -23.6% to -11.9%) to 16.1% (95% CI, 10.1% to 21.8%) for a single observer. Pooled 95% LOAs of RMD in measuring the sum of multiple lesions ranged from -19.2% (95% CI, -23.7% to -14.9%) to 19.5% (95% CI, 15.2% to 23.9%) between observers, and -9.8% (95% CI, -19.0% to -0.3%) to 13.1% (95% CI, 3.6% to 22.6%) for a single observer. Pooled 95% LOA of RMD in calculating the interval change of tumour burden with a single lesion ranged from -31.3% (95% CI, -46.0% to -16.5%) to 30.3% (95% CI, 15.3% to 44.8%) between observers. Studies on calculating the interval change of tumour burden for a single observer or with multiple lesions were lacking.

CONCLUSION

Interobserver RMD in measuring single tumour burden and calculating its interval change may exceed the 20% cut-off for progression. Variability decreased when tumour burden was measured by a single observer or assessed by the sum of multiple lesions.

Volumetric response classification in metastatic solid tumors on MSCT: initial results in a whole-body setting

PURPOSE

To examine technical parameters of measurement accuracy and differences in tumor response classification using RECIST 1.1 and volumetric assessment in three common metastasis types (lung nodules, liver lesions, lymph node metastasis) simultaneously.

MATERIALS AND METHODS

56 consecutive patients (32 female) aged 41-82 years with a wide range of metastatic solid tumors were examined with MSCT for baseline and follow up. Images were evaluated by three experienced radiologists using manual measurements and semi-automatic lesion segmentation. Institutional ethics review was obtained and all patients gave written informed consent. Data analysis comprised interobserver variability operationalized as coefficient of variation and categorical response classification according to RECIST 1.1 for both manual and volumetric measures. Continuous data were assessed for statistical significance with Wilcoxon signed-rank test and categorical data with Fleiss kappa.

RESULTS

Interobserver variability was 6.3% (IQR 4.6%) for manual and 4.1% (IQR 4.4%) for volumetrically obtained sum of relevant diameters (p<0.05, corrected). 4-8 patients' response to therapy was classified differently across observers by using volumetry compared to standard manual measurements. Fleiss kappa revealed no significant difference in categorical agreement of response classification between manual (0.7558) and volumetric (0.7623) measurements.

CONCLUSION

Under standard RECIST thresholds there was no advantage of volumetric compared to manual response evaluation. However volumetric assessment yielded significantly lower interobserver variability. This may allow narrower thresholds for volumetric response classification in the future.

Comparison between CT volume measurement and histopathological assessment of response to neoadjuvant therapy in rectal cancer

OBJECTIVES

The aim of this study was to compare volume measurements on computed tomography (CT) images with histopathological assessments of chemoradiotherapy (CRT)-induced tumor regression in locally advanced rectal cancer (RC).

METHODS

In 25 patients (13 males, 12 females; median age, 63 years; age range, 44-79 years) with locally advanced RC treated with preoperative CRT and surgery, two radiologists measured tumor volume on CT images before and after CRT. CT-based tumor volumetry and the modified response evaluation criteria in solid tumors (mRECISTs) were compared with T and N downstaging after CRT, and with the tumor regression grade (TRG).

RESULTS

Tumor volumes were significantly smaller on CT images after CRT. The tumors regressed in 52% (13/25), 36% (9/25) and 40% (10/25) of patients, based on T downstaging, TRG and mRECIST findings, respectively. In terms of T downstaging, the pre- and post-CRT tumor volumes of responders and non-responders to the treatment differed statistically, while their tumor volume reduction rates and volume reductions according to the 65% mRECIST threshold did not. In terms of N downstaging and TRG, the differences between the responders' and the non-responders' pre- and post-CRT tumor volumes, tumor volume reduction rates, and mRECIST thresholds were never statistically significant.

CONCLUSION

Measuring tumor size on CT images is of limited value in predicting the histopathological response to preoperative CRT in RC patients, so it may be unwise to select surgical treatment strategies based on CT volumetry.

Comparison of response evaluation criteria in solid tumors with volumetric measurements for estimation of tumor burden in pancreatic adenocarcinoma and hepatocellular carcinoma

BACKGROUND

Response evaluation criteria in solid tumors (RECIST) is the accepted method for determining tumor progression. However, RECIST may not estimate disease burden accurately because the axial plane often does not produce the actual longest diameter. Volumetric measurements may be an alternative to better determine tumor size. Our aim was to compare volumetric measurements with RECIST in pancreatic ductal adenocarcinomas (PDA) and hepatocellular carcinomas (HCC).

METHODS

RECIST and volumetric measurements were determined in 9 patients with metastatic PDA and 17 patients with HCC who subsequently underwent liver transplantation. Gross pathologic measurements after hepatectomy also were analyzed for volumes.

RESULTS

Three-dimensional diameter in volumetric analysis was 38% and 36% higher than RECIST diameter in PDA and HCC, respectively (P < .01). However, RECIST yielded 78% and 23% larger estimated tumor volumes than volumetric analysis in PDA and HCC, respectively (P < .01). Gross pathologic volume in HCC showed a linear correlation with both volumetric analysis (r = .95; P < .01) and RECIST (r = .96; P < .01) but RECIST significantly overestimated gross pathologic volume by an average of 28% (P < .01) whereas volumetric analysis was similar to gross pathologic volume (P = .56). In categorizing treatment response in PDA, RECIST and volumetric analysis were in moderate agreement (κ = .49).

CONCLUSIONS

RECIST significantly may overestimate tumor burden compared with volumetric measurements in both PDA and HCC. Volumetric analysis may be the preferred method to detect tumor progression.