Inter-observer reproducibility of semi-automatic tumor diameter measurement and volumetric analysis in patients with lung cancer

ViNe-Seg: deep-learning-assisted segmentation of visible neurons and subsequent analysis embedded in a graphical user interface

SUMMARY

Segmentation of neural somata is a crucial and usually the most time-consuming step in the analysis of optical functional imaging of neuronal microcircuits. In recent years, multiple auto-segmentation tools have been developed to improve the speed and consistency of the segmentation process, mostly, using deep learning approaches. Current segmentation tools, while advanced, still encounter challenges in producing accurate segmentation results, especially in datasets with a low signal-to-noise ratio. This has led to a reliance on manual segmentation techniques. However, manual methods, while customized to specific laboratory protocols, can introduce variability due to individual differences in interpretation, potentially affecting dataset consistency across studies. In response to this challenge, we present ViNe-Seg: a deep-learning-based semi-automatic segmentation tool that offers (i) detection of visible neurons, irrespective of their activity status; (ii) the ability to perform segmentation during an ongoing experiment; (iii) a user-friendly graphical interface that facilitates expert supervision, ensuring precise identification of Regions of Interest; (iv) an array of segmentation models with the option of training custom models and sharing them with the community; and (v) seamless integration of subsequent analysis steps.

AVAILABILITY AND IMPLEMENTATION

ViNe-Seg code and documentation are publicly available at https://github.com/NiRuff/ViNe-Seg and can be installed from https://pypi.org/project/ViNeSeg/.

CYFRA 21-1, CA 125 and CEA provide additional prognostic value in NSCLC patients with stable disease at first CT scan

BACKGROUND

Serum tumor markers (STM) may complement imaging and provide additional clinical information for patients with non-small cell lung cancer (NSCLC).

OBJECTIVE

To determine whether STMs can predict outcomes in patients with stable disease (SD) after initial treatment.

METHODS

This single-center, prospective, observational trial enrolled 395 patients with stage III/IV treatment-naïve NSCLC; of which 263 patients were included in this analysis. Computed Tomography (CT) scans were performed and STMs measured before and after initial treatment (two cycles of chemotherapy and/or an immune checkpoint inhibitor or tyrosine kinase inhibitor); analyses were based on CT and STM measurements obtained at first CT performed after cycle 2 only PFS and OS were analyzed by Kaplan-Meier curves and Cox-proportional hazard models.

RESULTS

When patients with SD (n = 100) were split into high- and low-risk groups based on CYFRA 21-1, CEA and CA 125 measurements using an optimized cut-off, a 4-fold increase risk of progression or death was estimated for high- vs low-risk SD patients (PFS, HR 4.17; OS, 3.99; both p < 0.0001). Outcomes were similar between patients with high-risk SD or progressive disease (n = 35) (OS, HR 1.17) and between patients with low-risk SD or partial response (n = 128) (PFS, HR 0.98; OS, 1.14).

CONCLUSIONS

STMs can provide further guidance in patients with indeterminate CT responses by separating them into high- and low-risk groups for future PFS and OS events.

Discrepant Assessments of Progressive Disease in Clinical Trials between Routine Clinical Reads and Formal RECIST 1.1 Interpretations

Purpose To analyze the frequency of discrepant interpretations of progressive disease (PD) between routine clinical and formal Response Evaluation Criteria in Solid Tumors (RECIST) 1.1 interpretations in patients enrolled in solid tumor clinical trials and investigate the causes of discordance. Materials and Methods This retrospective study included patients in solid tumor clinical trials undergoing imaging response assessments based on RECIST 1.1 from January to July 2021. Routine clinical interpretations (RCIs) performed as part of standard workflow and not requiring formal use of any established response criteria were compared with separate local core laboratory interpretations (CLIs) by specially trained radiologists who used software that tracks target lesion measurements, changes in nontarget lesions, and appearance of new lesions longitudinally. The comparison focused on discordant interpretations of PD. Results Among 1053 patients who had both RCIs and CLIs performed, PD was diagnosed on one or both reads in 327 patients (median age, 63.6 [range, 22.4-83.2] years; 57.8% female patients). The RCIs and CLIs agreed with PD status in 65% (213 of 327) of assessments. In 32% (105 of 327) of assessments, RCIs overdiagnosed PD when CLIs diagnosed stable disease, and in 3% (nine of 327), CLIs diagnosed PD when RCIs diagnosed stable disease. Reasons for discrepant RCIs of PD included erroneous target lesion measurements (58%, 61 of 105), erroneous diagnosis of nontarget progression (30%, 32 of 105), and misclassification of new lesions as cancer (11%, 12 of 105). Most patients (93%, 98 of 105) with RCI overdiagnosis of PD remained in the clinical trial for one or more treatment cycles. Conclusion PD was frequently overdiagnosed on RCIs versus formal RECIST 1.1 CLIs which could result in patients removed from the clinical trial inappropriately. Keywords: Oncology, Cancer, Tumor Response, MR Imaging, CT © RSNA, 2023 See also commentary by Margolis and Ruchalski in this issue.

Artificial intelligence and machine learning in cancer imaging

An increasing array of tools is being developed using artificial intelligence (AI) and machine learning (ML) for cancer imaging. The development of an optimal tool requires multidisciplinary engagement to ensure that the appropriate use case is met, as well as to undertake robust development and testing prior to its adoption into healthcare systems. This multidisciplinary review highlights key developments in the field. We discuss the challenges and opportunities of AI and ML in cancer imaging; considerations for the development of algorithms into tools that can be widely used and disseminated; and the development of the ecosystem needed to promote growth of AI and ML in cancer imaging.

Comparison of gross pathology inspection and 9.4 T magnetic resonance imaging in the evaluation of radiofrequency ablation lesions in the left ventricle of the swine heart

Aims: Gross pathology inspection (patho) is the gold standard for the morphological evaluation of focal myocardial pathology. Examination with 9.4 T magnetic resonance imaging (MRI) is a new method for very accurate display of myocardial pathology. The aim of this study was to demonstrate that lesions can be measured on high-resolution MRI images with the same accuracy as on pathological sections and compare these two methods for the evaluation of radiofrequency (RF) ablation lesion dimensions in swine heart tissue during animal experiment.

Methods: Ten pigs underwent radiofrequency ablations in the left ventricle during animal experiment. After animal euthanasia, hearts were explanted, flushed with ice-cold cardioplegic solution to relax the whole myocardium, fixed in 10% formaldehyde and scanned with a 9.4 T magnetic resonance system. Anatomical images were processed using ImageJ software. Subsequently, the hearts were sliced, slices were photographed and measured in ImageJ software. Different dimensions and volumes were compared.

Results: The results of both methods were statistically compared. Depth by MRI was 8.771 ± 2.595 mm and by patho 9.008 ± 2.823 mm; p = 0.198. Width was 10.802 ± 2.724 mm by MRI and 11.125 ± 2.801 mm by patho; p = 0.049. Estuary was 2.006 ± 0.867 mm by MRI and 2.001 ± 0.872 mm by patho; p = 0.953. The depth at the maximum diameter was 4.734 ± 1.532 mm on MRI and 4.783 ± 1.648 mm from the patho; p = 0.858. The volumes of the lesions calculated using a formula were 315.973 ± 257.673 mm³ for MRI and 355.726 ± 255.860 mm³ for patho; p = 0.104. Volume directly measured from MRI with the “point-by-point” method was 671.702 ± 362.299 mm³.

Conclusion: Measurements obtained from gross pathology inspection and MRI are fully comparable. The advantage of MRI is that it is a non-destructive method enabling repeated measurements in all possible anatomical projections.

Monitoring tumor growth rate to predict immune checkpoint inhibitors’ treatment outcome in advanced NSCLC

Introduction:

Radiological response assessment to immune checkpoint inhibitor is challenging due to atypical pattern of response and commonly used RECIST 1.1 criteria do not take into account the kinetics of tumor behavior. Our study aimed at evaluating the tumor growth rate (TGR) in addition to RECIST 1.1 criteria to assess the benefit of immune checkpoint inhibitors (ICIs).

Methods:

Tumor real volume was calculated with a dedicated computed tomography (CT) software that semi-automatically assess tumor volume. Target lesions were identified according to RECIST 1.1. For each patient, we had 3 measurement of tumor volume. CT-1 was performed 8–12 weeks before ICI start, the CT at baseline for ICI was CT0, while CT + 1 was the first assessment after ICI. We calculated the percentage increase in tumor volume before (TGR1) and after immunotherapy (TGR2). Finally, we compared TGR1 and TGR2. If no progressive disease (PD), the group was disease control (DC). If PD but TGR2 < TGR1, it was called LvPD and if TGR2 ⩾ TGR1, HvPD.

Results:

A total of 61 patients who received ICIs and 33 treated with chemotherapy (ChT) were included. In ICI group, 18 patients were HvPD, 22 LvPD, 21 DC. Median OS was 4.4 months (95% CI: 2.0–6.8, reference) for HvPD, 7.1 months (95% CI 5.4–8.8) for LvPD, p = 0.018, and 20.9 months (95% CI: 12.5–29.3) for DC, p < 0.001. In ChT group, 7 were categorized as HvPD, 17 as LvPD and 9 as DC. No difference in OS was observed in the ChT group (p = 0.786)

Conclusion:

In the presence of PD, a decrease in TGR may result in a clinical benefit in patients treated with ICI but not with chemotherapy. Monitoring TGR changes after ICIs administration can help physician in deciding to treat beyond PD.

Reliability of 3 Strategies of Orbital Tumor Volume Measurement Using Phantom Modeling

PURPOSE

The reliability of 3 volume measurement strategies was investigated using MRI and a simple method for creating phantom orbit tumors.

METHODS

Water-based starch was molded into orbital "tumors" of 3 shapes (sphere, ovoid, diffuse); water displacement was used to calculate volume. "Tumors" were placed into 3D-printed orbit phantoms, MRIs were obtained and volume analysis was performed. Observers measured tumor volume using ellipsoid volume (EV), manual segmentation, and semi-automated segmentation strategies. Intraclass correlation coefficients were calculated comparing observer measurements to true volumes. The coefficient of repeatability determined the percentage of tumor volume change required for each method to detect tumor growth.

RESULTS

Intraclass correlation coefficients comparing measured volumes to true volumes using EV, manual segmentation, and semi-automated segmentation were 0.61, 0.98, and 0.99 for spherical, 0.64, 0.97, and 0.98 for ovoid, and 0.18, 0.82, and 0.87 for diffuse tumors. Semi-automated segmentation followed by manual segmentation had the highest correlation between measured and true tumor volume for all 3 tumor geometries. EV had low correlation with true volume for all tumor geometries. Diffuse tumors had high variability and low correlation for all 3 measurement techniques.

CONCLUSIONS

This study shows the reliability of 3 strategies to measure orbital tumor volume with MRI based on tumor geometry, using a simple phantom model. EV, the most commonly employed strategy in clinical practice, had low correlation and high variability across tumor shapes. Using manual segmentation and semi-automated segmentation, a measured change in volume greater than 25% may be considered true growth, while the EV strategy required a 40%-400% change in volume to reliably measure tumor growth.

Baseline total metabolic tumour volume on 2-deoxy-2-[18F]fluoro-d-glucose positron emission tomography-computed tomography as a promising biomarker in patients with advanced non-small cell lung cancer treated with first-line pembrolizumab

INTRODUCTION

Immune checkpoint inhibitors (ICIs) have become the standard of care in the management of advanced non-small cell lung cancer (NSCLC). Nevertheless, only a small proportion of patients benefit from ICIs. The aim of the present study is to assess whether 2-deoxy-2-[18F]fluoro-d-glucose positron emission tomography-computed tomography ([18F]FDG-PET/CT)-derived parameters may be used as biomarkers in patients with advanced NSCLC receiving first-line pembrolizumab.

MATERIALS AND METHODS

This is a monocentric retrospective cohort study including patients with advanced NSCLC (stage IV) and Programmed death-ligand 1 (PD-L1) expression ≥50% treated with pembrolizumab. A control group of patients treated with epidermal growth factor receptor (EGFR) inhibitors for EGFR-mutated NSCLC was also enrolled. Only patients with a positive [18F]18F-FDG PET/CT result within 60 days from treatment initiation were included.Total metabolic tumour volume (tMTV) was calculated for each lesion using a dedicated software (PET VCAR; GE Healthcare), which semiautomatically delineates the tumour's contours with a maximum standardised uptake value (SUVmax) threshold of 42% within the lesion. tMTV was obtained summing each lesion's MTV. Potential prognostic parameters for overall survival (OS) were analysed (tMTV, SUVmax, bone/liver metastasis, neutrophil:lymphocyte ratio ≥4, Eastern Cooperative Oncology Group performance status ≥2, lactate dehydrogenase above the upper limit of normal).

RESULTS

Overall, 34 patients treated with first line-pembrolizumab and 40 patients treated with EGFR tyrosine kinase inhibitors were included. In the pembrolizumab group, the median follow-up was 20.3, while the median OS was 4.7 months (95% confidence interval [CI] = 0.3-9.1) for patients with tMTV ≥75 cm³ vs not reached (NR) for patients with tMTV <75 cm³ (95% CI = NR-NR; hazard ratio [HR] = 5.37; 95% CI = 1.72-16.77; p = 0.004). No difference was found in the control group (HR = 1.43; 95% CI = 0.61-3.34; p = 0.411).

CONCLUSION

Our data suggest that tMTV ≥75cm³ can be used as a prognostic biomarker of poor outcomes in patients with PD-L1-high advanced NSCLC treated with first-line pembrolizumab. This information could be useful for the selection of patients who may require the addition of chemotherapy to pembrolizumab.

Variability of quantitative measurements of metastatic liver lesions: a multi-radiation-dose-level and multi-reader comparison

PURPOSE

To evaluate the variability of quantitative measurements of metastatic liver lesions by using a multi-radiation-dose-level and multi-reader comparison.

METHODS

Twenty-three study subjects (mean age, 60 years) with 39 liver lesions who underwent a single-energy dual-source contrast-enhanced staging CT between June 2015 and December 2015 were included. CT data were reconstructed with seven different radiation dose levels (ranging from 25 to 100%) on the basis of a single CT acquisition. Four radiologists independently performed manual tumor measurements and two radiologists performed semi-automated tumor measurements. Interobserver, intraobserver, and interdose sources of variability for longest diameter and volumetric measurements were estimated and compared using Wilcoxon rank-sum tests and intraclass correlation coefficients.

RESULTS

Inter- and intraobserver variabilities for manual measurements of the longest diameter were higher compared to semi-automated measurements (p < 0.001 for overall). Inter- and intraobserver variabilities of volume measurements were higher compared to the longest diameter measurement (p < 0.001 for overall). Quantitative measurements were statistically different at < 50% radiation dose levels for semi-automated measurements of the longest diameter, and at 25% radiation dose level for volumetric measurements. The variability related to radiation dose was not significantly different from the inter- and intraobserver variability for the measurements of the longest diameter.

CONCLUSION

The variability related to radiation dose is comparable to the inter- and intraobserver variability for measurements of the longest diameter. Caution should be warranted in reducing radiation dose level below 50% of a conventional CT protocol due to the potentially detrimental impact on the assessment of lesion response in the liver.

Retrospective comparison of approaches to evaluating inter-observer variability in CT tumour measurements in an academic health centre

BACKGROUND

A growing number of research studies have reported inter-observer variability in sizes of tumours measured from CT scans. It remains unclear whether the conventional statistical measures correctly evaluate the CT measurement consistency for optimal treatment management and decision-making. We compared and evaluated the existing measures for evaluating inter-observer variability in CT measurement of cancer lesions.

METHODS

13 board-certified radiologists repeatedly reviewed 10 CT image sets of lung lesions and hepatic metastases selected through a randomisation process. A total of 130 measurements under RECIST 1.1 (Response Evaluation Criteria in Solid Tumors) guidelines were collected for the demonstration. Intraclass correlation coefficient (ICC), Bland-Altman plotting and outlier counting methods were selected for the comparison. The each selected measure was used to evaluate three cases with observed, increased and decreased inter-observer variability.

RESULTS

The ICC score yielded a weak detection when evaluating different levels of the inter-observer variability among radiologists (increased: 0.912; observed: 0.962; decreased: 0.990). The outlier counting method using Bland-Altman plotting with 2SD yielded no detection at all with its number of outliers unchanging regardless of level of inter-observer variability. Outlier counting based on domain knowledge was more sensitised to different levels of the inter-observer variability compared with the conventional measures (increased: 0.756; observed: 0.923; improved: 1.000). Visualisation of pairwise Bland-Altman bias was also sensitised to the inter-observer variability with its pattern rapidly changing in response to different levels of the inter-observer variability.

CONCLUSIONS

Conventional measures may yield weak or no detection when evaluating different levels of the inter-observer variability among radiologists. We observed that the outlier counting based on domain knowledge was sensitised to the inter-observer variability in CT measurement of cancer lesions. Our study demonstrated that, under certain circumstances, the use of standard statistical correlation coefficients may be misleading and result in a sense of false security related to the consistency of measurement for optimal treatment management and decision-making.

Intervention to Reduce Interobserver Variability in Computed Tomographic Measurement of Cancer Lesions Among Experienced Radiologists

While a growing number of research studies have reported the inter-observer variability in computed tomographic (CT) measurements, there are very few interventional studies performed. We aimed to assess whether a peer benchmarking intervention tool may have an influence on reducing interobserver variability in CT measurements and identify possible barriers to the intervention. In this retrospective study, 13 board-certified radiologists repeatedly reviewed 10 CT image sets of lung lesions and hepatic metastases during 3 noncontiguous time periods (T1, T2, T3). Each preselected case contained normal anatomy cephalad and caudal to the lesion of interest. Lesion size measurement under RECISTS 1.1 guidelines, choice of CT slice, and time spent on measurement were captured. Prior to their final measurements, the participants were exposed to the intervention designed to reduce the number of measurements deviating from the median. Chi-square test was performed to identify radiologist-dependent factors associated with the variability. The percent of deviating measurements during T1 and T2 were 20.0% and 23.1%, respectively. There was no statistically significant change in the number of deviating measurements upon the presentation of the intervention despite the decrease in percent from 23.1% to 17.7%. The identified barriers to the intervention include clinical disagreements among radiologists. Specifically, the inter-observer variability was associated with the controversy over the choice of CT image slice (P = 0.045) and selection of start-point, axis, and end-point (P = 0.011). Clinical disagreements rather than random errors were barriers to reducing interobserver variability in CT measurement among experienced radiologists. Future interventions could aim to resolve the disagreement in an interactive approach.

Modified Quality Threshold Clustering for Temporal Analysis and Classification of Lung Lesions

Lung cancer is the type of cancer that most often kills after the initial diagnosis. To aid the specialist in its diagnosis, temporal evaluation is a potential tool for analyzing indeterminate lesions, which may be benign or malignant, during treatment. With this goal in mind, a methodology is herein proposed for the analysis, quantification, and visualization of changes in lung lesions. This methodology uses a modified version of the quality threshold clustering algorithm to associate each voxel of the lesion to a cluster, and changes in the lesion over time are defined in terms of voxel moves to another cluster. In addition, statistical features are extracted for classification of the lesion as benign or malignant. To develop the proposed methodology, two databases of pulmonary lesions were used, one for malignant lesions in treatment (public) and the other for indeterminate cases (private). We determined that the density change percentage varied from 6.22% to 36.93% of lesion volume in the public database of malignant lesions under treatment and from 19.98% to 38.81% in the private database of lung nodules. Additionally, other inter-cluster density change measures were obtained. These measures indicate the degree of change in the clusters and how each of them is abundant in relation to volume. From the statistical analysis of regions in which the density changes occurred, we were able to discriminate lung lesions with an accuracy of 98.41%, demonstrating that these changes can indicate the true nature of the lesion. In addition to visualizing the density changes occurring in lesions over time, we quantified these changes and analyzed the entire set through volumetry, which is the technique most commonly used to analyze changes in pulmonary lesions.

A review of automatic lung tumour segmentation in the era of 4DCT

Aim

To review the literature on auto-contouring methods of lung tumour volumes on four-dimensional computed tomography (4DCT).

Background

Manual delineation of lung tumour on 4DCT has been the gold standard in clinical practice. However, it is resource intensive due to the high volume of data which results in longer contouring duration and uncertainties in defining target. Auto-contouring may present as an attractive alternative by decreasing manual inputs required, thus improving the contouring process. This review aims to assess the accuracy, variability and contouring duration of automatic contouring compared with manual contouring in lung cancer on 4DCT datasets.

Materials and methods

A search and review of literature were conducted to identify studies regarding lung tumour contouring on 4DCT. Manual and auto-contours were assessed and compared based on accuracy, variability and contouring duration.

Results

Thirteen studies were included in this review and their results were compared. Accuracy of auto-contours was found to be comparable to manual contours. Auto-contouring resulted in lesser inter-observer variation when compared to manual contouring, however there was no significant reduction in intra-observer variability. Additionally, contouring duration was reduced with auto-contouring although long computation time could present as a bottleneck.

Conclusion

Auto-contouring is reliable and efficient, producing accurate contours with better consistency compared to manual contours. However, manual inputs would still be required both before and after auto-propagation.

Discrepancies of assessments in a RECIST 1.1 phase II clinical trial - association between adjudication rate and variability in images and tumors selection

Background

In imaging-based clinical trials, it is common practice to perform double reads for each image, discrepant interpretations can result from these two different evaluations. In this study we analyzed discrepancies that occurred between local investigators (LI) and blinded independent central review (BICR) by comparing reader-selected imaging scans and lesions. Our goal was to identify the causes of discrepant declarations of progressive disease (PD) between LI and BICR in a clinical trial.

Methods

We retrospectively analyzed imaging data from a RECIST 1.1-based, multi-sites, phase II clinical trial of 179 patients with adult small cell lung cancer, treated with Cabazitaxel compared to Topotecan. Any discrepancies in the determination of PD between LI and BICR readers were reviewed by a third-party adjudicator. For each imaging time point and reader, we recorded the selected target lesions, non-target lesions, and new lesions. Odds ratios were calculated to measure the association between discrepant declarations of PD and the differences in reviewed imaging scans (e.g. same imaging modality but with different reconstruction parameters) and selected lesions. Reasons for discrepancies were analyzed.

Results

The average number of target lesions found by LI and BICR was respectively 2.9 and 3.4 per patient (p < 0.05), 18.4% of these target lesions were actually non-measurable. LI and BICR performed their evaluations based on different baseline imaging scans for 59% of the patients, they selected at least one different target lesion in 85% of patients. A total of 36.7% of patients required adjudication. Reasons of adjudication included differences in 1) reporting new lesions (53.7%), 2) the measured change of the tumor burden (18.5%), and 3) the progression of non-target lesions (11.2%). The rate of discrepancy was not associated with the selection of non-measurable target lesions or with the readers’ assessment of different images. Paradoxically, more discrepancies occurred when LI and BICR selected exactly the same target lesions at baseline compared to when readers selected not exactly the same lesions.

Conclusions

For a large proportion of evaluations, LI and BICR did not select the same imaging scans and target lesions but with a limited impact on the rate of discrepancy. The majority of discrepancies were explained by the difference in detecting new lesions.

Trial Registration

ARD12166 (https://clinicaltrials.gov/ct2/show/NCT01500720).

Validity of RECIST Version 1.1 for Response Assessment in Metastatic Cancer: A Prospective, Multireader Study

Purpose To determine the relationship between target lesion selection with use of Response Evaluation Criteria in Solid Tumors (RECIST) version 1.1 and classification of therapeutic response in patients with metastatic cancer undergoing systemic cytotoxic and/or targeted therapies. Materials and Methods This prospective multireader study was conducted between July 2015 and July 2017. Three hundred sixteen consecutive participants with metastatic cancer underwent 932 CT examinations to monitor systemic treatment. CT studies were independently read by three radiologists. Readers identified a maximum of five lesions total (and a maximum of two lesions per organ). Dedicated oncology tumor response software was used. The Fleiss κ statistic was used to analyze interreader agreement in the assignment of individual response classes (complete response, partial response, progressive disease, or stable disease) and in the differentiation between progressive and nonprogressive disease. Results Readers selected the same set of target lesions in 128 of the 316 participants (41%) and selected a different set in 188 (59%). When target lesion selection was concordant, agreement was high (assignment of treatment response category: κ = 0.97; 95% confidence interval [CI]: 0.91, 1.0; differentiation between progressive and nonprogressive disease: κ = 0.98; 95% CI: 0.90, 1.0). When target lesion selection was discordant, agreement was significantly reduced (assignment of treatment response category: κ = 0.58; 95% CI: 0.54, 0.62; differentiation between progressive and nonprogressive disease: κ = 0.6; 95% CI: 0.59, 0.70). With concordant target lesion selection, readers agreed regarding diagnosis of progression in 97.7% of participants (95% CI: 95.4%, 100.0%); with discordant target lesion selection, readers agreed in only 55.3% (95% CI: 47.9%, 62.6%) (P < .01). Conclusion In patients with metastatic cancer undergoing systemic treatment, different cancer sites may appear similarly suitable and thus likely to be selected as target lesions but may yield inconsistent or even conflicting results with Response Evaluation Criteria in Solid Tumors (RECIST) version 1.1. This indicates that the current, limited set of target lesions in RECIST 1.1 may not reflect overall tumor load or response to therapy. © RSNA, 2018 See also the editorial by Sosna in this issue.

Precision of manual two-dimensional segmentations of lung and liver metastases and its impact on tumour response assessment using RECIST 1.1

Background

Response evaluation criteria in solid tumours (RECIST) has significant limitations in terms of variability and reproducibility, which may not be independent. The aim of the study was to evaluate the precision of manual bi-dimensional segmentation of lung, liver metastases, and to quantify the uncertainty in tumour response assessment.

Methods

A total of 520 segmentations of metastases from six livers and seven lungs were independently performed by ten physicians and ten scientists on CT images, reflecting the variability encountered in clinical practice. Operators manually contoured the tumours, firstly independently according to the RECIST and secondly on a preselected slice. Diameters and areas were extracted from the segmentations. Mean standard deviations were used to build regression models and 95% confidence intervals (95% CI) were calculated for each tumour size and for limits of progressive disease (PD) and partial response (PR) derived from RECIST 1.1.

Results

Thirteen aberrant segmentations (2.5%) were observed without significant differences between the physicians and scientists; only the mean area of liver tumours (p = 0.034) and mean diameter of lung tumours (p = 0.021) differed significantly. No difference was observed between the methods. Inter-observer agreement was excellent (intra-class correlation >0.90) for all variables. In liver, overlaps of the 95% CI with the 95% CI of limits of PD or PR were observed for diameters above 22.7 and 37.9 mm, respectively. An overlap of 95% CIs was systematically observed for area. No overlaps were observed in lung.

Conclusions

Although the experience of readers might not affect the precision of segmentation in lung and liver, the results of manual segmentation performed for tumour response assessment remain uncertain for large liver metastases.

Electronic supplementary material

The online version of this article (doi:10.1186/s41747-017-0015-4) contains supplementary material, which is available to authorized users.

Evaluation of inter-observer variability according to RECIST 1.1 and its influence on response classification in CT measurement of liver metastases

OBJECTIVE

The aim of this study is the evaluation of inter-observer variability in the measurement of liver metastases according to RECIST and its influence on response classification.

PATIENTS AND METHODS

A total of 100 radiologists measured liver target lesions, on pre- and post-chemotherapy CT scans of three patients. Each observer filled out a questionnaire about his personal and work features. The evaluations of a well experienced radiologist, considered as "the gold standard", were compared to those taken by the observers. The percentage of the observers in agreement with the reviewer, in terms of the response category and in terms of inter-observer variability, was calculated for each patient.

RESULTS

The percentage of the inter-observer agreement was elevated. Most of the observers in agreement with the reviewer were senior radiologists, while those who disagreed were junior radiologist, but this result did not reach a statistical significance. More than 30% of observers disagreed with the reviewer at least in one of the three cases.

CONCLUSIONS

RECIST measurements are reproducible on a large and heterogeneous population of radiologists. Age and expertise of the radiologist remain the most critical factors: this suggests a revision by well-experienced radiologists in clinical trials.

Semi-automatic 3D-volumetry of liver metastases from neuroendocrine tumors to improve combination therapy with 177Lu-DOTATOC and 90Y-DOTATOC

PURPOSE

Patients with neuroendocrine tumors (NET) often present with disseminated liver metastases and can be treated with a number of different nuclides or nuclide combinations in peptide receptor radionuclide therapy (PRRT) depending on tumor load and lesion diameter. For quantification of disseminated liver lesions, semi-automatic lesion detection is helpful to determine tumor burden and tumor diameter in a time efficient manner. Here, we aimed to evaluate semi-automated measurement of total metastatic burden for therapy stratification.

METHODS

Nineteen patients with liver metastasized NET underwent contrast-enhanced 1.5 T MRI using gadolinium-ethoxybenzyl diethylenetriaminepentaacetic acid. Liver metastases (n=1537) were segmented using Fraunhofer MEVIS Software for three-dimensional (3D) segmentation. All lesions were stratified according to longest 3D diameter >20 mm or ≤20 mm and relative contribution to tumor load was used for therapy stratification.

RESULTS

Mean count of lesions ≤20 mm was 67.5 and mean count of lesions >20 mm was 13.4. However, mean contribution to total tumor volume of lesions ≤20 mm was 24%, while contribution of lesions >20 mm was 76%.

CONCLUSION

Semi-automatic lesion analysis provides useful information about lesion distribution in predominantly liver metastasized NET patients prior to PRRT. As conventional manual lesion measurements are laborious, our study shows this new approach is more efficient and less operator-dependent and may prove to be useful in the decision making process selecting the best combination PRRT in each patient.

Systematic analysis of measurement variability in lung cancer with multidetector computed tomography

OBJECTIVE:

To systematically analyze the nature of measurement variability in lung cancer with multidetector computed tomography (CT) scans.

METHODS:

Multidetector CT scans of 67 lung cancer patients were analyzed. Unidimensional (Response Evaluation Criteria in Solid Tumor criteria), bidimensional (World Health Organization criteria), and volumetric measurements were performed independently by ten radiologists and were repeated after at least 5 months. Repeatability and reproducibility measurement variations were estimated by analyzing reliability, agreement, variation coefficient, and misclassification statistically. The relationship of measurement variability with various sources was also analyzed.

RESULTS:

Analyses of 69 lung tumors with an average size of 1.1–12.1 cm (mean 4.3 cm) indicated that volumetric technique had the minimum measurement variability compared to the unidimensional or bidimensional technique. Tumor characteristics (object effect) could be the primary factor to influence measurement variability while the effect of raters (subjective effect) was faint. Segmentation and size in tumor characteristics were associated with measurement variability, and some mathematical function was established between the volumetric variability and tumor size.

CONCLUSION:

Volumetric technique has the minimum variability in measuring lung cancer, and measurement variability is associated with tumor size by nonlinear mathematical function.

From diagnosis to therapy in lung cancer: management of CT detected pulmonary nodules, a summary of the 2015 Chinese-German Lung Cancer Expert Panel

The first Chinese-German Lung Cancer Expert Panel was held in November 2015 one day after the 7th Chinese-German Lung Cancer Forum, Shanghai. The intention of the meeting was to discuss strategies for the diagnosis and treatment of lung cancer within the context of lung cancer screening. Improved risk classification criteria and novel imaging approaches for screening populations are highly required as more than half of lung cancer cases are false positive during the initial screening round if the National Lung Screening Trial (NLST) demographic criteria [≥30 pack years (PY) of cigarettes, age ≥55 years] are applied. Moreover, if the NLST criteria are applied to the Chinese population a high number of lung cancer patients are not diagnosed due to non-smoking related risk factors in China. The primary goal in the evaluation of pulmonary nodules (PN) is to determine whether they are malignant or benign. Volumetric based screening concepts such as investigated in the Dutch-Belgian randomized lung cancer screening trial (NELSON) seem to achieve higher specificity. Chest CT is the best imaging technique to identify the origin and location of the nodule since 20% of suspected PN found on chest X-ray turn out to be non-pulmonary lesions. Moreover, novel state-of-the-art CT systems can reduce the radiation dose for lung cancer screening acquisitions down to a level of 0.1 mSv with improved image quality to novel reconstruction techniques and thus reduce concerns related to chest CT as the primary screening technology. The aim of the first part of this manuscript was to summarize the current status of novel diagnostic techniques used for lung cancer screening and minimally invasive treatment techniques for progressive PNs that were discussed during the first Chinese-German Lung Cancer. This part should serve as an educational part for the readership of the techniques that were discussed during the Expert Panel. The second part summarizes the consensus recommendations that were interdisciplinary discussed by the Expert Panel.

Semiautomatic Analysis on Computed Tomography in Locally Advanced or Metastatic Non-Small Cell Lung Cancer: Reproducibility and Prognostic Significance of Unidimensional and 3-dimensional Measurements

PURPOSE

The aim of the study was to compare both reproducibility and prognostic value of lesion size measurements obtained manually and semiautomatically on computed tomography in advanced non-small cell lung cancer (NSCLC).

MATERIALS AND METHODS

Manual axial longest diameter, semiautomatic axial longest diameter, and volume of NSCLC lesions were independently analyzed by 4 readers at baseline and after at least 1 cycle of platinum-based chemotherapy. The prognostic value of the proportional change in lesion size between baseline and follow-up CT was evaluated using either RECIST or experimental thresholds derived from the quartiles of the changes as assessed manually or semiautomatically.

RESULTS

Semiautomatic axial longest diameter (concordance correlation coefficient [CCC]: 0.980 to 0.987; variation coefficient [VC%]: 6% to 7.3%) and volume (CCC: 0.974 to 0.991; VC%: 5.6% to 9.5%) were more reproducible than manual axial longest diameter (CCC: 0.950 to 0.984; VC%: 6.4% to 11.7%). RECIST categories did not stratify patients with different survival durations. For 3/4 readers, a decrease of ≤ 70% in lesion volume was associated with shorter survival (median survival: 11 mo, P < 0.05; hazard ratio: 5 to 22.2, P < 0.05).

CONCLUSIONS

In advanced NSCLC, semiautomatic measures were more reproducible than manual diameter, and volumetric measurement may better predict patient survival.

Comparison of CT volumetric measurement with RECIST response in patients with lung cancer

PURPOSE

To examine the correlations between uni-dimensional RECIST and volumetric measurements in patients with lung adenocarcinoma and to assess their association with overall survival (OS) and progression-free survival (PFS).

MATERIALS AND METHODS

In this study of patients receiving chemotherapy for lung cancer in the setting of a clinical trial, response was prospectively evaluated using RECIST 1.0. Retrospectively, volumetric measurements were recorded and response was assessed by two different volumetric methods at each followup CT scan using a semi-automated segmentation algorithm. We subsequently evaluated the correlation between the uni-dimensional RECIST measurements and the volumetric measurements and performed landmark analyses for OS and PFS at the completion of the first and second follow-ups. Kaplan-Meier curves together with log-rank tests were used to evaluate the association between the different response criteria and patient outcome.

RESULTS

Forty-two patients had CT scans at baseline, after the first follow up scan and second followup scan, and then every 8 weeks. The uni-dimensional RECIST measurements and volumetric measurements were strongly correlated, with a Spearman correlation coefficient (ρ) of 0.853 at baseline, ρ=0.861 at the first followup, ρ=0.843 at the 2nd followup, and ρ=0.887 overall between-subject. On first follow-up CT, partial responders and non responders as assessed by an "ellipsoid" volumetric criteria showed a significant difference in OS (p=0.008, 1-year OS of 70% for partial responders and 46% for non responders). There was no difference between the groups when assessed by RECIST criteria on first follow-up CT (p=0.841, 1-year OS rate of 64% for partial responders and 64% for non responders).

CONCLUSION

Volumetric response on first follow-up CT may better predict OS than RECIST response.

CLINICAL RELEVANCE STATEMENT

Assessment of tumor size and response is of utmost importance in clinical trials. Volumetric measurements may help to better predict OS than uni-dimensional RECIST criteria.

Observer variability in RECIST-based tumour burden measurements: a meta-analysis

BACKGROUND

Response Evaluation Criteria in Solid Tumours (RECIST)-based tumour burden measurements involve observer variability, the extent of which ought to be determined.

METHODS

A literature search identified studies on observer variability during manual measurements of tumour burdens via computed tomography according to the RECIST guideline. The 95% limit of agreement (LOA) values of relative measurement difference (RMD) were pooled using a random-effects model.

RESULTS

Twelve studies were included. Pooled 95% LOAs of RMD in measuring unidimensional longest diameters of single lesions ranged from -22.1% (95% confidence interval [CI], -30.3% to -14.0%) to 25.4% (95% CI, 17.2% to 33.5%) between observers and -17.8% (95% CI, -23.6% to -11.9%) to 16.1% (95% CI, 10.1% to 21.8%) for a single observer. Pooled 95% LOAs of RMD in measuring the sum of multiple lesions ranged from -19.2% (95% CI, -23.7% to -14.9%) to 19.5% (95% CI, 15.2% to 23.9%) between observers, and -9.8% (95% CI, -19.0% to -0.3%) to 13.1% (95% CI, 3.6% to 22.6%) for a single observer. Pooled 95% LOA of RMD in calculating the interval change of tumour burden with a single lesion ranged from -31.3% (95% CI, -46.0% to -16.5%) to 30.3% (95% CI, 15.3% to 44.8%) between observers. Studies on calculating the interval change of tumour burden for a single observer or with multiple lesions were lacking.

CONCLUSION

Interobserver RMD in measuring single tumour burden and calculating its interval change may exceed the 20% cut-off for progression. Variability decreased when tumour burden was measured by a single observer or assessed by the sum of multiple lesions.

Quantitative radiology: applications to oncology

Oncologists, clinician-scientists, and basic scientists collect computed tomography, magnetic resonance, and positron emission tomography images in the process of caring for patients, managing clinical trials, and investigating cancer biology. As we have developed more sophisticated means for noninvasively delineating and characterizing neoplasms, these image data have come to play a central role in oncology. In parallel, the increasing complexity and volume of these data have necessitated the development of quantitative methods for assessing tumor burden, and by proxy, disease-free survival.

Do Diametric Measurements Provide Sufficient and Reliable Tumor Assessment? An Evaluation of Diametric, Areametric, and Volumetric Variability of Lung Lesion Measurements on Computerized Tomography Scans

Diametric analysis is the standard approach utilized for tumor measurement on medical imaging. However, the availability of newer more sophisticated techniques may prove advantageous. An evaluation of diameter, area, and volume was performed on 64 different lung lesions by three trained users. These calculations were obtained using a free DICOM viewer and standardized measuring procedures. Measurement variability was then studied using relative standard deviation (RSD) and intraclass correlation. Volumetric measurements were shown to be more precise than diametric. With minimal RSD and variance between different users, volumetric analysis was demonstrated as a reliable measurement technique. Additionally, the diameters were used to calculate an estimated area and volume; thereafter the estimated area and volume were compared against the actual measured values. The results in this study showed independence of the estimated and actual values. Estimated area deviated an average of 43.5% from the actual measured, and volume deviated 88.03%. The range of this variance was widely scattered and without trend. These results suggest that diametric measurements cannot be reliably correlated to actual tumor size. Access to appropriate software capable of producing volume measurements has improved drastically and shows great potential in the clinical assessment of tumors. Its applicability merits further consideration.

Smaller but denser: postmortem changes alter the CT characteristics of subdural hematomas

PURPOSE

The aim of this study was to investigate if (1) the volume of subdural hematomas (SDH), midline shift, and CT density of subdural hematomas are altered by postmortem changes and (2) if these changes are dependent on the postmortem interval (PMI).

MATERIALS AND METHODS

Ante mortem computed tomography (AMCT) of the head was compared to corresponding postmortem CT (PMCT) in 19 adults with SDH. SDH volume, midline shift, and hematoma density were measured on both AMCT and PMCT and their differences assessed using Wilcoxon-Signed Rank Test. Spearman's Rho Test was used to assess significant correlations between the PMI and the alterations of SDH volume, midline shift, and hematoma density.

RESULTS

Mean time between last AMCT and PMCT was 109 h, mean PMI was 35 h. On PMCT mean midline displacement was decreased by 57% (p < 0.001); mean SDH volume was decreased by 38% (p < 0.001); and mean hematoma density was increased by 18% (p < 0.001) in comparison to AMCT. There was no correlation between the PMI and the normalization of the midline shift (p = 0.706), the reduction of SDH volume (p = 0.366), or the increase of hematoma density (p = 0.140).

CONCLUSIONS

This study reveals that normal postmortem changes significantly affect the extent and imaging characteristics of subdural hematoma and may therefore affect the interpretation of these findings on PMCT. Radiologists and forensic pathologists who use PMCT must be aware of these phenomena in order to correctly interpret PMCT findings in cases of subdural hemorrhages.

Assessing the usefulness of 18F-fluorodeoxyglucose PET-CT scan after stereotactic body radiotherapy for early-stage non-small cell lung cancer

BACKGROUND

Although stereotactic body radiation therapy (SBRT) is an established treatment option for early-stage lung cancer, there are no guidelines for reassessing patients for local treatment failure or intrathoracic recurrence after treatment. This study reports the sensitivity, specificity, and positive and negative predictive values for 18F-fluorodeoxyglucose (FDG) PET-CT scanning when used to evaluate patients after SBRT.

METHODS

Charts were reviewed of all patients who received SBRT and a subsequent FDG PET-CT scan at a university hospital over a 5-year period. Pretreatment and 3-month posttreatment tumor characteristics on PET-CT scan and outcome data (adverse events from SBRT, need for repeat biopsy, rate of local treatment failure and recurrent disease, and all-cause mortality) were recorded.

RESULTS

Eighty-eight patients were included in the study. Fourteen percent of patients (12 of 88) had positive 3-month PET scans. Of the positive results, 67% (eight of 12) were true positives. Eighty-six percent (76 of 88 patients) had negative 3-month FDG PET-CT scans, with 89% (68 of 76) true negatives. FDG PET-CT scan performed 3 months after SBRT for non-small cell lung cancer (NSCLC) had a sensitivity of 50% (95% CI, 0.26-0.75), a specificity of 94% (95% CI, 0.89-1.0), a positive predictive value of 67% (95% CI, 0.4-0.93), and a negative predictive value of 89% (95% CI, 0.83- 0.96).

CONCLUSIONS

FDG PET-CT scan 3 months after treatment of NSCLC with SBRT was a specific but insensitive test for the detection of recurrence or treatment failure. Serial CT scans should be used for early surveillance following SBRT, whereas FDG PET-CT scans should be reserved to define suspected metastatic disease or to evaluate new abnormalities on CT scan, or for possible reassessment later in the follow-up period after radiation-related inflammation subsides.