Intra- and inter-observer variability in measurement of target lesions: implication on response evaluation according to RECIST 1.1

Generalizability of lesion detection and segmentation when ScaleNAS is trained on a large multi-organ dataset and validated in the liver

BACKGROUND

Tumor assessment through imaging is crucial for diagnosing and treating cancer. Lesions in the liver, a common site for metastatic disease, are particularly challenging to accurately detect and segment. This labor-intensive task is subject to individual variation, which drives interest in automation using artificial intelligence (AI).

PURPOSE

Evaluate AI for lesion detection and lesion segmentation using CT in the context of human performance on the same task. Use internal testing to determine how an AI-developed model (ScaleNAS) trained on lesions in multiple organs performs when tested specifically on liver lesions in a dataset integrating real-world and clinical trial data. Use external testing to evaluate whether ScaleNAS's performance generalizes to publicly available colorectal liver metastases (CRLM) from The Cancer Imaging Archive (TCIA).

METHODS

The CUPA study dataset included patients whose CT scan of chest, abdomen, or pelvis at Columbia University between 2010-2020 indicated solid tumors (CUIMC, n = 5011) and from two clinical trials in metastatic colorectal cancer, PRIME (n = 1183) and Amgen (n = 463). Inclusion required ≥1 measurable lesion; exclusion criteria eliminated 1566 patients. Data were divided at the patient level into training (n = 3996), validation (n = 570), and testing (n = 1529) sets. To create the reference standard for training and validation, each case was annotated by one of six radiologists, randomly assigned, who marked the CUPA lesions without access to any previous annotations. For internal testing we refined the CUPA test set to contain only patients who had liver lesions (n = 525) and formed an enhanced reference standard through expert consensus reviewing prior annotations. For external testing, TCIA-CRLM (n = 197) formed the test set. The reference standard for TCIA-CRLM was formed by consensus review of the original annotation and contours by two new radiologists. Metrics for lesion detection were sensitivity and false positives. Lesion segmentation was assessed with median Dice coefficient, under-segmentation ratio (USR), and over-segmentation ratio (OSR). Subgroup analysis examined the influence of lesion size ≥ 10  mm (measurable by RECIST1.1) versus all lesions (important for early identification of disease progression).

RESULTS

ScaleNAS trained on all lesions achieved sensitivity of 71.4% and Dice of 70.2% for liver lesions in the CUPA internal test set (3,495 lesions) and sensitivity of 68.2% and Dice 64.2% in the TCIA-CRLM external test set (638 lesions). Human radiologists had mean sensitivity of 53.5% and Dice of 73.9% in CUPA and sensitivity of 84.1% and Dice of 88.4% in TCIA-CRLM. Performance improved for ScaleNAS and radiologists in the subgroup of lesions that excluded sub-centimeter lesions.

CONCLUSIONS

Our study presents the first evaluation of ScaleNAS in medical imaging, demonstrating its liver lesion detection and segmentation performance across diverse datasets. Using consensus reference standards from multiple radiologists, we addressed inter-observer variability and contributed to consistency in lesion annotation. While ScaleNAS does not surpass radiologists in performance, it offers fast and reliable results with potential utility in providing initial contours for radiologists. Future work will extend this model to lung and lymph node lesions, ultimately aiming to enhance clinical applications by generalizing detection and segmentation across tissue types.

Models and Biomarkers for Local Response Prediction in Early-Stage and Oligometastatic Non-small Cell Lung Cancer Patients Treated With Stereotactic Body Radiation Therapy Using Machine Learning

Background A minority of patients receiving stereotactic body radiation therapy (SBRT) for non-small cell lung cancer (NSCLC) are not good responders. Radiomic features can be used to generate predictive algorithms and biomarkers that can determine treatment outcomes and stratify patients to their therapeutic options. This study investigated and attempted to validate the radiomic and clinical features obtained from early-stage and oligometastatic NSCLC patients who underwent SBRT, to predict local response. Methodology A single-institution, Institutional Review Board (IRB)-approved retrospective review was conducted on adult patients with early-stage and oligometastatic SBRT-treated NSCLC at the Jewish General Hospital. The study included 98 patients (82 with early-stage NSCLC and 16 with oligometastatic disease), with a median age of 76 years and a male-to-female ratio of 46:52. A total of 116 lesions were treated with SBRT between 2009 and 2022. Radiomics features (n = 107) were extracted from CT planning scans using PyRadiomics, and clinical data were collected for all 98 patients. Local response was assessed according to Response Evaluation Criteria In Solid Tumors (RECIST 1.1) criteria. Classification models, including support vector machines, random forests, adaptive boosting, and multi-layer perceptrons (MLPs), were used. Models were trained using a fivefold cross-validation scheme. Their performances were measured with receiver operating characteristic plots on the validation folds. Using the importance of the permutation feature, predictive biomarkers were identified. Results The most predictive model, incorporating all patients and using an MLP classifier with Adaptive Synthetic (ADASYN) sampling, a combined-input approach, and a radiomic filter, achieved an area under the curve (AUC) of 0.94 ± 0.05. When oligometastatic patients were omitted, the best model (AUC 0.95 ± 0.06) was also predictive, using a support vector classification (SVC) radial basis function (RBF) classifier, ADASYN sampling, and a clinical-based input. Treatment site and performance status, along with radiomic features such as first-order root-mean-squared-intensity, first-order skewness, and gray-level nonuniformity, were found to be predictive biomarkers. Conclusions The predictive models generated and the biomarkers identified could be used in clinical decision support systems for SBRT-treated NSCLC patients. Additionally, treatment site, performance status, and radiomic features were the most predictive variables.

Reliability of Automated RECIST 1.1 and Volumetric RECIST Target Lesion Response Evaluation in Follow-Up CT-A Multi-Center, Multi-Observer Reading Study

OBJECTIVES

To evaluate the performance of a custom-made convolutional neural network (CNN) algorithm for fully automated lesion tracking and segmentation, as well as RECIST 1.1 evaluation, in longitudinal computed tomography (CT) studies compared to a manual Response Evaluation Criteria in Solid Tumors (RECIST 1.1) evaluation performed by three radiologists.

METHODS

Baseline and follow-up CTs of patients with stage IV melanoma (n = 58) was investigated in a retrospective reading study. Three radiologists performed manual measurements of metastatic lesions. Fully automated segmentations were generated, and diameters and volumes were computed from the segmentation results, with subsequent RECIST 1.1 evaluation. We measured (1) the intra- and inter-reader variability in the manual diameter measurements, (2) the agreement between manual and automated diameter measurements, as well as the resulting RECIST 1.1 categories, and (3) the agreement between the RECIST 1.1 categories derived from automated diameter measurement compared to automated volume measurements.

RESULTS

In total, 114 target lesions were measured at baseline and follow-up. The intraclass correlation coefficients (ICCs) for the intra- and inter-reader reliability of the diameter measurements were excellent, being >0.90 for all readers. There was moderate to almost perfect agreement when comparing the timepoint response category derived from the mean manual diameter measurements from all three readers with those derived from automated diameter measurements (Cohen's k 0.67-0.76). The agreement between the manual and automated volumetric timepoint responses was substantial (Fleiss' k 0.66-0.68) and that between the automated diameter and volume timepoint responses was substantial to almost perfect (Cohen's k 0.81).

CONCLUSIONS

The automated diameter measurement of preselected target lesions in follow-up CT is reliable and can potentially help to accelerate RECIST evaluation.

Feasibility of an artificial intelligence system for tumor response evaluation

PURPOSE

The objective of this study was to evaluate the feasibility of using Artificial Intelligence (AI) to measure the long-diameter of tumors for evaluating treatment response.

METHODS

Our study included 48 patients with lung-specific target lesions and conducted 277 measurements. The radiologists recorded the long-diameter in axial imaging plane of the target lesions for each measurement. Meanwhile, AI software was utilized to measure the long-diameter in both the axial imaging plane and in three dimensions (3D). Statistical analyses including the Bland-Altman plot, Spearman correlation analysis, and paired t-test to ascertain the accuracy and reliability of our findings.

RESULTS

The Bland-Altman plot showed that the AI measurements had a bias of -0.28 mm and had limits of agreement ranging from - 13.78 to 13.22 mm (P = 0.497), indicating agreement with the manual measurements. However, there was no agreement between the 3D measurements and the manual measurements, with P < 0.001. The paired t-test revealed no statistically significant difference between the manual measurements and AI measurements (P = 0.497), whereas a statistically significant difference was observed between the manual measurements and 3D measurements (P < 0.001).

CONCLUSIONS

The application of AI in measuring the long-diameter of tumors had significantly improved efficiency and reduced the incidence of subjective measurement errors. This advancement facilitated more convenient and accurate tumor response evaluation.

RECIST 1.1 assessments variability: a systematic pictorial review of blinded double reads

Reader variability is intrinsic to radiologic oncology assessments, necessitating measures to enhance consistency and accuracy. RECIST 1.1 criteria play a crucial role in mitigating this variability by standardizing evaluations, aiming to establish an accepted "truth" confirmed by histology or patient survival. Clinical trials utilize Blind Independent Centralized Review (BICR) techniques to manage variability, employing double reads and adjudicators to address inter-observer discordance effectively. It is essential to dissect the root causes of variability in response assessments, with a specific focus on the factors influencing RECIST evaluations. We propose proactive measures for radiologists to address variability sources such as radiologist expertise, image quality, and accessibility of contextual information, which significantly impact interpretation and assessment precision. Adherence to standardization and RECIST guidelines is pivotal in diminishing variability and ensuring uniform results across studies. Variability factors, including lesion selection, new lesion appearance, and confirmation bias, can have profound implications on assessment accuracy and interpretation, underscoring the importance of identifying and addressing these factors. Delving into the causes of variability aids in enhancing the accuracy and consistency of response assessments in oncology, underscoring the role of standardized evaluation protocols and mitigating risk factors that contribute to variability. Access to contextual information is crucial. CRITICAL RELEVANCE STATEMENT: By understanding the causes of diagnosis variability, we can enhance the accuracy and consistency of response assessments in oncology, ultimately improving patient care and clinical outcomes. KEY POINTS: Baseline lesion selection and detection of new lesions play a major role in the occurrence of discordance. Image interpretation is influenced by contextual information, the lack of which can lead to diagnostic uncertainty. Radiologists must be trained in RECIST criteria to reduce errors and variability.

Ovarian cancer beyond imaging: integration of AI and multiomics biomarkers

High-grade serous ovarian cancer is the most lethal gynaecological malignancy. Detailed molecular studies have revealed marked intra-patient heterogeneity at the tumour microenvironment level, likely contributing to poor prognosis. Despite large quantities of clinical, molecular and imaging data on ovarian cancer being accumulated worldwide and the rise of high-throughput computing, data frequently remain siloed and are thus inaccessible for integrated analyses. Only a minority of studies on ovarian cancer have set out to harness artificial intelligence (AI) for the integration of multiomics data and for developing powerful algorithms that capture the characteristics of ovarian cancer at multiple scales and levels. Clinical data, serum markers, and imaging data were most frequently used, followed by genomics and transcriptomics. The current literature proves that integrative multiomics approaches outperform models based on single data types and indicates that imaging can be used for the longitudinal tracking of tumour heterogeneity in space and potentially over time. This review presents an overview of studies that integrated two or more data types to develop AI-based classifiers or prediction models.Relevance statement Integrative multiomics models for ovarian cancer outperform models using single data types for classification, prognostication, and predictive tasks.Key points• This review presents studies using multiomics and artificial intelligence in ovarian cancer.• Current literature proves that integrative multiomics outperform models using single data types.• Around 60% of studies used a combination of imaging with clinical data.• The combination of genomics and transcriptomics with imaging data was infrequently used.

Discrepant Assessments of Progressive Disease in Clinical Trials between Routine Clinical Reads and Formal RECIST 1.1 Interpretations

Purpose To analyze the frequency of discrepant interpretations of progressive disease (PD) between routine clinical and formal Response Evaluation Criteria in Solid Tumors (RECIST) 1.1 interpretations in patients enrolled in solid tumor clinical trials and investigate the causes of discordance. Materials and Methods This retrospective study included patients in solid tumor clinical trials undergoing imaging response assessments based on RECIST 1.1 from January to July 2021. Routine clinical interpretations (RCIs) performed as part of standard workflow and not requiring formal use of any established response criteria were compared with separate local core laboratory interpretations (CLIs) by specially trained radiologists who used software that tracks target lesion measurements, changes in nontarget lesions, and appearance of new lesions longitudinally. The comparison focused on discordant interpretations of PD. Results Among 1053 patients who had both RCIs and CLIs performed, PD was diagnosed on one or both reads in 327 patients (median age, 63.6 [range, 22.4-83.2] years; 57.8% female patients). The RCIs and CLIs agreed with PD status in 65% (213 of 327) of assessments. In 32% (105 of 327) of assessments, RCIs overdiagnosed PD when CLIs diagnosed stable disease, and in 3% (nine of 327), CLIs diagnosed PD when RCIs diagnosed stable disease. Reasons for discrepant RCIs of PD included erroneous target lesion measurements (58%, 61 of 105), erroneous diagnosis of nontarget progression (30%, 32 of 105), and misclassification of new lesions as cancer (11%, 12 of 105). Most patients (93%, 98 of 105) with RCI overdiagnosis of PD remained in the clinical trial for one or more treatment cycles. Conclusion PD was frequently overdiagnosed on RCIs versus formal RECIST 1.1 CLIs which could result in patients removed from the clinical trial inappropriately. Keywords: Oncology, Cancer, Tumor Response, MR Imaging, CT © RSNA, 2023 See also commentary by Margolis and Ruchalski in this issue.

Performance of an automated registration-based method for longitudinal lesion matching and comparison to inter-reader variability

Objective.Patients with metastatic disease are followed throughout treatment with medical imaging, and accurately assessing changes of individual lesions is critical to properly inform clinical decisions. The goal of this work was to assess the performance of an automated lesion-matching algorithm in comparison to inter-reader variability (IRV) of matching lesions between scans of metastatic cancer patients.Approach.Forty pairs of longitudinal PET/CT and CT scans were collected and organized into four cohorts: lung cancers, head and neck cancers, lymphomas, and advanced cancers. Cases were also divided by cancer burden: low-burden (<10 lesions), intermediate-burden (10-29), and high-burden (30+). Two nuclear medicine physicians conducted independent reviews of each scan-pair and manually matched lesions. Matching differences between readers were assessed to quantify the IRV of lesion matching. The two readers met to form a consensus, which was considered a gold standard and compared against the output of an automated lesion-matching algorithm. IRV and performance of the automated method were quantified using precision, recall, F1-score, and the number of differences.Main results.The performance of the automated method did not differ significantly from IRV for any metric in any cohort (p> 0.05, Wilcoxon paired test). In high-burden cases, the F1-score (median [range]) was 0.89 [0.63, 1.00] between the automated method and reader consensus and 0.93 [0.72, 1.00] between readers. In low-burden cases, F1-scores were 1.00 [0.40, 1.00] and 1.00 [0.40, 1.00], for the automated method and IRV, respectively. Automated matching was significantly more efficient than either reader (p< 0.001). In high-burden cases, median matching time for the readers was 60 and 30 min, respectively, while automated matching took a median of 3.9 minSignificance.The automated lesion-matching algorithm was successful in performing lesion matching, meeting the benchmark of IRV. Automated lesion matching can significantly expedite and improve the consistency of longitudinal lesion-matching.

CT-based radiomics prediction of complete response after stereotactic body radiation therapy for patients with lung metastases

PURPOSE

Stereotactic body radiotherapy (SBRT) is a key treatment modality for lung cancer patients. This study aims to develop a machine learning-based prediction model of complete response for lung oligometastatic cancer patients undergoing SBRT.

MATERIALS AND METHODS

CT images of 80 pulmonary oligometastases from 56 patients treated with SBRT were analyzed. The gross tumor volumes (GTV) were contoured on CT images. Patients that achieved complete response (CR) at 4 months were defined as responders. For each GTV, 107 radiomic features were extracted using the Pyradiomics software. The concordance correlation coefficients (CCC) between the region of interest (ROI)-based radiomics features obtained by the two segmentations were calculated. Pairwise feature interdependencies were evaluated using the Spearman rank correlation coefficient. The association of clinical variables and radiomics features with CR was evaluated with univariate logistic regression. Two supervised machine learning models, the logistic regression (LR) and the classification and regression tree analysis (CART), were trained to predict CR. The models were cross-validated using a five-fold cross-validation. The performance of models was assessed by receiver operating characteristic curve (ROC) and class-specific accuracy, precision, recall, and F1-measure evaluation metrics.

RESULTS

Complete response was associated with four radiomics features, namely the surface to volume ratio (SVR; p = 0.003), the skewness (Skew; p = 0.027), the correlation (Corr; p = 0.024), and the grey normalized level uniformity (GNLU; p = 0.015). No significant relationship between clinical parameters and CR was found. In the validation set, the developed LR and CART machine learning models had an accuracy, precision, and recall of 0.644 and 0.750, 0.644 and 0.651, and 0.635 and 0.754, respectively. The area under the curve for CR prediction was 0.707 and 0.753 for the LR and CART models, respectively.

CONCLUSION

This analysis demonstrates that radiomics features obtained from pretreatment CT could predict complete response of lung oligometastases following SBRT.

Elevated pretreatment neutrophil-to-lymphocyte ratio indicate low survival rate in apatinib-treated patients with non-small cell lung cancer: A STROBE-compliant article

This study aimed to analyze the predictive value of the neutrophil-to-lymphocyte ratio (NLR) to better clarify which patients with advanced non-small cell lung cancer (NSCLC) would benefit most from apatinib after multiline treatment for drug resistance. This observational cohort study involved patients with advanced NSCLC who were treated with apatinib between May 2016 to May 2018. The participants in this study had previously been treated with at least two treatment regimens. Multivariate logistic regression and Cox proportional risk models were used to evaluate the overall survival (OS) and progression-free survival (PFS) of the pretreatment NLR. A total of 125 patients were reviewed. The median age was 64 years (range, 33-92); and 32.8% of the patients were female. Only 0.8% of the patients had an Eastern Cooperative Oncology Group Performance Status (ECOG-PS) score ≥ 2. In multivariate analysis, pretreatment NLR ≥ 5 had an independent correlation with inferior OS (median 2.07 vs 3.40 months; HR 1.493, 95% CI 1.022-2.182; P = .038) and inferior PFS (median 1.83 vs 2.76 months; HR 1.478, 95% CI 1.015-2.153; P = .042). Elevated pretreatment NLR is associated with shorter OS and PFS in patients with advanced NSCLC treated with apatinib after multiline treatment for drug resistance.

Comparison of gross pathology inspection and 9.4 T magnetic resonance imaging in the evaluation of radiofrequency ablation lesions in the left ventricle of the swine heart

Aims: Gross pathology inspection (patho) is the gold standard for the morphological evaluation of focal myocardial pathology. Examination with 9.4 T magnetic resonance imaging (MRI) is a new method for very accurate display of myocardial pathology. The aim of this study was to demonstrate that lesions can be measured on high-resolution MRI images with the same accuracy as on pathological sections and compare these two methods for the evaluation of radiofrequency (RF) ablation lesion dimensions in swine heart tissue during animal experiment.

Methods: Ten pigs underwent radiofrequency ablations in the left ventricle during animal experiment. After animal euthanasia, hearts were explanted, flushed with ice-cold cardioplegic solution to relax the whole myocardium, fixed in 10% formaldehyde and scanned with a 9.4 T magnetic resonance system. Anatomical images were processed using ImageJ software. Subsequently, the hearts were sliced, slices were photographed and measured in ImageJ software. Different dimensions and volumes were compared.

Results: The results of both methods were statistically compared. Depth by MRI was 8.771 ± 2.595 mm and by patho 9.008 ± 2.823 mm; p = 0.198. Width was 10.802 ± 2.724 mm by MRI and 11.125 ± 2.801 mm by patho; p = 0.049. Estuary was 2.006 ± 0.867 mm by MRI and 2.001 ± 0.872 mm by patho; p = 0.953. The depth at the maximum diameter was 4.734 ± 1.532 mm on MRI and 4.783 ± 1.648 mm from the patho; p = 0.858. The volumes of the lesions calculated using a formula were 315.973 ± 257.673 mm³ for MRI and 355.726 ± 255.860 mm³ for patho; p = 0.104. Volume directly measured from MRI with the “point-by-point” method was 671.702 ± 362.299 mm³.

Conclusion: Measurements obtained from gross pathology inspection and MRI are fully comparable. The advantage of MRI is that it is a non-destructive method enabling repeated measurements in all possible anatomical projections.

iRECIST-based versus non-standardized free text reporting of CT scans for monitoring metastatic renal cell carcinoma: a retrospective comparison

Purpose

Therapy decision for patients with metastatic renal cell carcinoma (mRCC) is highly dependent on disease monitoring based on radiological reports. The purpose of the study was to compare non-standardized, common practice free text reporting (FTR) on disease response with reporting based on response evaluation criteria in solid tumors modified for immune-based therapeutics (iRECIST).

Methods

Fifty patients with advanced mRCC were included in the retrospective, single-center study. CT scans had been evaluated and FTR prepared in accordance with center’s routine practice. For study purposes, reports were re-evaluated using a dedicated computer program that applied iRECIST. Patients were followed up over a period of 22.8 ± 7.9 months in intervals of 2.7 ± 1.8 months.

Weighted kappa statistics was run to assess strength of agreement. Logistic regression was used to identify predictors for different rating.

Results

Agreement between FTR and iRECIST-based reporting was moderate (kappa 0.38 [95% CI 0.2–0.6] to 0.70 [95% CI 0.5–0.9]). Tumor response or progression according to FTR were not confirmed with iRECIST in 19 (38%) or 11 (22%) patients, respectively, in at least one follow-up examination. With FTR, new lesions were frequently not recognized if they were already identified in the recent prior follow-up examination (odds ratio for too favorable rating of disease response compared to iRECIST: 5.4 [95% CI 2.9–10.1].

Conclusions

Moderate agreement between disease response according to FTR or iRECIST in patients with mRCC suggests the need of standardized quantitative radiological assessment in daily clinical practice.

Supplementary Information

The online version contains supplementary material available at 10.1007/s00432-022-03997-0.

Interobserver and intraobserver variability of RECIST assessment in ovarian cancer

OBJECTIVES

Measurement of Response Evaluation Criteria In Solid Tumors (RECIST) relies on reproducible unidimensional tumor measurements. This study assessed intraobserver and interobserver variability of target lesion selection and measurement, according to RECIST version 1.1 in patients with ovarian cancer.

METHODS

Eight international radiologists independently viewed 47 images demonstrating malignant lesions in patients with ovarian cancer and selected and measured lesions according to RECIST V.1.1 criteria. Thirteen images were viewed twice. Interobserver variability of selection and measurement were calculated for all images. Intraobserver variability of selection and measurement were calculated for images viewed twice. Lesions were classified according to their anatomical site as pulmonary, hepatic, pelvic mass, peritoneal, lymph nodal, or other. Lesion selection variability was assessed by calculating the reproducibility rate. Lesion measurement variability was assessed with the intra-class correlation coefficient.

RESULTS

From 47 images, 82 distinct lesions were identified. For lesion selection, the interobserver and intraobserver reproducibility rates were high, at 0.91 and 0.93, respectively. Interobserver selection reproducibility was highest (reproducibility rate 1) for pelvic mass and other lesions. Intraobserver selection reproducibility was highest (reproducibility rate 1) for pelvic mass, hepatic, nodal, and other lesions. Selection reproducibility was lowest for peritoneal lesions (interobserver reproducibility rate 0.76 and intraobserver reproducibility rate 0.69). For lesion measurement, the overall interobserver and intraobserver intraclass correlation coefficients showed very good concordance of 0.84 and 0.94, respectively. Interobserver intraclass correlation coefficient showed very good concordance for hepatic, pulmonary, peritoneal, and other lesions, and ranged from 0.84 to 0.97, but only moderate concordance for lymph node lesions (0.58). Intraobserver intraclass correlation coefficient showed very good concordance for all lesions, ranging from 0.82 to 0.99. In total, 85% of total measurement variability resulted from interobserver measurement difference.

CONCLUSIONS

Our study showed that while selection and measurement concordance were high, there was significant interobserver and intraobserver variability. Most resulted from interobserver variability. Compared with other lesions, peritoneal lesions had the lowest selection reproducibility, and lymph node lesions had the lowest measurement concordance. These factors need consideration to improve response assessment, especially as progression free survival remains the most common endpoint in phase III trials.

Combining computed tomography and biologically effective dose in radiomics and deep learning improves prediction of tumor response to robotic lung stereotactic body radiation therapy

PURPOSE

The aim of this study is to improve the performance of machine learning (ML) models in predicting response of non-small cell lung cancer (NSCLC) to stereotactic body radiation therapy (SBRT) by integrating image features from pre-treatment computed tomography (CT) with features from the biologically effective dose (BED) distribution.

MATERIALS AND METHODS

Image features, consisting of crafted radiomic features or machine-learned features extracted using a convolutional neural network, were calculated from pre-treatment CT data and from dose distributions converted into BED for 80 NSCLC lesions over 76 patients treated with robotic guided SBRT. ML models using different combinations of features were trained to predict complete or partial response according to response criteria in solid tumors, including radiomics CT (Rad_CT ), radiomics CT and BED (Rad_CT,BED ), deep learning (DL) CT (DL_CT ), and DL CT and BED (DL_CT,BED ). Training of ML included feature selection by neighborhood component analysis followed by ensemble ML using robust boosting. A model was considered as acceptable when the sum of average sensitivity and specificity on test data in repeated cross validations was at least 1.5.

RESULTS

Complete or partial response occurred in 58 out of 80 lesions. The best models to predict the tumor response were those using BED variables, achieving significantly better area under curve (AUC) and accuracy than those using only features from CT, including a Rad_CT,BED model using three radiomic features from BED, which scored an accuracy of 0.799 (95% confidence intervals (0.75-0.85)) and AUC of 0.773 (0.688-0.846), and a DL_CT,BED model also using three variables with an accuracy of 0.798 (0.649-0.829) and AUC of 0.812 (0.755-0.867).

CONCLUSION

According to our results, the inclusion of BED features improves the response prediction of ML models for lung cancer patients undergoing SBRT, regardless of the use of radiomic or DL features.

RECIST 1.1 and lesion selection: How to deal with ambiguity at baseline?

Response Evaluation Criteria In Solid Tumors (RECIST) is still the predominant criteria base for assessing tumor burden in oncology clinical trials. Despite several improvements that followed its first publication, RECIST continues to allow readers a lot of freedom in their evaluations. Notably in the selection of tumors at baseline. This subjectivity is the source of many suboptimal evaluations. When starting a baseline analysis, radiologists cannot always identify tumor malignancy with any certainty. Also, with RECIST, some findings can be deemed equivocal by radiologists with no confirmatory ground truth to rely on. In the specific case of Blinded Independent Central Review clinical trials with double reads using RECIST, the selection of equivocal tumors can have two major consequences: inter-reader variability and modified sensitivity of the therapeutic response. Apart from the main causes leading to the selection of an equivocal lesion, due to the uncertainty of the radiological characteristics or due to the censoring of on-site evaluations, several other situations can be described more precisely. These latter involve cases where an equivocal is selected as target or non-target lesions, the management of equivocal lymph nodes and the case of few target lesions. In all cases, awareness of the impact of selecting a non-malignant lesion will lead radiologists to make selections in the most rational way. Also, in clinical trials where the primary endpoint differs between phase 2 (response-related) and phase 3 (progression-related) trials, our impact analysis will help them to devise strategies for the management of equivocal lesions.

Radiologists and Clinical Trials: Part 1 The Truth About Reader Disagreements

The debate over human visual perception and how medical images should be interpreted have persisted since X-rays were the only imaging technique available. Concerns over rates of disagreement between expert image readers are associated with much of the clinical research and at times driven by the belief that any image endpoint variability is problematic. The deeper understanding of the reasons, value, and risk of disagreement are somewhat siloed, leading, at times, to costly and risky approaches, especially in clinical trials. Although artificial intelligence promises some relief from mistakes, its routine application for assessing tumors within cancer trials is still an aspiration. Our consortium of international experts in medical imaging for drug development research, the Pharma Imaging Network for Therapeutics and Diagnostics (PINTAD), tapped the collective knowledge of its members to ground expectations, summarize common reasons for reader discordance, identify what factors can be controlled and which actions are likely to be effective in reducing discordance. Reinforced by an exhaustive literature review, our work defines the forces that shape reader variability. This review article aims to produce a singular authoritative resource outlining reader performance's practical realities within cancer trials, whether they occur within a clinical or an independent central review.

Deep-learning approach with convolutional neural network for classification of maximum intensity projections of dynamic contrast-enhanced breast magnetic resonance imaging

PURPOSE

We aimed to evaluate deep learning approach with convolutional neural networks (CNNs) to discriminate between benign and malignant lesions on maximum intensity projections of dynamic contrast-enhanced breast magnetic resonance imaging (MRI).

METHODS

We retrospectively gathered maximum intensity projections of dynamic contrast-enhanced breast MRI of 106 benign (including 22 normal) and 180 malignant cases for training and validation data. CNN models were constructed to calculate the probability of malignancy using CNN architectures (DenseNet121, DenseNet169, InceptionResNetV2, InceptionV3, NasNetMobile, and Xception) with 500 epochs and analyzed that of 25 benign (including 12 normal) and 47 malignant cases for test data. Two human readers also interpreted these test data and scored the probability of malignancy for each case using Breast Imaging Reporting and Data System. Sensitivity, specificity, accuracy, and area under the receiver operating characteristic curve (AUC) were calculated.

RESULTS

The CNN models showed a mean AUC of 0.830 (range, 0.750-0.895). The best model was InceptionResNetV2. This model, Reader 1, and Reader 2 had sensitivities of 74.5%, 72.3%, and 78.7%; specificities of 96.0%, 88.0%, and 80.0%; and AUCs of 0.895, 0.823, and 0.849, respectively. No significant difference arose between the CNN models and human readers (p > 0.125).

CONCLUSION

Our CNN models showed comparable diagnostic performance in differentiating between benign and malignant lesions to human readers on maximum intensity projection of dynamic contrast-enhanced breast MRI.

Volumetric tumor response assessment is inefficient without overt clinical benefit compared to conventional, manual veterinary response assessment in canine nasal tumors

Accurate assessment of tumor response to therapy is critical in guiding management of veterinary oncology patients and is most commonly performed using response evaluation criteria in solid tumors criteria. This process can be time consuming and have high intra- and interobserver variability. The primary aim of this serial measurements, secondary analysis study was to compare manual linear tumor response assessment to semi-automated, contoured response assessment in canine nasal tumors. The secondary objective was to determine if tumor measurements or clinical characteristics, such as stage, would correlate to progression-free interval. Three investigators evaluated paired CT scans of skulls of 22 dogs with nasal tumors obtained prior to and following radiation therapy. The automatically generated tumor volumes were not useful for canine nasal tumors in this study, characterized by poor intraobserver agreement between automatically generated contours and hand-adjusted contours. The radiologist's manual linear method of determining response evaluation criteria in solid tumors categorization and tumor volume is significantly faster (P < .0001) but significantly underestimates nasal tumor volume (P < .05) when compared to a contour-based method. Interobserver agreement was greater for volume determination using the contour-based method when compared to response evaluation criteria in solid tumors categorization utilizing the same method. However, response evaluation criteria in solid tumors categorization and percentage volume change were strongly correlated, providing validity to response evaluation criteria in solid tumors as a rapid method of tumor response assessment for canine nasal tumors. No clinical characteristics or tumor measurements were significantly associated with progression-free interval.

Efficient Anomaly Detection with Generative Adversarial Network for Breast Ultrasound Imaging

We aimed to use generative adversarial network (GAN)-based anomaly detection to diagnose images of normal tissue, benign masses, or malignant masses on breast ultrasound. We retrospectively collected 531 normal breast ultrasound images from 69 patients. Data augmentation was performed and 6372 (531 × 12) images were available for training. Efficient GAN-based anomaly detection was used to construct a computational model to detect anomalous lesions in images and calculate abnormalities as an anomaly score. Images of 51 normal tissues, 48 benign masses, and 72 malignant masses were analyzed for the test data. The sensitivity, specificity, and area under the receiver operating characteristic curve (AUC) of this anomaly detection model were calculated. Malignant masses had significantly higher anomaly scores than benign masses (p < 0.001), and benign masses had significantly higher scores than normal tissues (p < 0.001). Our anomaly detection model had high sensitivities, specificities, and AUC values for distinguishing normal tissues from benign and malignant masses, with even greater values for distinguishing normal tissues from malignant masses. GAN-based anomaly detection shows high performance for the detection and diagnosis of anomalous lesions in breast ultrasound images.

Evaluation of the Efficacy of Neoadjuvant Chemotherapy for Breast Cancer

Neoadjuvant chemotherapy is increasingly used in breast cancer, especially for downstaging the primary tumor in the breast and the metastatic axillary lymph node. Accurate evaluations of the response to neoadjuvant chemotherapy provide important information on the impact of systemic therapies on breast cancer biology, prognosis, and guidance for further therapy. Moreover, pathologic complete response is a validated and valuable surrogate prognostic factor of survival after therapy. Evaluations of neoadjuvant chemotherapy response are very important in clinical work and basic research. In this review, we will elaborate on evaluations of the efficacy of neoadjuvant chemotherapy in breast cancer and provide a clinical evaluation procedure for neoadjuvant chemotherapy.

Multi-objective Parameter Auto-tuning for Tissue Image Segmentation Workflows

We propose a software platform that integrates methods and tools for multi-objective parameter auto-tuning in tissue image segmentation workflows. The goal of our work is to provide an approach for improving the accuracy of nucleus/cell segmentation pipelines by tuning their input parameters. The shape, size, and texture features of nuclei in tissue are important biomarkers for disease prognosis, and accurate computation of these features depends on accurate delineation of boundaries of nuclei. Input parameters in many nucleus segmentation workflows affect segmentation accuracy and have to be tuned for optimal performance. This is a time-consuming and computationally expensive process; automating this step facilitates more robust image segmentation workflows and enables more efficient application of image analysis in large image datasets. Our software platform adjusts the parameters of a nuclear segmentation algorithm to maximize the quality of image segmentation results while minimizing the execution time. It implements several optimization methods to search the parameter space efficiently. In addition, the methodology is developed to execute on high-performance computing systems to reduce the execution time of the parameter tuning phase. These capabilities are packaged in a Docker container for easy deployment and can be used through a friendly interface extension in 3D Slicer. Our results using three real-world image segmentation workflows demonstrate that the proposed solution is able to (1) search a small fraction (about 100 points) of the parameter space, which contains billions to trillions of points, and improve the quality of segmentation output by × 1.20, × 1.29, and × 1.29, on average; (2) decrease the execution time of a segmentation workflow by up to 11.79× while improving output quality; and (3) effectively use parallel systems to accelerate parameter tuning and segmentation phases.

Interobserver Variation in Response Evaluation Criteria in Solid Tumors 1.1

PURPOSE

Response Evaluation Criteria in Solid Tumors (RECIST 1.1) is the gold standard for imaging response evaluation in cancer trials. We sought to evaluate consistency of applying RECIST 1.1 between 2 conventionally trained radiologists, designated as A and B; identify reasons for variation; and reconcile these differences for future studies.

METHODS

The study was approved as an institutional quality check exercise. Since no identifiable patient data was collected or used, a waiver of informed consent was granted. Imaging case report forms of a concluded multicentric breast cancer trial were retrospectively reviewed. Cohen's kappa was used to rate interobserver agreement in Response Evaluation Data (target response, nontarget response, new lesions, overall response). Significant variations were reassessed by a senior radiologist to extrapolate reasons for disagreement. Methods to improve agreement were similarly ascertained.

RESULTS

Sixty one cases with total of 82 data-pairs were evaluated (35 data-pairs in visit 5, 47 in visit 9). Both radiologists showed moderate agreement in target response (n = 82; ĸ = 0.477; 95% confidence interval [CI]: 0.314-0.640-), nontarget response (n = 82; ĸ = 0.578; 95% CI: 0.213-0.944) and overall response evaluation in both visits (n = 82; ĸ = 0.510; 95% CI: 0.344-0.676). Further assessment demonstrated "Prevalence effect" of Kappa in some cases which led to underestimation of agreement. Percent agreement of overall response was 74.39% while percent variation was 25.6%. Differences in interpreting RECIST 1.1 and in radiological image interpretation were the primary sources of variation. The commonest overall response was "Partial Response" (Rad A:45/82; Rad B:63/82).

CONCLUSION

Inspite of moderate interobserver agreement, qualitative interpretation differences in some cases increased interobserver variability. Protocols such as Adjudication, to reduce easily avoidable inconsistencies are or should be a part of the Standard Operating Procedure in imaging institutions. Based on our findings, a standard checklist has been developed to help reduce the interpretation error-margin for future studies. Such check-lists may improve interobserver agreement in the preadjudication phase thereby improving quality of results and reducing adjudication per case ratio.

CLINICAL RELEVANCE

Improving data reliability when using RECIST 1.1 will reflect in better cancer clinical trial outcomes. A checklist can be of use to imaging centers to assess and improve their own processes.

Circulating Biomarkers for Prediction of Objective Response to Chemotherapy in Pancreatic Cancer Patients

Pancreatic cancer is a lethal disease with increasing incidence. Most patients present with advanced disease, for which palliative systemic chemotherapy is the only therapeutic option. Despite improved median survival rates with FOLFIRINOX or gemcitabine chemotherapy compared to the best supportive care, many individual patients may not benefit from chemotherapy. Biomarkers are needed to predict who will benefit from chemotherapy and to monitor a patient’s response to chemotherapy. This review summarizes current research and future perspectives on circulating biomarkers for systemic chemotherapy response.

Correlation of patient survival with clinical tumor measurements in malignant pleural mesothelioma

OBJECTIVES

To evaluate differences in the tumor response classifications that result from clinical measurements and to compare these response classifications with overall survival for patients with malignant pleural mesothelioma (MPM).

METHODS

One hundred thirty-one computed tomography (CT) scans were collected from 41 MPM patients enrolled in a clinical trial. Primary measurements had been acquired by clinical radiologists at a single center during routine clinical workflow, and the variability of these measurements was investigated. Retrospective measurements were acquired by a single radiologist in compliance with the study protocol based on the modified response evaluation criteria in solid tumors (RECIST). Differences in response classification categories by the two measurement approaches were evaluated and compared with patient survival.

RESULTS

Eleven (27%) of the 41 MPM patients had primary measurements at baseline or at follow-up that deviated from the guidelines of the clinical trial protocol. Among the 41 baseline scans, no statistical difference was observed in summed tumor measurements between primary and retrospective measurements. Response classification based on primary and retrospective measurements was different in 23 (26%) of the 90 follow-up scans, and best response was the different in seven (17%) of the 41 patients. Using Harrell's C statistic as a measure of correlation, response based on retrospective measurements correlated better with survival (C = 0.62) than did response based on primary measurements (C = 0.57).

CONCLUSIONS

Strict compliance with the measurement protocol yields tumor response classifications that may differ from those obtained in clinical practice. Response based on retrospective measurements correlated better with survival than did response based on primary measurements.

KEY POINTS

• Response classifications could be different between clinical primary and retrospective measurements for malignant pleural mesothelioma. • Response classifications obtained by strict compliance with the trial-specific protocol correlated better with survival than the classifications based on primary measurements. • Quality assurance and radiologist training measures should be used to ensure the integrity of image-based tumor measurements in mesothelioma clinical trials.

Inter-observer variability influences the Lugano classification when restaging lymphoma

Background

Lymphoma is an important and potentially curable oncological disease in South Africa. The staging and restaging of lymphoma have evolved over the years, with the latest international consensus guideline being the Lugano classification (LC). Prior to routine implementation of the LC, its robustness in the local setting should be determined.

Objectives

To determine the Inter-observer variability in response assignment when applying the LC in patients with lymphoma who were staged and restaged with computed tomography. In case of excessive discordance, specific mitigating measures will have to be taken before and during any proposed implementation of the LC.

Method

A total of 61 computed tomography scans in 21 patients were evaluated independently by four reviewers according to the LC, of which 21 scans were done at baseline, 21 at initial restaging and 19 at follow-up restaging. A retrospective comparative analysis was performed. Kappa values were calculated to determine agreement between observers.

Results

Only a moderate inter-observer agreement of 52% in the overall response classification was demonstrated. The most important sources of discrepancy were inconsistency in the assessment of target lesion regression to normal, determining the percentage change in the summed cross-sectional area of the target lesions and ascribing new lesions as either due to lymphoma or other causes.

Conclusion

Implementing the Lugano classification when restaging lymphoma is desirable to improve consistency and to conform to international guidelines. However, our study shows substantial inter-observer variability in response classification, potentially altering the treatment plan. Dedicated training and continuous quality control should, therefore, accompany the process.

Improvement of radiology reporting in a clinical cancer network: impact of an optimised multidisciplinary workflow

PURPOSE

To assess the effectiveness of implementing a quality improvement project in a clinical cancer network directed at the response assessment of oncology patients according to RECIST-criteria.

METHODS

Requests and reports of computed tomography (CT) studies from before (n = 103) and after (n = 112) implementation of interventions were compared. The interventions consisted of: a multidisciplinary working agreement with a clearly described workflow; subspecialisation of radiologists; adaptation of the Picture Archiving and Communication System (PACS); structured reporting.

RESULTS

The essential information included in the requests and the reports improved significantly after implementation of the interventions. In the requests, mentioning start date increased from 2% to 49%; date of baseline CT from 7% to 64%; nadir date from 1% to 41%. In the reports, structured layout increased from 14% to 86%; mentioning target lesions from 18% to 80% and non-target lesions from 11% to 80%; measurements stored in PACS increased from 76% to 97%; labelled key images from 38% to 95%; all p values < 0.001.

CONCLUSION

The combination of implementation of an optimised workflow, subspecialisation and structured reporting led to significantly better quality radiology reporting for oncology patients receiving chemotherapy. The applied multifactorial approach can be used within other radiology subspeciality areas as well.

KEY POINTS

• Undeveloped subspecialisation makes adherence to RECIST guidelines difficult in general hospitals. • A clinical cancer network provides opportunities to improve healthcare. • Optimised workflow, subspecialisation and structured reporting substantially improve request and report quality. • Good interdisciplinary communication between oncologists, radiologists and others contributes to quality improvement.

Intraperitoneal chemotherapy for ovarian cancer using sustained-release implantable devices

INTRODUCTION

Epithelial ovarian cancer (EOC) remains to be the most lethal of all gynecological malignancies mainly due to its asymptomatic nature. The late stages are manifested with predominant metastases confined to the peritoneal cavity. Although there has been a substantial progress in the treatment avenue with different therapeutic interventions, the overall survival rate of patients remain poor due to relapse and drug resistance.

AREAS COVERED

The pharmacokinetic advantages offered by intraperitoneal (IP) chemotherapy due to peritoneal-plasma barrier can be potentially exploited for EOC relapse treatment. The ability to retain high concentrations of chemo-drugs with high AUC peritoneum/plasma for prolonged durations in the peritoneal cavity can be utilized effectively through the clinical adoption of drug delivery systems (DDSs) which obviates the need for indwelling catheters. The metronomic dosing strategy could enhance anti-tumor efficacy with a continuous, low dose of chemo-drugs providing minimal systemic toxicity.

EXPERT OPINION

The development of a feasible, non-catheter based, IP DDS, retaining the peritoneal-drug levels, with less systemic levels could offer significant survival advantages as a patient-compliant therapeutic strategy. Suturable-implantable devices based on metronomic dosing, eluting drug in a sustained manner at low doses, could be implanted surgically post-debulking for treatment of refractory EOC patients.

Are we reproducible in measurement of NET liver metastasis?

Accurate measurement of well-differentiated neuroendocrine tumours (NET) liver metastases is critical to determine tumour slope and to assess treatment efficacy. Our objectives were to determine which CT or MRI sequence is the most reproducible to measure NET liver metastases and to assess the percentage of variability of measurements. Intra and inter-observer variability were studied on triphasic abdominal CT or liver MRI in 22 and 32 NET patients respectively. Patients were treatment-naïve or under somatostatin analogues. A maximum of 5 liver target lesions per patient was defined and three radiologists measured them on each sequence. Reproducibility were analysed by calculating the relative variation (RV) as defined by RECIST criteria. We analysed 1656 target measurements for CT and 3384 for MRI. Intra-observers RV were better than inter-observers. T2 for MRI and portal-phase for CT were associated with the lowest measurement variability. The MRI sequence offering the best intra and inter-observer reproducibility is the T2W-sequence. MRI allows more reproducible measurement than CT (inter-observer RV <20% in 96.8% for MRI and 81% for CT). Our study demonstrates intermediate to high imaging reproducibility of liver metastases measurements in NET patients. Non-enhanced MRI should be preferred to triphasic-CT for follow-up, assessment of treatment and trials.

Observer variability in RECIST-based tumour burden measurements: a meta-analysis

BACKGROUND

Response Evaluation Criteria in Solid Tumours (RECIST)-based tumour burden measurements involve observer variability, the extent of which ought to be determined.

METHODS

A literature search identified studies on observer variability during manual measurements of tumour burdens via computed tomography according to the RECIST guideline. The 95% limit of agreement (LOA) values of relative measurement difference (RMD) were pooled using a random-effects model.

RESULTS

Twelve studies were included. Pooled 95% LOAs of RMD in measuring unidimensional longest diameters of single lesions ranged from -22.1% (95% confidence interval [CI], -30.3% to -14.0%) to 25.4% (95% CI, 17.2% to 33.5%) between observers and -17.8% (95% CI, -23.6% to -11.9%) to 16.1% (95% CI, 10.1% to 21.8%) for a single observer. Pooled 95% LOAs of RMD in measuring the sum of multiple lesions ranged from -19.2% (95% CI, -23.7% to -14.9%) to 19.5% (95% CI, 15.2% to 23.9%) between observers, and -9.8% (95% CI, -19.0% to -0.3%) to 13.1% (95% CI, 3.6% to 22.6%) for a single observer. Pooled 95% LOA of RMD in calculating the interval change of tumour burden with a single lesion ranged from -31.3% (95% CI, -46.0% to -16.5%) to 30.3% (95% CI, 15.3% to 44.8%) between observers. Studies on calculating the interval change of tumour burden for a single observer or with multiple lesions were lacking.

CONCLUSION

Interobserver RMD in measuring single tumour burden and calculating its interval change may exceed the 20% cut-off for progression. Variability decreased when tumour burden was measured by a single observer or assessed by the sum of multiple lesions.

Inter-observer variability of response evaluation criteria for hepatocellular carcinoma treated with chemoembolization

BACKGROUND

Data comparing EASL and mRECIST criteria for response evaluation in treatment of hepatocellular carcinoma are rare. We evaluated inter-observer variability by these two response evaluation criteria in treatment-naïve patients undergoing chemoembolization.

METHODS

For 133 patients undergoing chemoembolization, two radiologists independently measured sum of bi-dimensional and uni-dimensional diameters at baseline using both EASL criteria and mRECIST, and their changes on first follow-up for up to 5 target lesions.

RESULTS

Concordance correlation coefficients for sum of bi-dimensional and uni-dimensional diameters at baseline between two observers were 0.992 and 0.988, respectively. However, those for their changes on follow-up were 0.865 and 0.877, respectively. Similarly, mean differences in sum of bi-dimensional and uni-dimensional diameters at baseline between two observers were small; -0.455 and 0.079 cm, respectively. However, mean differences in changes (%) in sum of bi-dimensional and uni-dimensional diameters on first follow-up between observers increased by -9.715% and -9.320%, respectively. Regarding tumour numbers, kappa-value between observers was 0.942. For treatment response (complete or partial response, stable disease and progression), kappa-value was 0.941 by both criteria. When only up to two target lesions were assessed, kappa-value was 1.000 by both criteria.

CONCLUSIONS

Inter-observer agreements using both response evaluation criteria were excellent, especially when up to two targets were assessed.

Radiological predictors of cytoreductive outcomes in patients with advanced ovarian cancer

OBJECTIVE

To assess site of disease on preoperative computed tomography (CT) to predict surgical debulking in patients with ovarian cancer.

DESIGN

Two-phase retrospective cohort study.

SETTING

West London Gynaecological Cancer Centre, UK.

POPULATION

Women with stage 3 or 4, ovarian, fallopian or primary peritoneal cancer undergoing cytoreductive surgery.

METHODS

Preoperative CT images were reviewed by experienced radiologists to assess the presence or absence of disease at predetermined sites. Multivariable stepwise logistic regression models determined sites of disease which were significantly associated with surgical outcomes in the test (n = 111) and validation (n = 70) sets.

MAIN OUTCOME MEASURES

Sensitivity and specificity of CT in predicting surgical outcome.

RESULTS

Stepwise logistic regression identified that the presence of lung metastasis, pleural effusion, deposits on the large-bowel mesentery and small-bowel mesentery, and infrarenal para-aortic nodes were associated with debulking status. Logistic regression determined a surgical predictive score which was able to significantly predict suboptimal debulking (n = 94, P = 0.0001) with an area under the curve (AUC) of 0.749 (95% confidence interval [95% CI]: 0.652, 0.846) and a sensitivity of 69.2%, specificity of 71.4%, positive predictive value of 75.0% and negative predictive value of 65.2%. These results remained significant in a recent validation set. There was a significant difference in residual disease volume in the test and validation sets (P < 0.001) in keeping with improved optimal debulking rates.

CONCLUSIONS

The presence of disease at some sites on preoperative CT scan is significantly associated with suboptimal debulking and may be an indication for a change in surgical planning.

Improved efficiency in clinical workflow of reporting measured oncology lesions via PACS-integrated lesion tracking tool

OBJECTIVE. Imaging provides evidence for the response to oncology treatment by the serial measurement of reference lesions. Unfortunately, the identification, comparison, measurement, and documentation of several reference lesions can be an inefficient process. We tested the hypothesis that optimized workflow orchestration and tight integration of a lesion tracking tool into the PACS and speech recognition system can result in improvements in oncologic lesion measurement efficiency. SUBJECTS AND METHODS. A lesion management tool tightly integrated into the PACS workflow was developed. We evaluated the effect of the use of the tool on measurement reporting time by means of a prospective time-motion study on 86 body CT examinations with 241 measureable oncologic lesions with four radiologists. RESULTS. Aggregated measurement reporting time per lesion was 11.64 seconds in standard workflow, 16.67 seconds if readers had to register measurements de novo, and 6.36 seconds for each subsequent follow-up study. Differences were statistically significant (p < 0.05) for each reader, except for one difference for one reader. CONCLUSION. Measurement reporting time can be reduced by using a PACS workflow-integrated lesion management tool, especially for patients with multiple follow-up examinations, reversing the onetime efficiency penalty at baseline registration.

Re-evaluating transarterial chemoembolization for the treatment of hepatocellular carcinoma: Consensus recommendations and review by an International Expert Panel

Patients with unresectable hepatocellular carcinoma (HCC) usually receive transarterial chemoembolization (TACE) or systemic therapies with intermediate and advanced-stage disease. However, intermediate-stage HCC patients often have unsatisfactory clinical outcomes with repeated TACE and there is considerable uncertainty surrounding the criteria for repeating or stopping TACE treatment. In July 2012, an Expert Panel Opinion on Interventions in Hepatocellular Carcinoma (EPOIHCC) was re-convened in Shanghai in an attempt to provide a consensus on the practice of TACE, particularly in regard to evaluating TACE 'failure'. To that end, current clinical practice throughout Asia was reviewed in detail including safety and efficacy data on TACE alone as well as in combination with targeted systemic therapies for intermediate HCC. This review summarizes the evidence discussed at the meeting and provides expert recommendations regarding the use of TACE for unresectable intermediate-stage HCC. A key consensus of the Expert Panel was that the current definitions of TACE failure are not useful in differentiating between situations where TACE is no longer effective in controlling disease locally vs. systemically. By redefining these concepts, it may be possible to provide a clearer indication of when TACE should be repeated and more importantly, when TACE should be discontinued.

Current approaches and challenges in managing and monitoring treatment response in ovarian cancer

Epithelial ovarian cancer is the leading cause of death among gynecologic malignancies. Treatment of recurrent ovarian cancer remains a challenge despite advances in surgical and chemotherapeutic options. A goal of many providers is to detect recurrences as early as possible and initiate treatment though there is controversy as to whether this impacts outcome. Elevations in CA125 and radiological findings may precede symptoms of recurrence by several months. While detection of recurrences by physical exam alone is unusual, a thorough exam in conjunction with reported symptoms and elevated CA125 is sufficient to detect 80-90% of recurrences. A spiral CT scan may be used to confirm recurrence in the setting of asymptomatic CA125 elevation and a PET/CT can yield additional insight if the CT is inconclusive. Initiating chemotherapy prior to the development of symptoms, even in the setting of elevated CA125, does not impact overall survival primarily because the efficacy of available treatments in the recurrent setting is poor. More information about tumor biology and ways to predict which patients will benefit from available treatment options is required. Consequently, the approach to post-treatment surveillance should be individualized taking into account the clinical benefit of the second-line therapy, versus the costs and morbidity of the surveillance method.

Intraobserver and interobserver variability in computed tomography size and attenuation measurements in patients with renal cell carcinoma receiving antiangiogenic therapy: implications for alternative response criteria

BACKGROUND

Alternative response criteria have been proposed in patients with metastatic renal cell carcinoma (mRCC) who are receiving vascular endothelial growth factor (VEGF)-targeted therapy, including 10% tumor shrinkage as an indicator of response/outcome. However, to the authors' knowledge, intraobserver and interobserver measurement variability have not been defined in this setting. The objective of the current study was to determine intraobserver and interobserver agreement of computed tomography (CT) size and attenuation measurements to establish reproducible response indicators.

METHODS

Seventy-one patients with mRCC with 179 target lesions were enrolled in phase 2 and phase 3 trials of VEGF-targeted therapies and retrospectively studied with Institutional Review Board approval. Two radiologists independently measured the long axis diameter and mean attenuation of target lesions at baseline and on follow-up CT. Concordance correlation coefficients and Bland-Altman plots were used to assess intraobserver and interobserver agreement.

RESULTS

High concordance correlation coefficients (range, 0.8602-0.9984) were observed in all types of measurements. The 95% limits of agreement for the percentage change of the sum longest diameter was -7.30% to 7.86% for intraobserver variability, indicating that 10% tumor shrinkage represents a true change in tumor size when measured by a single observer. The 95% limits of interobserver variability were -16.3% to 15.4%. On multivariate analysis, the location of the lesion was found to significantly contribute to interobserver variability (P = .048). The 95% limits of intraobserver agreement for the percentage change in CT attenuation were -18.34% to 16.7%.

CONCLUSIONS

In patients with mRCC who are treated with VEGF inhibitors, 10% tumor shrinkage is a reproducible radiologic response indicator when baseline and follow-up studies are measured by a single radiologist. Lesion location contributes significantly to measurement variability and should be considered when selecting target lesions.

Inter-observer reproducibility of semi-automatic tumor diameter measurement and volumetric analysis in patients with lung cancer

OBJECTIVES

Therapy monitoring in oncologic patient requires precise measurement methods. In order to improve the precision of measurements, we used a semi-automated generic segmentation algorithm to measure the size of large lung cancer tumors. The reproducibility of computer-assisted measurements were assessed and compared with manual measurements.

METHODS

CT scans of 24 consecutive lung cancer patients who were referred to our hospital over a period of 6 months were analyzed. The tumor sizes were measured manually by 3 independent radiologists, according to World Health Organization (WHO) and the Revised Response Evaluation Criteria in Solid Tumors (RECIST) guidelines. At least 10 months later, measurements were repeated semi-automatically on the same scans by the same radiologists. The inter-observer reproducibility of all measurements was assessed and compared between manual and semi-automated measurements.

RESULTS

Manual measurements of the tumor longest diameter were significantly (p < 0.05) smaller compared with the semi-automated measurements. The intra-rater correlations coefficients were significantly higher for measurements of longest diameter (intra-class correlation coefficients: 0.998 vs. 0.986; p < 0.001) and area (0.995 vs. 0.988; p = 0.032) using semi-automated compared with manual method. The variation coefficient for manual measurement of the tumor area (WHO guideline, 15.7% vs. 7.3%) and the longest diameter (RECIST guideline, 7.7% vs. 2.7%) was 2-3 times that of semi-automated measurement.

CONCLUSIONS

By using computer-assisted size assessment in primary lung tumor, interobserver-variability can be reduced to about half to one-third compared to standard manual measurements. This indicates a high potential value for therapy monitoring in lung cancer patients.