Measurements of Hepatic Metastasis on MR Imaging:: Assessment of Interobserver and Intersequence Variability

Apparent Variation in Measurement Size of Colorectal Cancer Metastases to the Liver on Dual Contrast MRI

PURPOSE

The aim of this study was to formally investigate the apparent variation in lesion size of hepatic metastatic lesions from colorectal cancer on hepatobiliary phase (HBP) and dual contrast images of magnetic resonance imaging performed with both hepatobiliary and extracellular contrast agents.

METHODS

Patients with known colorectal carcinoma who had undergone dual contrast liver magnetic resonance imaging were identified in our institutional database. Metastatic lesions were measured semiautomatically on both HBP and dual contrast images with a custom software tool that automatically identifies the lesion edge and thereby the lesion diameter. Lesion measurements from both sets of images were compared with a Student t test and Bland-Altman analysis. Lesions were also measured on both HBP and dual contrast images by 2 fellowship-trained abdominal radiologists. Measurements from the software and radiologists were compared with a Student t test and Bland-Altman analysis; interreader agreement was evaluated with the intraclass correlation coefficient.

RESULTS

A total of 70 liver lesions in 39 patients was identified. Software-based measurements were significantly larger on HBP than dual contrast images ( P < 0.001), with a mean lesion size of 10.9 ± 4.2 mm for HBP and 10.5 ± 4.2 mm for dual contrast measurements. Radiologist-based measurements showed a similar trend, with HBP measurements being significantly larger than dual contrast measurements ( P < 0.001). Bland-Altman analysis indicated a mean bias ± 2 SD of +0.4 ± 1.6 mm for software-based measurements and +0.9 ± 2.9 mm and +0.7 ± 2.1 mm for readers 1 and 2, respectively. The intraclass correlation coefficient for interreader agreement was 0.9.

CONCLUSIONS

Both software-based and radiologist-based measurements of colorectal cancer liver metastases are significantly larger on HBP than dual contrast images. Based on these findings, we recommend that longitudinal assessment be performed consistently on either HBP or dual contrast phases to avoid introduction of avoidable variability.

Variability of quantitative measurements of metastatic liver lesions: a multi-radiation-dose-level and multi-reader comparison

PURPOSE

To evaluate the variability of quantitative measurements of metastatic liver lesions by using a multi-radiation-dose-level and multi-reader comparison.

METHODS

Twenty-three study subjects (mean age, 60 years) with 39 liver lesions who underwent a single-energy dual-source contrast-enhanced staging CT between June 2015 and December 2015 were included. CT data were reconstructed with seven different radiation dose levels (ranging from 25 to 100%) on the basis of a single CT acquisition. Four radiologists independently performed manual tumor measurements and two radiologists performed semi-automated tumor measurements. Interobserver, intraobserver, and interdose sources of variability for longest diameter and volumetric measurements were estimated and compared using Wilcoxon rank-sum tests and intraclass correlation coefficients.

RESULTS

Inter- and intraobserver variabilities for manual measurements of the longest diameter were higher compared to semi-automated measurements (p < 0.001 for overall). Inter- and intraobserver variabilities of volume measurements were higher compared to the longest diameter measurement (p < 0.001 for overall). Quantitative measurements were statistically different at < 50% radiation dose levels for semi-automated measurements of the longest diameter, and at 25% radiation dose level for volumetric measurements. The variability related to radiation dose was not significantly different from the inter- and intraobserver variability for the measurements of the longest diameter.

CONCLUSION

The variability related to radiation dose is comparable to the inter- and intraobserver variability for measurements of the longest diameter. Caution should be warranted in reducing radiation dose level below 50% of a conventional CT protocol due to the potentially detrimental impact on the assessment of lesion response in the liver.

Sequence and Observer Variability in Gadoxectic Acid-Enhanced MRI Lesion Measurements in Hepatocellular Carcinoma

RATIONALE AND OBJECTIVES

The purpose of the study was to investigate interobserver and intersequence variability in measuring hepatocellular carcinoma on magnetic resonance imaging (MRI).

METHODS AND MATERIALS

Twenty treatment-naïve lesions on Gadoxetic Acid enhanced MRI scans from 20 patients were retrospectively measured by six reviewers with different levels of experience, twice, six weeks apart, on eight different MRI sequences, in randomized order. The sequences include arterial, hepatobiliary, transitional, portal venous, T2, and diffusion weighted images. The single longest diameter (SLD) and longest diameter perpendicular to the longest overall diameter were measured on axial images and products of diameters calculated in accordance to response evaluation criteria in solid tumors v1.1 and World Health Organization response criteria respectively. Lesion-wise intraclass correlation coefficients were used to estimate measurement agreement.

RESULTS

All intraclass correlation coefficients were greater than 0.95. No substantive differences between SLD and products of diameters metrics. Means (∼2.8 mm, SLD) and standard deviations (∼2 mm, SLD) were similar across sequences and observers. Similarly, pairwise comparison between observers grouped by experience showed statistically significant differences, but the effect size was minor (∼2 mm). Arterial and HPB-weighted images had similar mean dimensions (2.76 cm) while the smallest mean was in the transitional phase (2.62 cm). A lesion was not measured on 140 occasions (7%), mostly in ADC.

CONCLUSION

There is high interobserver and intersequence reliability despite small differences between observers based on experience level. Our results suggest that accurate measurements can be made on arterial phase despite the possibility of indistinct margins. Lesions, however, are more likely to be missed on diffusion-related sequences.

Validity of RECIST Version 1.1 for Response Assessment in Metastatic Cancer: A Prospective, Multireader Study

Purpose To determine the relationship between target lesion selection with use of Response Evaluation Criteria in Solid Tumors (RECIST) version 1.1 and classification of therapeutic response in patients with metastatic cancer undergoing systemic cytotoxic and/or targeted therapies. Materials and Methods This prospective multireader study was conducted between July 2015 and July 2017. Three hundred sixteen consecutive participants with metastatic cancer underwent 932 CT examinations to monitor systemic treatment. CT studies were independently read by three radiologists. Readers identified a maximum of five lesions total (and a maximum of two lesions per organ). Dedicated oncology tumor response software was used. The Fleiss κ statistic was used to analyze interreader agreement in the assignment of individual response classes (complete response, partial response, progressive disease, or stable disease) and in the differentiation between progressive and nonprogressive disease. Results Readers selected the same set of target lesions in 128 of the 316 participants (41%) and selected a different set in 188 (59%). When target lesion selection was concordant, agreement was high (assignment of treatment response category: κ = 0.97; 95% confidence interval [CI]: 0.91, 1.0; differentiation between progressive and nonprogressive disease: κ = 0.98; 95% CI: 0.90, 1.0). When target lesion selection was discordant, agreement was significantly reduced (assignment of treatment response category: κ = 0.58; 95% CI: 0.54, 0.62; differentiation between progressive and nonprogressive disease: κ = 0.6; 95% CI: 0.59, 0.70). With concordant target lesion selection, readers agreed regarding diagnosis of progression in 97.7% of participants (95% CI: 95.4%, 100.0%); with discordant target lesion selection, readers agreed in only 55.3% (95% CI: 47.9%, 62.6%) (P < .01). Conclusion In patients with metastatic cancer undergoing systemic treatment, different cancer sites may appear similarly suitable and thus likely to be selected as target lesions but may yield inconsistent or even conflicting results with Response Evaluation Criteria in Solid Tumors (RECIST) version 1.1. This indicates that the current, limited set of target lesions in RECIST 1.1 may not reflect overall tumor load or response to therapy. © RSNA, 2018 See also the editorial by Sosna in this issue.

Gastroenteropancreatic neuroendocrine tumors: impact of consistent contrast agent selection on radiologists' confidence in hepatic lesion assessment on restaging MRIs

PURPOSE

To evaluate the impact of contrast agent selection on radiologists' confidence in assessing liver lesions on follow-up magnetic resonance imaging (MRI) studies in patients with neuroendocrine tumors.

METHODS

This Institutional Review Board-approved, retrospective study performed at a tertiary cancer center and a quaternary care urban academic hospital included all 694 follow-up abdominal MRI studies from 179 patients with gastroenteropancreatic neuroendocrine tumor performed from 01/01/2010 to 05/31/2015. Primary outcome measure was radiologists' confidence in assessing liver lesions on follow-up MRI. MRI reports were reviewed to abstract radiologists' confidence, classified as "equivocal" if any equivocal connotation (mention of limitation due to differences in contrast agent or follow-up recommendation with specific contrast agent) was present; or "unequivocal" if a precise, confident comparison to prior was documented without the use of ambiguous terms. A fellowship-trained radiologist separately evaluated 100 randomly selected reports and images to calculate interobserver agreement with the report classification (equivocal vs. unequivocal) and with the original MRI report, respectively. Chi-square test was used to compare the proportion of equivocal reports when "same" or "different" contrast agent was used for successive examinations.

RESULTS

Rates of equivocal reports were higher when different contrast agents were used for successive examinations compared to examinations with same contrast agent (13.2% [21/159] vs. 1.8% [10/535]; p < 0.0001). There was very good interobserver agreement for assessment of radiologist confidence (κ = 0.92 for report review, κ = 0.82 for image review).

CONCLUSIONS

Consistent use of contrast agent for follow-up MRIs allows more confident assessment of liver lesions in patients with neuroendocrine tumors.

Measurement variability of liver metastases from neuroendocrine tumors on different magnetic resonance imaging sequences

PURPOSE

To assess dimension measurement variability of liver metastases from neuroendocrine tumors (LMNET) on different magnetic resonance imaging (MRI) sequences.

MATERIAL AND METHODS

In this institutional review board-approved retrospective study from January 2011 to December 2012, all liver MRI examinations performed at our department in patients with at least one measurable LMNET according to response evaluation criteria in solid tumors (RECIST1.1) were included. Up to two lesions were selected on T2-weighted MR images. Three reviewers independently measured long axes of 135 hepatic metastases in 30 patients (16 men, 14 women, mean age 61±11.4 (SD) years; range 28-78 years), during two separate reading sessions, on T2-weighted, diffusion-weighted MRI (DWI) (b; 50, 400, 800 s/mm²) and arterial, portal and late phases after intravenous administration of a gadolinium chelate. Intraclass-correlation coefficients and Bland-Altman plots were used to assess intra-and interobserver variability.

RESULTS

Intra- and interobserver agreements ranged between 0.87-0.98, and 0.88-0.97, respectively. Intersequence agreements ranged between 0.92 [95%CI: 0.82-0.98] and 0.98 [95%CI: 0.93-0.99]. 95% limits of agreement for measurements were -10.2%,+8.9% for DWI (b=50s/mm²) versus -21.9%,+24.2% and -15.8,+17.2% for arterial and portal phases, respectively.

CONCLUSION

An increase<9% in measurement and a decrease of -10% on DWI should not be considered as true changes, with 95% confidence, versus 24% and -22% on arterial and 17%, -16% on portal phases, respectively. DWI might thus be the most reliable MR sequence for monitoring size variations of LMNETs.

Biases in the reporting of hepatocellular carcinoma tumor sizes on the liver transplant waiting list

We investigated the possibility that patients with hepatocellular carcinoma (HCC) listed for liver transplant with tumors just outside stage T2 size criteria may be inaccurately reported as just meeting the tumor size criteria for transplant. The United Network for Organ Sharing/Standard Transplant Analysis and Research database identified 12,958 patients listed for liver transplants with HCC exception points from 2006 to 2013, 9,168 of whom were listed with one tumor. A logistic power peak function was fitted to the single-tumor size histogram, with the fitted values representing unbiased expected values. The difference between the observed and expected tumor counts for 2.0 cm and 5.0 cm was 238 (22%) and 66 (57%), respectively. This suggests that up to 304 (3.0%) patients with tumors outside of transplant criteria had their measurements recorded at the margins of eligibility. A risk-adjusted Poisson model evaluated the ratio of observed to expected HCC recurrence by tumor size. There were 435 HCC recurrences among 6,049 transplants. Only 2.0-cm tumors had observed to expected recurrence differing from 1 (ratio 0.73, 95% confidence interval 0.57-0.94), indicating a 27% lower than expected rate of recurrence.

CONCLUSION

Higher than expected observed tumor counts at the lower transplant criteria margin were corroborated by lower than expected HCC recurrence, suggesting that tumor sizes at the margins of HCC transplant criteria may be subject to inaccurate reporting. (Hepatology 2017;66:1144-1150).