Diagnostic Performance and Interreader Agreement of the MRI Clear Cell Likelihood Score for Characterization of cT1a and cT1b Solid Renal Masses: An External Validation Study

Diagnostic accuracy of the Clear Cell Likelihood Score and selected MRI parameters in the characterization of indeterminate renal masses - a single-institution study

PURPOSE

We aimed to assess the diagnostic accuracy of the clear cell likelihood score (ccLS) and value of other selected magnetic resonance imaging (MRI) features in the characterization of indeterminate small renal masses (SRMs).

METHODS

Fifty patients with indeterminate SRMs discovered on MRI between 2012 and 2023 were included. The ccLS for the characterization of clear cell renal cell carcinoma (ccRCC) was calculated and compared to the final diagnosis (ccRCC vs. 'all other' masses).

RESULTS

The ccLS = 5 had a satisfactory accuracy of 64.0% and a very high specificity of 96.3%; however, its sensitivity of 26.1% was relatively low. Receiver operating curve (ROC) analysis revealed that from the selected MRI features, only T1 ratio and arterial to delayed enhancement (ADER) were good discriminators between ccRCC and other types of renal masses (area under curve, AUC = 0.707, p = 0.01; AUC = 0.673, p = 0.03; respectively). The cut-off points determined in ROC analysis using the Youden index were 0.73 (p = 0.01) for T1 ratio and 0.99 for ADER (p = 0.03). The logistic regression model demonstrated that ccLS = 5 and T1 ratio (OR = 15.5 [1.1-218.72], p = 0.04; OR = 0.002 [0.00-0.81], p = 0.04) were significant predictors of ccRCC.

CONCLUSIONS

The ccLS algorithm offers an encouraging method for the standardization of imaging protocols to aid in the diagnosis and management of SRMs in daily clinical practice by enhancing detectability of ccRCC and reducing the number of unnecessary invasive procedures for benign or indolent lesions. However, its diagnostic performance needs multi-center large cohort studies to validate it before it can be incorporated as a diagnostic algorithm and will guide future iterations of clinical guidelines. The retrospective nature of our study and small patient population confined to a single clinical center may impact the generalizability of the results; thus, future studies are required to define whether employment of the T1 ratio or ADER parameter may strengthen the diagnostic accuracy of ccRCC diagnosis.

Clear cell likelihood score may improve diagnosis and management of renal masses

The detection of solid renal masses has increased over time due to incidental findings during imaging studies conducted for unrelated medical conditions. Approximately 20% of lesions measuring less than 4 cm are benign and 80% are malignant. Clear cell renal cell carcinoma (ccRCC) is the most frequent among renal carcinomas, responsible for 65-80% of cases. The increased detection of renal masses facilitates early diagnosis and treatment. However, it also leads to more invasive interventions, which result in higher morbidity and costs. Currently, only histological analysis can offer an accurate diagnosis. Surgical nephron loss significantly elevates morbidity and mortality rates. Active surveillance represents a conservative management approach for patients diagnosed with a solid renal mass that is endorsed by both American Urological Association and the European Society for Medical Oncology. However, active surveillance is used in a minority of patients and varies across institutions. The lack of clinical studies using a standardized approach to incidentally detected small renal masses precludes the widespread use of active surveillance. Hence, there is an urgent need for better patient selection, distinguishing those who require surgery from those suitable for active surveillance. The clear cell likelihood score (ccLS) represents a novel MRI tool for assessing the probability of a renal mass being a ccRCC. In this study, we present a comprehensive review of renal masses and their evaluation using the ccLS to facilitate shared decision between urologists and patients.

Cost-Effectiveness Analysis of the Clear Cell Likelihood Score Against Renal Mass Biopsy for Evaluating Small Renal Masses

OBJECTIVE

To examine the cost-effectiveness of the clear cell likelihood score compared to renal mass biopsy (RMB) alone.

METHODS

The clear cell likelihood score, a new grading system based on multiparametric magnetic resonance imaging, has been proposed as a possible alternative to percutaneous RMB for identifying clear cell renal carcinoma in small renal masses and expediting treatment of high-risk patients. A decision analysis model was developed to compare a RMB strategy where all patients undergo biopsy and a clear cell likelihood score strategy where only patients that received an indeterminant score of 3 undergo biopsy. Effectiveness was assigned 1 for correct diagnoses and 0 for incorrect or indeterminant diagnoses. Costs were obtained from institutional fees and Medicare reimbursement rates. Probabilities were derived from literature estimates from radiologists trained in the clear cell likelihood score.

RESULTS

In the base case model, the clear cell likelihood score was both more effective (0.77 vs 0.70) and less expensive than RMB ($1629 vs $1966). Sensitivity analysis found that the nondiagnostic rate of RMB and the sensitivity of the clear cell likelihood score had the greatest impact on the model. In threshold analyses, the clear cell likelihood score was the preferred strategy when its sensitivity was greater than 62.7% and when an MRI cost less than $5332.

CONCLUSION

The clear cell likelihood score is a more cost-effective option than RMB alone for evaluating small renal masses for clear cell renal carcinoma.

Diagnostic Value of Clear Cell Likelihood Score v1.0 and v2.0 for Common Subtypes of Small Renal Masses: A Multicenter Comparative Study

BACKGROUND

Clear cell likelihood score (ccLS) is reliable for diagnosing small renal masses (SRMs). However, the diagnostic value of Clear cell likelihood score version 1.0 (ccLS v1.0) and v2.0 for common subtypes of SRMs might be a potential score extension.

PURPOSE

To compare the diagnostic performance and interobserver agreement of ccLS v1.0 and v2.0 for characterizing five common subtypes of SRMs.

STUDY TYPE

Retrospective.

POPULATION

797 patients (563 males, 234 females; mean age, 53 ± 12 years) with 867 histologically proven renal masses.

FIELD STRENGTH/SEQUENCES

3.0 and 1.5 T/T2 weighted imaging, T1 weighted imaging, diffusion-weighted imaging, a dual-echo chemical shift (in- and opposed-phase) T1 weighted imaging, multiphase dynamic contrast-enhanced imaging.

ASSESSMENT

Six abdominal radiologists were trained in the ccLS algorithm and independently scored each SRM using ccLS v1.0 and v2.0, respectively. All SRMs had definite pathological results. The pooled area under curve (AUC), accuracy, sensitivity, and specificity were calculated to evaluate the diagnostic performance of ccLS v1.0 and v2.0 for characterizing common subtypes of SRMs. The average κ values were calculated to evaluate the interobserver agreement of the two scoring versions.

STATISTICAL TESTS

Random-effects logistic regression; Receiver operating characteristic analysis; DeLong test; Weighted Kappa test; Z test. The statistical significance level was P < 0.05.

RESULTS

The pooled AUCs of clear cell likelihood score version 2.0 (ccLS v2.0) were statistically superior to those of ccLS v1.0 for diagnosing clear cell renal cell carcinoma (ccRCC) (0.907 vs. 0.851), papillary renal cell carcinoma (pRCC) (0.926 vs. 0.888), renal oncocytoma (RO) (0.745 vs. 0.679), and angiomyolipoma without visible fat (AMLwvf) (0.826 vs. 0.766). Interobserver agreement for SRMs between ccLS v1.0 and v2.0 is comparable and was not statistically significant (P = 0.993).

CONCLUSION

The diagnostic performance of ccLS v2.0 surpasses that of ccLS v1.0 for characterizing ccRCC, pRCC, RO, and AMLwvf. Especially, the standardized algorithm has optimal performance for ccRCC and pRCC. ccLS has potential as a supportive clinical tool.

EVIDENCE LEVEL

4.

TECHNICAL EFFICACY

Stage 2.

Differentiation between renal epithelioid angiomyolipoma and clear cell renal cell carcinoma using clear cell likelihood score

PURPOSE

Clear cell likelihood score (ccLS) may be a reliable diagnostic method for distinguishing renal epithelioid angiomyolipoma (EAML) and clear cell renal cell carcinoma (ccRCC). In this study, we aim to explore the value of ccLS in differentiating EAML from ccRCC.

METHODS

We performed a retrospective analysis in which 27 EAML patients and 60 ccRCC patients underwent preoperative magnetic resonance imaging (MRI) at our institution. Two radiologists trained in the ccLS algorithm scored independently and the consistency of their interpretation was evaluated. The difference of the ccLS score was compared between EAML and ccRCC in the whole study cohort and two subgroups [small renal masses (SRM; ≤ 4 cm) and large renal masses (LRM; > 4 cm)].

RESULTS

In total, 87 patients (59 men, 28 women; mean age, 55±11 years) with 90 renal masses (EAML: ccRCC = 1: 2) were identified. The interobserver agreement of two radiologists for the ccLS system to differentiate EAML from ccRCC was good (k = 0.71). The ccLS score in the EAML group and the ccRCC group ranged from 1 to 5 (73.3% in scores 1-2) and 2 to 5 (76.7% in scores 4-5), respectively, with statistically significant differences (P < 0.001). With the threshold value of 2, ccLS can distinguish EAML from ccRCC with the accuracy, sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) of 87.8%, 95.0%, 73.3%, 87.7%, and 88.0%, respectively. The AUC (area under the curve) was 0.913. And the distribution of the ccLS score between the two diseases was not affected by tumor size (P = 0.780).

CONCLUSION

The ccLS can distinguish EAML from ccRCC with high accuracy and efficiency.

MRI for the detection of small malignant renal masses: a systematic review and meta-analysis

Objective

We aimed to review the available evidence on the diagnostic performance of magnetic resonance imaging in differentiating malignant from benign small renal masses.

Methods

An electronic literature search of Web of Science, MEDLINE (Ovid and PubMed), Cochrane Library, EMBASE, and Google Scholar was performed to identify relevant articles up to 31 January 2023. We included studies that reported the diagnostic accuracy of using magnetic resonance imaging to differentiate small (≤4 cm) malignant from benign renal masses. The pooled sensitivity, specificity, positive likelihood ratio, negative likelihood ratio, and diagnostic odds ratio were calculated using the bivariate model and the hierarchical summary receiver operating characteristic model. The study quality evaluation was performed with the Quality Assessment of Diagnostic Accuracy Studies-2 tool.

Results

A total of 10 studies with 860 small renal masses (815 patients) were included in the current meta-analysis. The pooled sensitivity and specificity of the studies for the detection of malignant masses were 0.85 (95% CI 0.79-0.90) and 0.83 (95% CI 0.67-0.92), respectively.

Conclusions

MRI had a moderate diagnostic performance in differentiating small malignant renal masses from benign ones. Substantial heterogeneity was observed between studies for both sensitivity and specificity.

Insights into Renal Cell Carcinoma with Novel Imaging Approaches

This article presents a comprehensive overview of new imaging approaches and techniques for improving the assessment of renal masses and renal cell carcinoma. The Bosniak classification, version 2019, as well as the clear cell likelihood score, version 2.0, will be discussed as new imaging algorithms using established techniques. Additionally, newer modalities, such as contrast-enhanced ultrasound, dual energy computed tomography, and molecular imaging, will be discussed in conjunction with emerging radiomics and artificial intelligence techniques. Current diagnostic algorithms combined with newer approaches may be an effective way to overcome existing limitations in renal mass and RCC characterization.

External Validation of a Five-Tiered CT Algorithm for the Diagnosis of Clear Cell Renal Cell Carcinoma: A Retrospective Five-Reader Study

BACKGROUND. In 2022, a five-tiered CT algorithm was proposed for predicting whether a small (cT1a) solid renal mass represents clear cell renal cell carcinoma (ccRCC). OBJECTIVE. The purpose of this external validation study was to evaluate the proposed CT algorithm for diagnosis of ccRCC among small solid renal masses. METHODS. This retrospective study included 93 patients (median age, 62 years; 42 women, 51 men) with 97 small solid renal masses that were seen on corticomedullary phase contrast-enhanced CT performed between January 2012 and July 2022 and subsequently underwent surgical resection. Five readers (three attending radiologists, two clinical fellows) independently evaluated masses for the mass-to-cortex corticomedullary attenuation ratio and heterogeneity score; these scores were used to derive the CT score by use of the previously proposed CT algorithm. The CT score's sensitivity, specificity, and PPV for ccRCC were calculated at threshold of 4 or greater, and the NPV for ccRCC was calculated at a threshold of 3 or greater (consistent with thresholds in studies of the MRI-based clear cell likelihood score and the CT algorithm's initial study). The CT score's sensitivity and specificity for papillary RCC were calculated at a threshold of 2 or less. Interreader agreement was assessed using the Gwet agreement coefficient (AC1). RESULTS. Overall, 61 of 97 masses (63%) were malignant and 43 of 97 (44%) were ccRCC. Across readers, CT score had sensitivity ranging from 47% to 95% (pooled sensitivity, 74% [95% CI, 68-80%]), specificity ranging from 19% to 83% (pooled specificity, 59% [95% CI, 52-67%]), PPV ranging from 48% to 76% (pooled PPV, 59% [95% CI, 49-71%]), and NPV ranging from 83% to 100% (pooled NPV, 90% [95% CI, 84-95%]), for ccRCC. A CT score of 2 or less had sensitivity ranging from 44% to 100% and specificity ranging from 77% to 98% for papillary RCC (representing nine of 97 masses). Interreader agreement was substantial for attenuation score (AC1 = 0.70), poor for heterogeneity score (AC1 = 0.17), fair for five-tiered CT score (AC1 = 0.32), and fair for dichotomous CT score at a threshold of 4 or greater (AC1 = 0.24 [95% CI, 0.14-0.33]). CONCLUSION. The five-tiered CT algorithm for evaluation of small solid renal masses was tested in an external sample and showed high NPV for ccRCC. CLINICAL IMPACT. The CT algorithm may be used for risk stratification and patient selection for active surveillance by identifying patients unlikely to have ccRCC.

Renal Mass Imaging with MRI Clear Cell Likelihood Score: A User's Guide

Small solid renal masses (SRMs) are frequently detected at imaging. Nearly 20% are benign, making careful evaluation with MRI an important consideration before deciding on management. Clear cell renal cell carcinoma (ccRCC) is the most common renal cell carcinoma subtype with potentially aggressive behavior. Thus, confident identification of ccRCC imaging features is a critical task for the radiologist. Imaging features distinguishing ccRCC from other benign and malignant renal masses are based on major features (T2 signal intensity, corticomedullary phase enhancement, and the presence of microscopic fat) and ancillary features (segmental enhancement inversion, arterial-to-delayed enhancement ratio, and diffusion restriction). The clear cell likelihood score (ccLS) system was recently devised to provide a standardized framework for categorizing SRMs, offering a Likert score of the likelihood of ccRCC ranging from 1 (very unlikely) to 5 (very likely). Alternative diagnoses based on imaging appearance are also suggested by the algorithm. Furthermore, the ccLS system aims to stratify which patients may or may not benefit from biopsy. The authors use case examples to guide the reader through the evaluation of major and ancillary MRI features of the ccLS algorithm for assigning a likelihood score to an SRM. The authors also discuss patient selection, imaging parameters, pitfalls, and areas for future development. The goal is for radiologists to be better equipped to guide management and improve shared decision making between the patient and treating physician. © RSNA, 2023 Quiz questions for this article are available in the supplemental material. See the invited commentary by Pedrosa in this issue.

Performance of clear cell likelihood scores in characterizing solid renal masses at multiparametric MRI: an external validation study

PURPOSE

The purpose of this study is to evaluate the accuracy and interobserver agreement of ccLS in diagnosing clear cell renal cell carcinoma (ccRCC).

METHODS

This retrospective single-center study evaluated consecutive patients with solid renal masses who underwent mpMRI followed by percutaneous biopsy and/or surgical excision between January 2010 and December 2020. Predominantly (> 75%) cystic masses, masses with macroscopic fat and infiltrative masses were excluded. Two abdominal radiologists independently scored each renal mass according to the proposed ccLS algorithm. The diagnostic performance of ccLS categories for ccRCC was calculated using logistic regression modeling. Diagnostic accuracy for predicting ccRCC was calculated using 2 × 2 contingency tables. Interobserver agreement for ccLS was evaluated with Cohen's k statistic.

RESULTS

A total of 79 patients (mean age, 63 years ± 12 [SD], 50 men) with 81 renal masses were evaluated. The mean size was 36 mm ± 28 (range 10-160). Of the renal masses included, 44% (36/81) were ccRCC. The area under the receiver operating characteristic curve was 0.87 (95% CI 0.79-0.95). Using ccLS ≥ 4 to diagnose ccRCC, the sensitivity, specificity, and positive predictive value were 93% (95% CI 79, 99), 63% (95% CI 48, 77), and 67% (95% CI 58, 75), respectively. The negative predictive value of ccLS ≤ 2 was 93% (95% CI 64, 99). The proportion of ccRCC by ccLS category 1 to 5 were 10%, 0%, 10%, 57%, and 84%, respectively. Interobserver agreement was moderate (k = 0.47).

CONCLUSION

In this study, clear cell likelihood score had moderate interobserver agreement and resulted in 96% negative predictive value in excluding ccRCC.

Systematic review and meta-analysis of multiparametric MRI clear cell likelihood scores for classification of small renal masses

Purpose

To systematically assess the multiparametric MRI clear cell likelihood score (ccLS) algorithm for the classification of small renal masses (SRM).

Methods

We conducted an electronic literature search on Web of Science, MEDLINE (Ovid and PubMed), Cochrane Library, EMBASE, and Google Scholar to identify relevant articles from 2017 up to June 30, 2022. We included studies reporting the diagnostic performance of the ccLS for characterization of solid SRM. The bivariate model and hierarchical summary receiver operating characteristic (HSROC) model were used to pool sensitivity, specificity, positive likelihood ratio (LR+), negative likelihood ratio (LR−), and diagnostic odds ratio (DOR). The quality evaluation was performed with the Quality Assessment of Diagnostic Accuracy Studies-2 tool.

Results

A total of 6 studies with 825 renal masses (785 patients) were included in the current meta-analysis. The pooled sensitivity and specificity for cT1a renal masses were 0.80 (95% CI 0.75–0.85) and 0.74 (95% CI 0.65–0.81) at the threshold of ccLS ≥4, the pooled LR+, LR−, and DOR were 3.04 (95% CI 2.34-3.95), 0.27 (95% CI 0.22–0.33), and 11.4 (95% CI 8.2-15.9), respectively. The area under the HSROC curve was 0.84 (95% CI 0.81–0.87). For all cT1 renal masses, the pooled sensitivity and specificity were 0.80 (95% CI 0.74–0.85) and 0.76 (95% CI 0.67–0.83).

Conclusions

The ccLS had moderate to high accuracy for identifying ccRCC from other RCC subtypes and with a moderate inter-reader agreement. However, its diagnostic performance remain needs multi-center, large cohort studies to validate in the future.