Interrater Agreement of Bosniak Classification Version 2019 and Version 2005 for Cystic Renal Masses at CT and MRI

Bosniak Classification of Cystic Renal Masses: Looking Back, Looking Forward

RATIONALE AND OBJECTIVES

According to the 2019 update of the Bosniak classification, the main imaging features that need to be evaluated to achieve a correct characterization of renal cystic masses include the thickness of walls and septa, the number of septa, the appearance of walls and septa, the attenuation/intensity on non-contrast CT/MRI and the presence of unequivocally perceived or measurable enhancement of walls and septa. Despite the improvement deriving from a quantitative evaluation of imaging features, certain limitations seem to persist and some possible scenarios that can be encountered in clinical practice are still missing.

MATERIALS AND METHODS

A deep analysis of the 2019 update of the Bosniak classification was performed.

RESULTS

The most notable potential flaws concern: (1) the quantitative measurement of the walls and septa; (2) the fact that walls and septa > 2 mm are always referred to as "enhancing", not considering the alternative scenario; (3) the description of some class II masses partially overlaps with each other and with the definition of class I masses and (4) the morphological variations of cystic masses over time is not considered.

CONCLUSION

The present paper analyzes in detail the limitations of the 2019 Bosniak classification to improve this important tool and facilitate its use in daily radiological practice.

Deep learning system for malignancy risk prediction in cystic renal lesions: a multicenter study

OBJECTIVES

To develop an interactive, non-invasive artificial intelligence (AI) system for malignancy risk prediction in cystic renal lesions (CRLs).

METHODS

In this retrospective, multicenter diagnostic study, we evaluated 715 patients. An interactive geodesic-based 3D segmentation model was created for CRLs segmentation. A CRLs classification model was developed using spatial encoder temporal decoder (SETD) architecture. The classification model combines a 3D-ResNet50 network for extracting spatial features and a gated recurrent unit (GRU) network for decoding temporal features from multi-phase CT images. We assessed the segmentation model using sensitivity (SEN), specificity (SPE), intersection over union (IOU), and dice similarity (Dice) metrics. The classification model's performance was evaluated using the area under the receiver operator characteristic curve (AUC), accuracy score (ACC), and decision curve analysis (DCA).

RESULTS

From 2012 to 2023, we included 477 CRLs (median age, 57 [IQR: 48-65]; 173 men) in the training cohort, 226 CRLs (median age, 60 [IQR: 52-69]; 77 men) in the validation cohort, and 239 CRLs (median age, 59 [IQR: 53-69]; 95 men) in the testing cohort (external validation cohort 1, cohort 2, and cohort 3). The segmentation model and SETD classifier exhibited excellent performance in both validation (AUC = 0.973, ACC = 0.916, Dice = 0.847, IOU = 0.743, SEN = 0.840, SPE = 1.000) and testing datasets (AUC = 0.998, ACC = 0.988, Dice = 0.861, IOU = 0.762, SEN = 0.876, SPE = 1.000).

CONCLUSION

The AI system demonstrated excellent benign-malignant discriminatory ability across both validation and testing datasets and illustrated improved clinical decision-making utility.

CRITICAL RELEVANCE STATEMENT

In this era when incidental CRLs are prevalent, this interactive, non-invasive AI system will facilitate accurate diagnosis of CRLs, reducing excessive follow-up and overtreatment.

KEY POINTS

The rising prevalence of CRLs necessitates better malignancy prediction strategies. The AI system demonstrated excellent diagnostic performance in identifying malignant CRL. The AI system illustrated improved clinical decision-making utility.

Intra- and Interobserver Study Investigating the Adapted EFSUMB Bosniak Cyst Categorization Proposed for Contrast-Enhanced Ultrasound (CEUS) in 2020

BACKGROUND

To investigate the inter- and intraobserver variability in comparison to an expert gold standard of the new and modified renal cyst Bosniak classification proposed for contrast-enhanced ultrasound findings (CEUS) by the European Federation of Societies for Ultrasound in Medicine and Biology (EFSUMB) in 2020.

MATERIALS AND METHODS

84 CEUS examinations for the evaluation of renal cysts were evaluated retrospectively by six readers with different levels of ultrasound expertise using the modified Bosniak classification proposed for CEUS. All cases were anonymized, and each case was rated twice in randomized order. The consensus reading of two experts served as the gold standard, to which all other readers were compared. Statistical analysis was performed using Cohen's weighted kappa tests, where appropriate.

RESULTS

Intraobserver variability showed substantial to almost perfect agreement (lowest kappa κ=0.74; highest kappa κ=0.94), with expert level observers achieving the best results. Comparison to the gold standard was almost perfect for experts (highest kappa κ=0.95) and lower for beginner and intermediate level readers still achieving mostly substantial agreement (lowest kappa κ=0.59). Confidence of rating was highest for Bosniak classes I and IV and lowest for classes IIF and III.

CONCLUSION

Categorization of cystic renal lesions based on the Bosniak classification proposed by the EFSUMB in 2020 showed very good reproducibility. While even less experienced observers achieved mostly substantial agreement, training remains a major factor for better diagnostic performance.

Computed tomography-based prediction model for identifying patients with high probability of non-muscle-invasive bladder cancer

PURPOSE

To investigate computed tomography (CT)-based prediction model for identifying patients with high probability of non-muscle-invasive bladder cancer (NMIBC).

METHODS

This retrospective study evaluated 147 consecutive patients who underwent contrast-enhanced CT and surgery for bladder cancer. Using corticomedullary-to-portal venous phase images, two independent readers analyzed bladder muscle invasion, tumor stalk, and tumor size, respectively. Three-point scale (i.e., from 0 to 2) was applied for assessing the suspicion degree of muscle invasion or tumor stalk. A multivariate prediction model using the CT parameters for achieving high positive predictive value (PPV) for NMIBC was investigated. The PPVs from raw data or 1000 bootstrap resampling and inter-reader agreement using Gwet's AC1 were analyzed, respectively.

RESULTS

Proportion of patients with NMIBC was 81.0% (119/147). The CT criteria of the prediction model were as follows: (a) muscle invasion score < 2; (b) tumor stalk score > 0; and (c) tumor size < 3 cm. From the raw data, PPV of the model for NMIBC was 92.7% (51/55; 95% confidence interval [CI] 82.4-98.0) in reader 1 and 93.3% (42/45; 95% CI 81.7-98.6) in reader 2. From the bootstrap data, PPV was 92.8% (95% CI 85.2-98.3) in reader 1 and 93.4% (95% CI 84.9-99.9) in reader 2. The model's AC1 was 0.753 (95% CI 0.647-0.859).

CONCLUSION

The current CT-derived prediction model demonstrated high PPV for identifying patients with NMIBC. Depending on CT findings, approximately 30% of patients with bladder cancer may have a low need for additional MRI for interpreting vesical imaging-reporting and data system.

A Brief Review of Artificial Intelligence in Genitourinary Oncological Imaging

Genitourinary (GU) system is among the most commonly involved malignancy sites in the human body. Imaging plays a crucial role not only in diagnosis of cancer but also in disease management and its prognosis. However, interpretation of conventional imaging methods such as CT or MR imaging (MRI) usually demonstrates variability across different readers and institutions. Artificial intelligence (AI) has emerged as a promising technology that could improve the patient care by providing helpful input to human readers through lesion detection algorithms and lesion classification systems. Moreover, the robustness of these models may be valuable in automating time-consuming tasks such as organ and lesion segmentations. Herein, we review the current state of imaging and existing challenges in GU malignancies, particularly for cancers of prostate, kidney and bladder; and briefly summarize the recent AI-based solutions to these challenges.

Role of multiparametric MRI in characterization of complicated cystic renal masses

Background

Bosniak classification improves sensitivity and specificity for malignancy among cystic renal masses characterized with MRI. The quantitative parameters derived from diffusion-weighted imaging, and contrast enhancement, can be used in distinguishing between benign and malignant cystic renal masses.

Methods

This prospective observational study included 58 patients (39 male and 19 female) with complicated cystic renal mass initially diagnosed by US or CT. All patients underwent multiparametric MRI study (Pre- and Post-Gd-enhanced T1WI, T2WI and DWI) by using 3 Tesla MRI scanner. Each cystic renal lesion was assigned a category based on Bosniak classification. Demographic data were recorded. ADC ratio, dynamic enhancement parameters in both corticomedullary and nephrographic phases as well as absolute washout were calculated and compared using ROC curve analysis.

Results

The sensitivity, specificity, positive predictive value, negative predictive value, and overall accuracy of the multiparametric MRI in categorization of cystic renal masses according to Bosniak classification version 2019 were 90.32%, 100%, 100%, 90% and 94.83%, respectively, which was higher compared to biparametric MRI and conventional MRI.

Conclusions

Multiparametric MRI can be utilized to confidently evaluate cystic renal masses, overcoming the traditional limitations of overlapping morphological imaging features. Quantitative parameters derived from multiparametric MRI allow better evaluation of complex cystic renal tumors to distinguish between benign and malignant complex cystic renal lesions.

Inter-method agreement between wash-in and wash-out computed tomography for characterizing hyperattenuating adrenal lesions as adenomas or non-adenomas

OBJECTIVES

To investigate inter-method agreement between wash-out and wash-in computed tomography (CT) to determine whether hyperattenuating adrenal lesions are characterized as adenomas or non-adenomas.

METHODS

We evaluated 243 patients who underwent wash-out CT for a solid enhancing hyperattenuating (i.e., > 10 Hounsfield unit [HU]) adrenal mass of ≥ 1 to < 4 cm. Wash-out (absolute percentage wash-out [APW]; relative percentage wash-out [RPW]) and wash-in values (enhancement ratio [ER]; relative enhancement ratio [RER]) were analyzed by two independent readers. Diagnostic criteria of wash-out CT for adenoma were APW ≥ 60% or RPW ≥ 40% (conventional method). Three different criteria for wash-in CT were set: ER ≥ 3.0; RER ≥ 200%; and RER ≥ 210%. Concordance rate and inter-method agreement between wash-out and wash-in CT were investigated using Gwet's AC1.

RESULTS

For all lesions, concordance rates between wash-out and wash-in CT were > 83%. AC1 between conventional method and ER ≥ 3.0 or between conventional method and RER ≥ 200% were identically 0.843 for reader 1 and 0.776 for reader 2. AC1 between conventional method and RER ≥ 210% were 0.780 for reader 1 and 0.737 for reader 2. For lesions of > 10 to ≤ 30 HU, concordance rates between wash-out and wash-in CT were > 89%. AC1 between conventional method and ER ≥ 3.0 or between conventional method and RER ≥ 200% were identically 0.914 for reader 1 and 0.866 for reader 2. AC1 between conventional method and RER ≥ 210% were 0.888 for reader 1 and 0.874 for reader 2.

CONCLUSION

In approximately 90% of patients with a hyperattenuating adrenal lesion of ≥ 1 to < 4 cm and >10 to ≤ 30 HU, wash-out CT with 15-min contrast-enhanced images may be replaced by wash-in CT.

KEY POINTS

• An enhancement ratio of ≥ 3.0 or a relative enhancement ratio of ≥ 200% appears to be appropriate as the threshold of wash-in computed tomography (CT) comprising unenhanced and 1-min contrast-enhanced CT. • Measurement of enhancement ratio or relative enhancement ratio was reproducible. • We found good agreement between wash-in and wash-out CT for determining whether hyperattenuating adrenal lesions of ≥ 1 to < 4 cm and >10 to ≤ 30 Hounsfield unit would be characterized as adenomas.

Differentiating Benign From Malignant Cystic Renal Masses: A Feasibility Study of Computed Tomography Texture-Based Machine Learning Algorithms

OBJECTIVE

The Bosniak classification attempts to predict the likelihood of renal cell carcinoma (RCC) among cystic renal masses but is subject to interobserver variability and often requires multiphase imaging. Artificial intelligence may provide a more objective assessment. We applied computed tomography texture-based machine learning algorithms to differentiate benign from malignant cystic renal masses.

METHODS

This is an institutional review board-approved, Health Insurance Portability and Accountability Act-compliant retrospective study of 147 patients (mean age, 62.4 years; range, 28-89 years; 94 men) with 144 cystic renal masses (93 benign, 51 RCC); 69 were pathology proven (51 RCC, 18 benign), and 75 were considered benign based on more than 4 years of stability at follow-up imaging. Using a single image from a contrast-enhanced abdominal computed tomography scan, mean, SD, mean value of positive pixels, entropy, skewness, and kurtosis radiomics features were extracted. Random forest, multivariate logistic regression, and support vector machine models were used to classify each mass as benign or malignant with 10-fold cross validation. Receiver operating characteristic curves assessed algorithm performance in the aggregated test data.

RESULTS

For the detection of malignancy, sensitivity, specificity, positive predictive value, negative predictive value, and area under the curve were 0.61, 0.87, 0.72, 0.80, and 0.79 for the random forest model; 0.59, 0.87, 0.71, 0.79, and 0.80 for the logistic regression model; and 0.55, 0.86, 0.68, 0.78, and 0.76 for the support vector machine model.

CONCLUSION

Computed tomography texture-based machine learning algorithms show promise in differentiating benign from malignant cystic renal masses. Once validated, these may serve as an adjunct to radiologists' assessments.

CT-based Bosniak classification of cystic renal lesions: is version 2019 an improvement on version 2005?

OBJECTIVE

To compare the diagnostic performance and inter-reader agreement of the CT-based v2019 versus v2005 Bosniak classification systems for risk stratification of cystic renal lesions (CRL).

METHODS

This retrospective study included adult patients with CRL identified on CT scan between 2005 and 2018. The reference standard was histopathology or a minimum 4-year imaging follow-up. The studies were reviewed independently by five readers (three senior, two junior), blinded to pathology results and imaging follow-up, who assigned Bosniak categories based on the 2005 and 2019 versions. Diagnostic performance of v2005 and v2019 Bosniak classifications for distinguishing benign from malignant lesions was calculated by dichotomizing CRL into the potential for ablative therapy (III-IV) or conservative management (I-IIF). Inter-reader agreement was calculated using Light's Kappa.

RESULTS

One hundred thirty-nine patients with 149 CRL (33 malignant) were included. v2005 and v2019 Bosniak classifications achieved similar diagnostic performance with a sensitivity of 91% vs 91% and a specificity of 89% vs 88%, respectively. Inter-reader agreement for overall Bosniak category assignment was substantial for v2005 (κ = 0.78) and v2019 (κ = 0.75) between senior readers but decreased for v2019 when the Bosniak classification was dichotomized to conservative management (I-IIF) or ablative therapy (III-IV) (0.80 vs 0.71, respectively). For v2019, wall thickness was the morphological feature with the poorest inter-reader agreement (κ = 0.43 and 0.18 for senior and junior readers, respectively).

CONCLUSION

No significant improvement in diagnostic performance and inter-reader agreement was shown between v2005 and v2019. The observed decrease in inter-reader agreement in v2019 when dichotomized according to management strategy may reflect the more stringent morphological criteria.

KEY POINTS

• Versions 2005 and 2019 Bosniak classifications achieved similar diagnostic performance, but the specificity of higher risk categories (III and IV) was not increased while one malignant lesion was downgraded to v2019 Bosniak category II (i.e., not subjected to further follow-up). • Inter-reader agreement was similar between v2005 and v2019 but moderately decreased for v2019 when the Bosniak classification was dichotomized according to the potential need for ablative therapies (I-II-IIF vs III-IV).

Bosniak classification version 2019: a prospective comparison of CT and MRI

OBJECTIVE

To assess the diagnostic accuracy and agreement of CT and MRI in terms of the Bosniak classification version 2019 (BCv2019).

MATERIALS AND METHODS

A prospective multi-institutional study enrolled 63 patients with 67 complicated cystic renal masses (CRMs) discovered during ultrasound examination. All patients underwent CT and MRI scans and histopathology. Three radiologists independently assessed CRMs using BCv2019 and assigned Bosniak class to each CRM using CT and MRI. The final analysis included 60 histopathologically confirmed CRMs (41 were malignant and 19 were benign).

RESULTS

Discordance between CT and MRI findings was noticed in 50% (30/60) CRMs when data were analyzed in terms of the Bosniak classes. Of these, 16 (53.3%) were malignant. Based on consensus reviewing, there was no difference in the sensitivity, specificity, and accuracy of the BCv2019 with MRI and BCv2019 with CT (87.8%; 95% CI = 73.8-95.9% versus 75.6%; 95% CI = 59.7-87.6%; p = 0.09, 84.2%; 95% CI = 60.4-96.6% versus 78.9%; 95% CI = 54.4-93.9%; p = 0.5, and 86.7%; 95% CI = 64.0-86.6% versus 76.7%; 95% CI = 75.4-94.1%; p = 0.1, respectively). The number and thickness of septa and the presence of enhanced nodules accounted for the majority of variations in Bosniak classes between CT and MRI. The inter-reader agreement (IRA) was substantial for determining the Bosniak class in CT and MRI (k = 0.66; 95% CI = 0.54-0.76, k = 0.62; 95% CI = 0.50-0.73, respectively). The inter-modality agreement of the BCv219 between CT and MRI was moderate (κ = 0.58).

CONCLUSION

In terms of BCv2019, CT and MRI are comparable in the classification of CRMs with no significant difference in diagnostic accuracy and reliability.

KEY POINTS

• There is no significant difference in the sensitivity, specificity, and accuracy of the BCv2019 with MRI and BCv2019 with CT. • The number of septa and their thickness and the presence of enhanced nodules accounted for the majority of variations in Bosniak classes between CT and MRI. • The inter-reader agreement was substantial for determining the Bosniak class in CT and MRI and the inter-modality agreement of the BCv219 between CT and MRI was moderate.

Deep learning and radiomic feature-based blending ensemble classifier for malignancy risk prediction in cystic renal lesions

BACKGROUND

The rising prevalence of cystic renal lesions (CRLs) detected by computed tomography necessitates better identification of the malignant cystic renal neoplasms since a significant majority of CRLs are benign renal cysts. Using arterial phase CT scans combined with pathology diagnosis results, a fusion feature-based blending ensemble machine learning model was created to identify malignant renal neoplasms from cystic renal lesions (CRLs). Histopathology results were adopted as diagnosis standard. Pretrained 3D-ResNet50 network was selected for non-handcrafted features extraction and pyradiomics toolbox was selected for handcrafted features extraction. Tenfold cross validated least absolute shrinkage and selection operator regression methods were selected to identify the most discriminative candidate features in the development cohort. Feature's reproducibility was evaluated by intra-class correlation coefficients and inter-class correlation coefficients. Pearson correlation coefficients for normal distribution and Spearman's rank correlation coefficients for non-normal distribution were utilized to remove redundant features. After that, a blending ensemble machine learning model were developed in training cohort. Area under the receiver operator characteristic curve (AUC), accuracy score (ACC), and decision curve analysis (DCA) were employed to evaluate the performance of the final model in testing cohort.

RESULTS

The fusion feature-based machine learning algorithm demonstrated excellent diagnostic performance in external validation dataset (AUC = 0.934, ACC = 0.905). Net benefits presented by DCA are higher than Bosniak-2019 version classification for stratifying patients with CRL to the appropriate surgery procedure.

CONCLUSIONS

Fusion feature-based classifier accurately distinguished malignant and benign CRLs which outperformed the Bosniak-2019 version classification and illustrated improved clinical decision-making utility.

Nyquist sampling theorem and Bosniak classification, version 2019: effect of thin axial sections on categorization and agreement

OBJECTIVE

To determine if CT axial images reconstructed at current standard of care (SOC; 2.5-3 mm) or thin (≤ 1 mm) sections affect categorization and inter-rater agreement of cystic renal masses assessed with Bosniak classification, version 2019.

METHODS

In this retrospective single-center study, 3 abdominal radiologists reviewed 131 consecutive cystic renal masses from 100 patients performed with CT renal mass protocol from 2015 to 2021. Images were reviewed in two sessions: first with SOC and then the addition of thin sections. Individual and overall categorizations are reported, latter of which is based on majority opinion with 3-way discrepancies resolved by a fourth reader. Major categorization changes were defined as differences between classes I-II, IIF, or III-IV.

RESULTS

Thin sections led to a statistically significant major category change with class II for all readers individually (p = 0.004-0.041; McNemar test), upgrading 10-17% of class II masses, most commonly to class IIF followed by III. Modal reason for upgrades was due to identification of additional septa followed by larger measurement of enhancing features. Masses categorized as class I, III, or IV on SOC sections were unaffected, as were identification of protrusions. Inter-rater agreements using weighted Cohen's kappa were 0.679 for SOC and 0.691 for thin sections (both substantial).

CONCLUSION

Thin axial sections upgraded up to one in six class II masses to IIF or III through identification of additional septa or larger feature. Other classes, including III-IV, were unaffected. Inter-rater agreements were substantial regardless of section thickness.

KEY POINTS

• Thin axial sections (≤ 1 mm) compared to standard of care sections (2.5-3 mm) led to identification of additional septa but did not affect identification of protrusions. • Thin axial sections (≤ 1 mm) compared to standard of care sections (2.5-3 mm) can upgrade a small proportion of cystic renal masses from class II to IIF or III when applying Bosniak classification, version 2019. • Inter-rater agreements were substantial regardless of section thickness.