Bosniak classification of cystic renal masses version 2019 does not increase the interobserver agreement or the proportion of masses categorized into lower Bosniak classes for non-subspecialized readers on CT or MR

Validation of the updated Bosniak classification (2019) in pathologically confirmed CT-categorised cysts

INTRODUCTION

The updated Bosniak classification in 2019 (v2019) addresses vague imaging terms and revises the criteria with the intent to categorise a higher proportion of cysts in lower-risk groups and reduce benign cyst resections. The aim of the present study was to compare the diagnostic accuracy and inter-observer agreement rate of the original (v2005) and updated classifications (v2019).

METHOD

Resected/biopsied cysts were categorised according to Bosniak classifications (v2005 and v2019) and the diagnostic accuracy was assessed with reference to histopathological analysis. The inter-observer agreement of v2005 and v2019 was determined.

RESULTS

The malignancy rate of the cohort was 83.6% (51/61). Using v2019, a higher proportion of malignant cysts were categorised as Bosniak ≥ III (88.2% vs 84.3%) and a significantly higher percentage were categorised as Bosniak IV (68.9% vs 47.1%; p = 0.049) in comparison to v2005. v2019 would have resulted in less benign cyst resections (13.5% vs 15.7%). Calcified versus non-calcified cysts had lower rates of malignancy (57.1% vs 91.5%; RR,0.62; p = 0.002). The inter-observer agreement of v2005 was higher than that of v2019 (kappa, 0.70 vs kappa, 0.43).

DISCUSSION

The updated classification improves the categorisation of malignant cysts and reduces benign cyst resection. The low inter-observer agreement remains a challenge to the updated classification system.

Multiparametric MRI-Based Machine Learning Models for the Characterization of Cystic Renal Masses Compared to the Bosniak Classification, Version 2019: A Multicenter Study

RATIONALE AND OBJECTIVE

Accurate differentiation between benign and malignant cystic renal masses (CRMs) is challenging in clinical practice. This study aimed to develop MRI-based machine learning models for differentiating between benign and malignant CRMs and compare the best-performing model with the Bosniak classification, version 2019 (BC, version 2019).

METHODS

Between 2009 and 2021, consecutive surgery-proven CRM patients with renal MRI were enrolled in this multicenter study. Models were constructed to differentiate between benign and malignant CRMs using logistic regression (LR), random forest (RF), and support vector machine (SVM) algorithms, respectively. Meanwhile, two radiologists classified CRMs into I-IV categories according to the BC, version 2019 in consensus in the test set. A subgroup analysis was conducted to investigate the performance of the best-performing model in complicated CRMs (II-IV lesions in the test set). The performances of models and BC, version 2019 were evaluated using the area under the receiver operating characteristic curve (AUC). Performance was statistically compared between the best-performing model and the BC, version 2019.

RESULTS

278 and 48 patients were assigned to the training and test sets, respectively. In the test set, the AUC and accuracy of the LR model, the RF model, the SVM model, and the BC, version 2019 were 0.884 and 75.0%, 0.907 and 83.3%, 0.814 and 72.9%, and 0.893 and 81.2%, respectively. Neither the AUC nor the accuracy of the RF model that performed best were significantly different from the BC, version 2019 (P = 0.780, P = 0.065). The RF model achieved an AUC and accuracy of 0.880 and 81.0% in complicated CRMs.

CONCLUSIONS

The MRI-based RF model can accurately differentiate between benign and malignant CRMs with comparable performance to the BC, version 2019, and has good performance in complicated CRMs, which may facilitate treatment decision-making and is less affected by interobserver disagreements.

Bosniak Classification of Cystic Renal Masses: Looking Back, Looking Forward

RATIONALE AND OBJECTIVES

According to the 2019 update of the Bosniak classification, the main imaging features that need to be evaluated to achieve a correct characterization of renal cystic masses include the thickness of walls and septa, the number of septa, the appearance of walls and septa, the attenuation/intensity on non-contrast CT/MRI and the presence of unequivocally perceived or measurable enhancement of walls and septa. Despite the improvement deriving from a quantitative evaluation of imaging features, certain limitations seem to persist and some possible scenarios that can be encountered in clinical practice are still missing.

MATERIALS AND METHODS

A deep analysis of the 2019 update of the Bosniak classification was performed.

RESULTS

The most notable potential flaws concern: (1) the quantitative measurement of the walls and septa; (2) the fact that walls and septa > 2 mm are always referred to as "enhancing", not considering the alternative scenario; (3) the description of some class II masses partially overlaps with each other and with the definition of class I masses and (4) the morphological variations of cystic masses over time is not considered.

CONCLUSION

The present paper analyzes in detail the limitations of the 2019 Bosniak classification to improve this important tool and facilitate its use in daily radiological practice.

CEUS Bosniak Classification-Time for Differentiation and Change in Renal Cyst Surveillance

It is time for a change. CEUS is an established method that should be much more actively included in renal cyst monitoring strategies. This review compares the accuracies, strengths, and weaknesses of CEUS, CECT, and MRI in the classification of renal cysts. In order to avoid overstaging by CEUS, a further differentiation of classes IIF, III, and IV is required. A further development in the refinement of the CEUS-Bosniak classification aims to integrate CEUS more closely into the monitoring of renal cysts and to develop new and complex monitoring algorithms.

Differentiating Benign From Malignant Cystic Renal Masses: A Feasibility Study of Computed Tomography Texture-Based Machine Learning Algorithms

OBJECTIVE

The Bosniak classification attempts to predict the likelihood of renal cell carcinoma (RCC) among cystic renal masses but is subject to interobserver variability and often requires multiphase imaging. Artificial intelligence may provide a more objective assessment. We applied computed tomography texture-based machine learning algorithms to differentiate benign from malignant cystic renal masses.

METHODS

This is an institutional review board-approved, Health Insurance Portability and Accountability Act-compliant retrospective study of 147 patients (mean age, 62.4 years; range, 28-89 years; 94 men) with 144 cystic renal masses (93 benign, 51 RCC); 69 were pathology proven (51 RCC, 18 benign), and 75 were considered benign based on more than 4 years of stability at follow-up imaging. Using a single image from a contrast-enhanced abdominal computed tomography scan, mean, SD, mean value of positive pixels, entropy, skewness, and kurtosis radiomics features were extracted. Random forest, multivariate logistic regression, and support vector machine models were used to classify each mass as benign or malignant with 10-fold cross validation. Receiver operating characteristic curves assessed algorithm performance in the aggregated test data.

RESULTS

For the detection of malignancy, sensitivity, specificity, positive predictive value, negative predictive value, and area under the curve were 0.61, 0.87, 0.72, 0.80, and 0.79 for the random forest model; 0.59, 0.87, 0.71, 0.79, and 0.80 for the logistic regression model; and 0.55, 0.86, 0.68, 0.78, and 0.76 for the support vector machine model.

CONCLUSION

Computed tomography texture-based machine learning algorithms show promise in differentiating benign from malignant cystic renal masses. Once validated, these may serve as an adjunct to radiologists' assessments.

CT-based Bosniak classification of cystic renal lesions: is version 2019 an improvement on version 2005?

OBJECTIVE

To compare the diagnostic performance and inter-reader agreement of the CT-based v2019 versus v2005 Bosniak classification systems for risk stratification of cystic renal lesions (CRL).

METHODS

This retrospective study included adult patients with CRL identified on CT scan between 2005 and 2018. The reference standard was histopathology or a minimum 4-year imaging follow-up. The studies were reviewed independently by five readers (three senior, two junior), blinded to pathology results and imaging follow-up, who assigned Bosniak categories based on the 2005 and 2019 versions. Diagnostic performance of v2005 and v2019 Bosniak classifications for distinguishing benign from malignant lesions was calculated by dichotomizing CRL into the potential for ablative therapy (III-IV) or conservative management (I-IIF). Inter-reader agreement was calculated using Light's Kappa.

RESULTS

One hundred thirty-nine patients with 149 CRL (33 malignant) were included. v2005 and v2019 Bosniak classifications achieved similar diagnostic performance with a sensitivity of 91% vs 91% and a specificity of 89% vs 88%, respectively. Inter-reader agreement for overall Bosniak category assignment was substantial for v2005 (κ = 0.78) and v2019 (κ = 0.75) between senior readers but decreased for v2019 when the Bosniak classification was dichotomized to conservative management (I-IIF) or ablative therapy (III-IV) (0.80 vs 0.71, respectively). For v2019, wall thickness was the morphological feature with the poorest inter-reader agreement (κ = 0.43 and 0.18 for senior and junior readers, respectively).

CONCLUSION

No significant improvement in diagnostic performance and inter-reader agreement was shown between v2005 and v2019. The observed decrease in inter-reader agreement in v2019 when dichotomized according to management strategy may reflect the more stringent morphological criteria.

KEY POINTS

• Versions 2005 and 2019 Bosniak classifications achieved similar diagnostic performance, but the specificity of higher risk categories (III and IV) was not increased while one malignant lesion was downgraded to v2019 Bosniak category II (i.e., not subjected to further follow-up). • Inter-reader agreement was similar between v2005 and v2019 but moderately decreased for v2019 when the Bosniak classification was dichotomized according to the potential need for ablative therapies (I-II-IIF vs III-IV).

Proportion of malignancy in Bosniak classification of cystic renal masses version 2019 (v2019) classes: systematic review and meta-analysis

OBJECTIVES

Determine the proportion of malignancy within Bosniak v2019 classes.

METHODS

MEDLINE and EMBASE were searched. Eligible studies contained patients with cystic renal masses undergoing CT or MRI renal protocol examinations with pathology confirmation, applying Bosniak v2019. Proportion of malignancy was estimated within Bosniak v2019 class. Risk of bias was assessed using QUADAS-2.

RESULTS

We included 471 patients with 480 cystic renal masses. No class I malignant masses were observed. Pooled proportion of malignancy were class II, 12% (6/51, 95% CI 5-24%); class IIF, 46% (37/85, 95% CI 28-66%); class III, 79% (138/173, 95% CI 68-88%); and class IV, 84% (114/135, 95% CI 77-90%). Proportion of malignancy differed between Bosniak v2019 II-IV classes (p = 0.004). Four studies reported the proportion of malignancy by wall/septa feature. The pooled proportion of malignancy with 95% CI were class III thick smooth wall/septa, 77% (41/56, 95% CI 53-91%); class III obtuse protrusion ≤ 3 mm (irregularity), 83% (97/117, 95% CI 75-89%); and class IV nodule with acute angulation, 86% (50/58, 95% CI 75-93%) or obtuse angulation ≥ 4 mm, 83%, (64/77, 95% CI 73-90%). Subgroup analysis by wall/septa feature was limited by sample size; however, no differences were found comparing class III masses with irregularity to class IV masses (p = 0.74) or between class IV masses by acute versus obtuse angles (p = 0.62).

CONCLUSION

Preliminary data suggest Bosniak v2019 class IIF masses have higher proportion of malignancy compared to the original classification, controlling for pathologic reference standard. There are no differences in proportion of malignancy comparing class III masses with irregularities to class IV masses with acute or obtuse nodules.

KEY POINTS

• The proportion of malignancy in Bosniak v2019 class IIF cystic masses is 46% (37 malignant/85 total IIF masses, 95% confidence intervals (CI) 28-66%). • The proportion of malignancy in Bosniak v2019 class III cystic masses is 79% (138/173, 95% CI 68-88%) and in Bosniak v2019 class IV cystic masses is 84% (114/135, 95% CI 77-90%). • Class III cystic masses with irregularities had similar proportion of malignancy (83%, 97/117, 95% CI 75-89%) compared to Bosniak class IV masses (84%, 114/135, 95% CI 77-90%) overall (p = 0.74) with no difference within class IV masses by acute versus obtuse angulation (p = 0.62).

Deep learning and radiomic feature-based blending ensemble classifier for malignancy risk prediction in cystic renal lesions

BACKGROUND

The rising prevalence of cystic renal lesions (CRLs) detected by computed tomography necessitates better identification of the malignant cystic renal neoplasms since a significant majority of CRLs are benign renal cysts. Using arterial phase CT scans combined with pathology diagnosis results, a fusion feature-based blending ensemble machine learning model was created to identify malignant renal neoplasms from cystic renal lesions (CRLs). Histopathology results were adopted as diagnosis standard. Pretrained 3D-ResNet50 network was selected for non-handcrafted features extraction and pyradiomics toolbox was selected for handcrafted features extraction. Tenfold cross validated least absolute shrinkage and selection operator regression methods were selected to identify the most discriminative candidate features in the development cohort. Feature's reproducibility was evaluated by intra-class correlation coefficients and inter-class correlation coefficients. Pearson correlation coefficients for normal distribution and Spearman's rank correlation coefficients for non-normal distribution were utilized to remove redundant features. After that, a blending ensemble machine learning model were developed in training cohort. Area under the receiver operator characteristic curve (AUC), accuracy score (ACC), and decision curve analysis (DCA) were employed to evaluate the performance of the final model in testing cohort.

RESULTS

The fusion feature-based machine learning algorithm demonstrated excellent diagnostic performance in external validation dataset (AUC = 0.934, ACC = 0.905). Net benefits presented by DCA are higher than Bosniak-2019 version classification for stratifying patients with CRL to the appropriate surgery procedure.

CONCLUSIONS

Fusion feature-based classifier accurately distinguished malignant and benign CRLs which outperformed the Bosniak-2019 version classification and illustrated improved clinical decision-making utility.

Stratification of malignant renal neoplasms from cystic renal lesions using deep learning and radiomics features based on a stacking ensemble CT machine learning algorithm

Using nephrographic phase CT images combined with pathology diagnosis, we aim to develop and validate a fusion feature-based stacking ensemble machine learning model to distinguish malignant renal neoplasms from cystic renal lesions (CRLs). This retrospective research includes 166 individuals with CRLs for model training and 47 individuals with CRLs in another institution for model testing. Histopathology results are adopted as diagnosis criterion. Nephrographic phase CT scans are selected to build the fusion feature-based machine learning algorithms. The pretrained 3D-ResNet50 CNN model and radiomics methods are selected to extract deep features and radiomics features, respectively. Fivefold cross-validated least absolute shrinkage and selection operator (LASSO) regression methods are adopted to identify the most discriminative candidate features in the development cohort. Intraclass correlation coefficients and interclass correlation coefficients are employed to evaluate feature’s reproducibility. Pearson correlation coefficients for normal distribution features and Spearman’s rank correlation coefficients for non-normal distribution features are used to eliminate redundant features. After that, stacking ensemble machine learning models are developed in the training cohort. The area under the receiver operator characteristic curve (ROC), calibration curve, and decision curve analysis (DCA) are adopted in the testing cohort to evaluate the performance of each model. The stacking ensemble machine learning algorithm reached excellent diagnostic performance in the testing dataset. The calibration plot shows good stability when using the stacking ensemble model. Net benefits presented by DCA are higher than the Bosniak 2019 version classification when employing any machine learning algorithm. The fusion feature-based machine learning algorithm accurately distinguishes malignant renal neoplasms from CRLs, which outperformed the Bosniak 2019 version classification, and proves to be more applicable for clinical decision-making.

Diagnostic performance of the Bosniak classification, version 2019 for cystic renal masses: A systematic review and meta-analysis

Purpose

To systematically assess the diagnostic performance of the Bosniak classification, version 2019 for risk stratification of cystic renal masses.

Methods

We conducted an electronic literature search on Web of Science, MEDLINE (Ovid and PubMed), Cochrane Library, EMBASE, and Google Scholar to identify relevant articles between June 1, 2019 and March 31, 2022 that used the Bosniak classification, version 2019 for risk stratification of cystic renal masses. Summary estimates of sensitivity, specificity, positive likelihood ratio (LR+), negative likelihood ratio (LR−), and diagnostic odds ratio (DOR) were pooled with the bivariate model and hierarchical summary receiver operating characteristic (HSROC) model. The quality of the included studies was assessed with the Quality Assessment of Diagnostic Accuracy Studies-2 tool.

Results

A total of eight studies comprising 720 patients were included. The pooled sensitivity and specificity were 0.85 (95% CI 0.79–0.90) and 0.68 (95% CI 0.58–0.76), respectively, for the class III/IV threshold, with a calculated area under the HSROC curve of 0.84 (95% CI 0.81–0.87). The pooled LR+, LR−, and DOR were 2.62 (95% CI 2.0–3.44), 0.22 (95% CI 0.16–0.32), and 11.7 (95% CI 6.8–20.0), respectively. The Higgins I² statistics demonstrated substantial heterogeneity across studies, with an I² of 57.8% for sensitivity and an I² of 74.6% for specificity. In subgroup analyses, the pooled sensitivity and specificity for CT were 0.86 and 0.71, respectively, and those for MRI were 0.87 and 0.67, respectively. In five studies providing a head-to-head comparison between the two versions of the Bosniak classification, the 2019 version demonstrated significantly higher specificity (0.62 vs. 0.41, p < 0.001); however, it came at the cost of a significant decrease in sensitivity (0.88 vs. 0.94, p = 0.001).

Conclusions

The Bosniak classification, version 2019 demonstrated moderate sensitivity and specificity, and there was no difference in diagnostic accuracy between CT and MRI. Compared to version 2005, the Bosniak classification, version 2019 has the potential to significantly reduce overtreatment, but at the cost of a substantial decline in sensitivity.

Interrater Agreement of Bosniak Classification Version 2019 and Version 2005 for Cystic Renal Masses at CT and MRI

Background The Bosniak classification system for cystic renal masses was updated in 2019 in part to improve agreement compared with the 2005 version. Purpose To compare and investigate interrater agreement of Bosniak version 2019 and Bosniak version 2005 at CT and MRI. Materials and Methods In this retrospective single-center study, a blinded eight-reader assessment was performed in which 195 renal masses prospectively considered Bosniak IIF-IV (95 at CT, 100 at MRI, from 2006 to 2019 with version 2005) were re-evaluated with Bosniak versions 2019 and 2005. Radiologists (four faculty members, four residents) who were blinded to the initial clinical reading and histopathologic findings assessed all feature components and reported the overall Bosniak class for each system independently. Agreement was assessed with Gwet agreement coefficients. Uni- and multivariable linear regression models were developed to identify predictors of dispersion in the final Bosniak class assignment that could inform system refinement. Results A total of 185 patients were included (mean age, 63 years ± 13 [standard deviation]; 118 men). Overall interrater agreement was similar between Bosniak version 2019 and version 2005 (Gwet agreement coefficient: 0.51 [95% CI: 0.45, 0.57] vs 0.46 [95% CI: 0.42, 0.51]). This was true for experts (0.54 vs 0.49) and novices (0.50 vs 0.47) and at CT (0.56 vs 0.51) and MRI (0.52 vs 0.43). Nine percent of masses prospectively considered cystic using Bosniak version 2005 criteria were considered solid using version 2019 criteria. In general, masses were more commonly classified in lower categories when radiologists used Bosniak version 2019 criteria compared with version 2005 criteria. The sole predictor of dispersion in Bosniak version 2019 class assignment was dispersion in septa or wall quality (ie, smooth vs irregular thickening vs nodule; 72% [MRI] and 60% [CT] overall model variance explained; multivariable P < .001). Conclusion Overall interrater agreement was similar between Bosniak version 2019 and version 2005; disagreements in septa or wall quality were common and strongly predictive of variation in Bosniak class assignment. © RSNA, 2021 Online supplemental material is available for this article. See also the editorial by Eberhardt in this issue.

Ultrasound Liver Imaging Reporting and Data System (US LI-RADS) Visualization Score: a reliability analysis on inter-reader agreement

BACKGROUND & AIM

The American College of Radiology Ultrasound Liver Imaging Reporting and Data System (ACR US LI-RADS) Visualization Score conveys the expected level of sensitivity of screening and surveillance ultrasound exams in patients at risk for hepatocellular carcinoma (HCC). We sought to determine inter-reader agreement of the Visualization Score which is currently unknown.

METHODS

Consecutive 6998 ultrasound HCC screening and surveillance studies in 3115 patients from 2017 to 2020 were retrospectively retrieved. Of these, 6154 (87.9%) studies were Visualization A (No or minimal limitations), 709 (10.1%) were Visualization B (Moderate limitations), and 135 (1.9%) were Visualization C (Severe limitations). Randomly sampled 90 studies, with 30 studies in each Visualization category, were included for analysis. Nine radiologists (3 senior attendings, 3 junior attendings and 3 body imaging fellows) blinded to the original categorization independently reviewed each study and assigned a Visualization Score. Intraclass correlation coefficient (ICC) was used to quantify inter-reader agreement.

RESULTS

ICC among all 9 radiologists was 0.70 (95% CI 0.63-0.77). ICCs among senior attendings, junior attendings and body imaging fellows were 0.68 (CI 0.58-0.76), 0.72 (CI 0.62-0.80) and 0.76 (CI 0.68-0.83), respectively. Subgroup analysis by liver parenchyma was further performed. ICC was highest in the patient group with normal liver parenchyma (0.69, CI 0.56-0.81), followed by steatosis (0.66, CI 0.54-0.79) and cirrhosis (0.58, CI 0.43-0.73), respectively.

CONCLUSIONS

US LI-RADS Visualization Score is a reliable tool with good inter-reader agreement that can be used to indicate the expected level of sensitivity of a screening and surveillance ultrasound examination for detecting focal liver observations.

Comparison of Bosniak Classification of cystic renal masses version 2019 assessed by CT and MRI

OBJECTIVE

To compare imaging features in cystic masses imaged with both CT and MRI using Bosniak Classification version 2019 (Bosniak.v2019) and original Bosniak Classification (Bosniak.original).

MATERIALS AND METHODS

This IRB-approved, retrospective, cross-sectional study evaluated sixty-five consecutively identified cystic (≤ 25% enhancing) masses imaged by CT and MRI between 2009 and 2019: 35 with histologic diagnosis and 30 Bosniak.v2019 Class 2 and Class 2F cystic masses verified by an expert radiologist (R1) with minimum 5-year stability. Three radiologists (R2, R3, R4) independently evaluated CT, followed by MRI and assigned Bosniak.original and Bosniak.v2019 class in two sessions separated by ≥ 1 month and assessed the following: septa number, septa/wall thickness, and protrusions. Discrepancies were resolved by consensus with R1.

RESULTS

There was 70.8% agreement (kappa = 0.60, p = 0.0146) in class assigned by CT versus MRI for Bosniak.original and 72.3% agreement (kappa = 0.63, p = 0.006) for Bosniak.v2019. Increased septa number (p < 0.001) and more protrusions (p = 0.034) were identified on MRI, with no differences in septal/wall thickness (p = 0.067, 0.855) or protrusion size (p = 0.467). For both CT and MRI, Bosniak.v2019 improved specificity (79.0% [95% confidence interval 71.0-87.0%] CT, 70% [62.0-77.0%] MRI) compared to Bosniak.original (63.0% [56.0-69.0%] CT, 66.0% [58.0-74.0%] MRI) with maintained sensitivity and higher overall accuracy. Inter-observer agreement was similar-to-slightly higher for Bosniak.v2019 (K = 0.44 CT, 0.39 MRI) versus Bosniak.original (K = 0.35 CT, 0.37 MRI).

CONCLUSION

Class assignment differs in cystic masses evaluated by CT versus MRI for original and v2019 Bosniak Classification with similar-to-slightly higher agreement and improved specificity and higher overall accuracy on both CT and MRI with Bosniak version 2019.

Cross-sectional imaging assessment of renal masses with emphasis on MRI

Magnetic resonance imaging (MRI) is a useful complementary imaging tool for the diagnosis and characterization of renal masses, as it provides both morphologic and functional information. A core MRI protocol for renal imaging should include a T1-weighted sequence with in- and opposed-phase images (or, alternatively with DIXON technique), T2-weighted and diffusion-weighted images as well as a dynamic contrast-enhanced sequence with subtraction images, followed by a delayed post-contrast T1-weighted sequence. The main advantages of MRI over computed tomography include increased sensitivity for contrast enhancement, less sensitivity for detection of calcifications, absence of pseudoenhancement, and lack of radiation exposure. MRI may be applied for renal cystic lesion characterization, differentiation of renal cell carcinoma (RCC) from benign solid renal tumors, RCC histologic grading, staging, post-treatment follow-up, and active surveillance of patients with treated or untreated RCC.

Bosniak Classification of Cystic Renal Masses Version 2019: Comparison to Version 2005 for Class Distribution, Diagnostic Performance, and Interreader Agreement Using CT and MRI

Background: The Bosniak classification system for cystic renal masses (CRMs) was updated in 2019, requiring further investigation. Objective: To compare version 2005 and version 2019 of the Bosniak classification system in terms of class distribution, diagnostic performance, inter-reader agreement, and inter-modality agreement between CT and MRI. Methods: This retrospective study included 100 patients (mean age, 52.4±11.6 years; 68 men, 32 women) with 104 CRMs (74 malignant) who underwent CT, MRI, and resection between 2010 and 2019. Two radiologists independently evaluated CRMs in separate sessions for each combination of version and modality and assigned a Bosniak class. Diagnostic performance was compared using McNemar tests. Inter-reader and inter-modality agreement were analyzed using weighted kappa coefficients. Results: Across readers and modalities, proportion of class IIF was higher for version 2019 than version 2005 (reader 1: 28.8%-30.8% vs 6.7%-12.5%; reader 2: 26.0%-28.8% vs 8.7%-19.2%), although 95% CIs overlapped for reader 2 on CT. Proportion of class III was lower for version 2019 than version 2005 (reader 1: 33.7%-35.6% vs 49%-51.9%; reader 2: 31.7%-40.4% vs 37.5%-52.9%), although 95% CIs overlapped for all comparisons. Version 2019 demonstrated lower sensitivity for malignancy than version 2005 across readers and modalities (all p<.05); for example, using CT, sensitivity was 75.7% for both readers with version 2019, versus 85.1%-87.8% with version 2005. However, version 2019 demonstrated higher specificity than version 2005, which was significant (all p<.05) for reader 1 For example, using CT, specificity was 73.3% (reader 1) and 70.0% (reader 2) with version 2019, versus 50.0% (reader 1) and 56.7% (reader 2) with version 2005. Diagnostic accuracy was not different between versions (version 2005: 76.9%-85.6%; version 2019: 74.0%-78.8%). Inter-reader and inter- modality agreement were substantial for version 2005 (κ=0.676-0.782; 0.711-0.723) and version 2019 (κ=0.756-0.804; 0.704-0.781). Conclusion: Version 2019, versus version 2005, results in shift in CRM assignment from class III to class IIF. Version 2019 results in lower sensitivity, higher specificity, and similar accuracy versus version 2005. Inter-reader and inter-modality agreement are similar between versions. Clinical impact: Version 2019 facilitates recommending imaging surveillance for more CRMs.

Bosniak Classification version 2019: validation and comparison to original classification in pathologically confirmed cystic masses

OBJECTIVE

To evaluate Bosniak Classification v2019 definitions in pathologically confirmed cystic renal masses.

MATERIALS AND METHODS

Seventy-three cystic (≤ 25% solid) masses with histological confirmation (57 malignant, 16 benign) imaged by CT (N = 28) or CT+MRI (N = 56) between 2009 and 2019 were independently evaluated by three blinded radiologists using Bosniak v2019 and original classifications. Discrepancies were resolved by consensus with a fourth blinded radiologist. Overall class and v2019 features were compared to pathology.

RESULTS

Inter-observer agreement was slightly improved comparing v2019 to Original Bosniak Classification (kappa = 0.26-0.47 versus 0.24-0.34 respectively). v2019 proportion of IIF and III masses (20.5% [15/73, 95% confidence interval (CI) 12.0-31.6%], 38.6% [28/73, 95% CI 27.2-50.5%]) differed from the original classification (6.8% [5/73, 95% CI 2.3-15.3%], 61.6% [45/73, 95% CI 49.5-72.8%]) with overlapping proportion of malignancy in each class. Mean septa number (7 ± 4 [range 1-10]) was not associated with malignancy (p = 0.89). Mean wall and septa thicknesses were 3 ± 3 (1-14) and 3 ± 2 (1-10) mm and higher in malignancies (p = 0.03 and 0.20 respectively). Areas under the receiver-operator-characteristic curve for wall and septa thickness were 0.66 (95% CI 0.54-0.79) and 0.61 (95% CI 0.45-0.78) with an optimal cut point of ≥ 3 mm (sensitivity 33.3%, specificity 86.7% and sensitivity 53%, specificity 73% respectively). Proportion of malignancy occurring in masses with the v2019 features "irregularity" (76.9% [10/13], 95% CI 46.2-94.9%) and "nodule" (89.7% [26/29], 95% CI 72.7-97.8%) overlapped. Angle of "nodule" (p = 0.27) was not associated with malignancy.

CONCLUSION

Bosniak v2019 definitions for wall/septa thickness and protrusions are associated with malignancy. Overall, Bosniak v2019 categorizes a higher proportion of malignant masses in Class IIF with slight improvement in inter-observer agreement.

KEY POINTS

• Considering Bosniak v2019 Class IIF cystic masses with many (≥ 4) smooth and thin septa, there was no association between the number of septa and malignancy (p = 0.89) in this study. • Increased cyst wall and septa thickness are associated with malignancy and a lower threshold of ≥ 3 mm maximized overall diagnostic accuracy compared to ≥ 4 mm threshold proposed for Bosniak v2019 Class 3. • An overlapping proportion of malignant masses is noted in Bosniak v2019 Class 3 masses with "irregularity" (76.9% [10/13], 95% CI 46.2-94.9%) compared to Bosniak v2019 Class 4 masses with "nodule" (89.7% [26/29], 95% CI 72.7-97.8%).