Validity and Reproducibility of the ADNEX MR Scoring System in the Diagnosis of Sonographically Indeterminate Adnexal Masses

ACR Appropriateness Criteria® Clinically Suspected Adnexal Mass, No Acute Symptoms: 2023 Update

Asymptomatic adnexal masses are commonly encountered in daily radiology practice. Although the vast majority of these masses are benign, a small subset have a risk of malignancy, which require gynecologic oncology referral for best treatment outcomes. Ultrasound, using a combination of both transabdominal, transvaginal, and duplex Doppler technique can accurately characterize the majority of these lesions. MRI with and without contrast is a useful complementary modality that can help characterize indeterminate lesions and assess the risk of malignancy is those that are suspicious. The American College of Radiology Appropriateness Criteria are evidence-based guidelines for specific clinical conditions that are reviewed annually by a multidisciplinary expert panel. The guideline development and revision process support the systematic analysis of the medical literature from peer reviewed journals. Established methodology principles such as Grading of Recommendations Assessment, Development, and Evaluation or GRADE are adapted to evaluate the evidence. The RAND/UCLA Appropriateness Method User Manual provides the methodology to determine the appropriateness of imaging and treatment procedures for specific clinical scenarios. In those instances where peer reviewed literature is lacking or equivocal, experts may be the primary evidentiary source available to formulate a recommendation.

O-RADS MRI scoring system: key points for correct application in inexperienced hands

OBJECTIVES

To evaluate the efficacy of the O-RADS MRI criteria in the stratification of risk of malignancy of solid or sonographically indeterminate ovarian masses and assess the interobserver agreement of this classification between experienced and inexperienced radiologists.

METHODS

This single-centre retrospective study included patients from 2019 to 2022 with sonographically indeterminate or solid ovarian masses who underwent MRI with a specific protocol for characterisation according to O-RADS MRI specifications. Each study was evaluated using O-RADS lexicon by two radiologists, one with 17 years of experience in gynaecological radiology and another with 4 years of experience in general radiology. Findings were classified as benign, borderline, or malignant according to histology or stability over time. Diagnostic performance and interobserver agreement were assessed.

RESULTS

A total of 183 patients with US indeterminate or solid adnexal masses were included. Fifty-seven (31%) did not have ovarian masses, classified as O-RADS 1. The diagnostic performance for scores 2-5 was excellent with a sensitivity, specificity, PPV, and NPV of 97.4%, 100%, 96.2%, and 100%, respectively by the experienced radiologist and 96.1%, 92.0%, 93.9%, and 94.8% by the inexperienced radiologist. Interobserver concordance was very high (Kappa index 0.92). Almost all the misclassified cases were due to misinterpretation of the classification similar to reports in the literature.

CONCLUSION

The diagnostic performance of O-RADS MRI determined by either experienced or inexperienced radiologists is excellent, facilitating decision-making with high diagnostic accuracy and high reproducibility. Knowledge of this classification and use of assessment tools could avoid frequent errors due to misinterpretation.

CRITICAL RELEVANCE STATEMENT

Up to 31% of ovarian masses are considered indeterminate by transvaginal US and 32% of solid lesions considered malignant by transvaginal US are benign. The O-RADs MRI accurately classifies these masses, even when used by inexperienced radiologists, thereby avoiding incorrect surgical approaches.

KEY POINTS

• O-RADS MRI accurately classifies indeterminate and solid ovarian masses by ultrasound. • There is excellent interobserver agreement between experienced and non-experienced radiologists. • O-RADS MRI is a helpful tool to assess clinical decision-making in ovarian tumours.

Modified MR scoring system for assessment of sonographically indeterminate ovarian and adnexal masses in the absence of dynamic contrast-enhanced

OBJECTIVES

Dynamic contrast-enhanced (DCE) MRI is not available in all imaging centres to investigate adnexal masses. We proposed modified magnetic resonance (MR) scoring system based on an assessment of the enhancement of the solid tissue on early phase postcontrast series and diffusion-weighted imaging (DWI) with apparent diffusion coefficient (ADC) map and investigated the validity of this protocols in the current study.

MATERIALS AND METHODS

In this cross-sectional retrospective study, pelvic MRI of a total of 245 patients with 340 adnexal masses were studied based on the proposed modified scoring system and ADNEX MR scoring system.

RESULTS

Modified scoring system with the sensitivity of 87.3% and specificity of 94.6% has an accuracy of 92.1%. Sensitivity, specificity, and accuracy of ADNEX MR scoring system is 96.6%, 91%, and 92.9%, respectively. The area under the receiver operating characteristic curve for the modified scoring system and ADNEX MR scoring system is 0.909 (with 0.870-0.938 95% confidence interval [CI]) and 0.938 (with 0.907-0.961 95% CI), respectively. Pairwise comparison of these area under the curves showed no significant difference (P = .053).

CONCLUSIONS

Modified scoring system is less sensitive than the ADNEX MR scoring system and more specific but the accuracy is not significantly different.

ADVANCES IN KNOWLEDGE

According to our study, MR scoring system based on subjective assessment of the enhancement of the solid tissue on early phase postcontrast series and DWI with ADC map could be applicable in imaging centres that DCE is not available.

Comparative Study of the Efficacy of the Ovarian-Adnexa Reporting and Data System Ultrasound Combined With Contrast-Enhanced Ultrasound and the ADNEX MR Scoring System in the Diagnosis of Adnexal Masses

OBJECTIVE

The aims of this study were to develop the Ovarian-Adnexa Reporting and Data System (O-RADS) and O-RADS + contrast-enhanced ultrasound (O-RADS CEUS) scoring system to distinguish adnexal masses (AMs) and to compare the diagnostic efficacy of these systems with that of a magnetic resonance imaging scoring system (ADNEX MR).

METHODS

We retrospectively evaluated 278 ovarian masses from 240 patients between May 2017 and July 2022. Pathology and adequate follow-up were used as reference standards for comparing the validity of O-RADS, O-RADS CEUS and ADNEX MR scoring to diagnose AMs. Area under the curve (AUC), sensitivity and specificity were calculated. The inter-class correlation coefficient (ICC) was calculated to evaluate inter-reader agreement (IRA) between the two sonographers and two radiologists who analyzed the findings with the three modalities.

RESULTS

The AUCs of O-RADS, O-RADS CEUS and ADNEX MR scores were 0.928 (95% confidence interval [CI]: 0.895-0.956), 0.951(95% CI: 0.919-0.973) and 0.964 (95% CI: 0.935-0.983), respectively. Their sensitivities were 95.7%, 94.3 and 91.4%, and their specificities were 81.3%, 92.3% and 97.1%, respectively. The three modalities had accuracies of 84.9%, 92.8% and 95.7%, respectively. O-RADS had the highest sensitivity but significantly lower specificity (p < 0.001), whereas the ADNEX MR scoring had the highest specificity (p < 0.001) but lower sensitivity (p < 0.001). O-RADS CEUS had intermediate sensitivity and specificity (p < 0.001).

CONCLUSION

The addition of CEUS significantly improves the efficacy of O-RADS in diagnosing AMs. The diagnostic efficacy of the combination is comparable to that of the ADNEX MR scoring system.

Ovarian-Adnexal Reporting and Data System for Magnetic Resonance Imaging (O-RADS MRI): Genesis and Future Directions

Imaging plays an important role in characterizing and risk-stratifying commonly encountered adnexal lesions. Recently, the American College of Radiology (ACR) released the Ovarian-Adnexal Reporting and Data System (O-RADS) for ultrasound and subsequently for magnetic resonance imaging (MRI). The goal of the recently developed ACR O-RADS MRI risk stratification system is to improve the quality of imaging reports as well as the reproducibility of evaluating adnexal lesions on MRI. This review focuses on exploring this new system and its future refinements.

Impact of DWI and ADC values in Ovarian-Adnexal Reporting and Data System (O-RADS) MRI score

PURPOSE

Introduce DWI and quantitative ADC evaluation in O-RADS MRI system and observe how diagnostic performance changes. Assess its validity and reproducibility between readers with different experience in female pelvic imaging. Finally, evaluate any correlation between ADC value and histotype in malignant lesions.

MATERIALS AND METHODS

In total, 173 patients with 213 indeterminate adnexal masses (AMs) on ultrasound were subjected to MRI examination, from which 140 patients with 172 AMs were included in the final analysis. Standardised MRI sequences were used, including DWI and DCE sequences. Two readers, blinded to histopathological data, retrospectively classified AMs according to the O-RADS MRI scoring system. A quantitative analysis method was applied by placing a ROI on the ADC maps obtained from single-exponential DWI sequences. AMs considered benign (O-RADS MRI score 2) were excluded from the ADC analysis.

RESULTS

Excellent inter-reader agreement was found in the classification of lesions according to the O-RADS MRI score (K = 0.936; 95% CI). Two ROC curves were created to determine the optimal cut-off value for the ADC variable between O-RADS MRI categories 3-4 and 4-5, respectively, 1.411 × 10^-3 mm²/sec and 0.849 × 10^-3 mm²/sec. Based on these ADC values, 3/45 and 22/62 AMs were upgraded, respectively, to score 4 and 5, while 4/62 AMs were downgraded to score 3. ADC values correlated significantly with the ovarian carcinoma histotype (p value < 0.001).

CONCLUSION

Our study demonstrates the prognostic potential of DWI and ADC values in the O-RADS MRI classification for better radiological standardisation and characterisation of AMs.

Ultrasound examination, MRI, or ROMA for discriminating between inconclusive adnexal masses as determined by IOTA Simple Rules: a prospective study

OBJECTIVE

To determine the best second-step approach for discriminating benign from malignant adnexal masses classified as inconclusive by International Ovarian Tumour Analysis Simple Rules (IOTA-SR).

METHODS

Single-center prospective study comprising a consecutive series of patients diagnosed as having an adnexal mass classified as inconclusive according to IOTA-SR. All women underwent Risk of Ovarian Malignancy Algorithm (ROMA) analysis, MRI interpreted by a radiologist, and ultrasound examination by a gynecological sonologist. Cases were clinically managed according to the result of the ultrasound expert examination by either serial follow-up for at least 1 year or surgery. Reference standard was histology (patient was submitted to surgery if any of the tests was suspicious) or follow-up (masses with no signs of malignancy after 12 months were considered benign). Diagnostic performance of all three approaches was calculated and compared. Direct cost analysis of the test used was also performed.

RESULTS

Eighty-two adnexal masses in 80 women (median age 47.6 years, range 16 to 73 years) were included. Seventeen patients (17 masses) were managed expectantly (none had diagnosis of ovarian cancer after at least 12 months of follow-up) and 63 patients (65 masses) underwent surgery and tumor removal (40 benign and 25 malignant tumors). Sensitivity and specificity for ultrasound, MRI, and ROMA were 96% and 93%, 100% and 81%, and 24% and 93%, respectively. The specificity of ultrasound was better than that for MRI (p=0.021), and the sensitivity of ultrasound was better than that for ROMA (p<0.001), sensitivity was better for MRI than for ROMA (p<0.001) and the specificity of ROMA was better than that for MRI (p<0.001). Ultrasound evaluation was the most effective and least costly method as compared with MRI and ROMA.

CONCLUSION

In this study, ultrasound examination was the best second-step approach in inconclusive adnexal masses as determined by IOTA-SR, but the findings require confirmation in multicenter prospective trials.

Accuracy and reproducibility of the O-RADS MRI risk stratification system based on enhanced non-DCE MRI in the assessment of adnexal masses

OBJECTIVE

Evaluation of the diagnostic performance and reproducibility of the Ovarian-Adnexal Reporting and Data System (O-RADS) Magnetic Resonance Imaging (MRI) risk stratification system based on enhanced non-dynamic contrast-enhanced (non-DCE) MRI in the diagnosis of adnexal masses.

METHODS

Patients who underwent conventional pelvic enhanced non-DCE MRI examination within one month prior to surgery formed the study population. Two experienced radiologists independently evaluated the images and assigned a score according to the O-RADS MRI risk stratification system. One of the radiologists reviewed the images and reassigned the scores after three months. Intra- and inter-observer agreement was evaluated with the k coefficient value. The adnexal masses that attained scores between 1 and 3 were considered benign, while those with scores of 4 or 5 were considered malignant. Analyses were conducted to determine the sensitivity, specificity, positive and negative predictive values, and receiver operating characteristic (ROC) curve, which were then used for evaluating the diagnostic efficacy of the developed system based on enhanced non-DCE MRI scan. The reference standard was histology.

RESULTS

A total of 308 patients (mean age: 42.09 ± 12.42 years, age range: 20-84 years) were enrolled in the study. Among the 362 adnexal masses from the included patients, there were 320 benign masses and 42 malignant masses. In the case of three readers, there were no malignant tumors scored 1-2. The O-RADS MRI score ≥ 4 was associated with malignancy resulted in a good diagnostic efficacy with the areas under the curve (AUC) values of 0.918 (95 % CI, 0.864-0.972), 0.905 (95 % CI, 0.842-0.968), and 0.882 (95 % CI, 0.815-0.950), the sensitivity values of 90.5 % (95 % CI, 87.5-93.5 %), 85.7 % (95 % CI, 82.1-89.3 %), and 83.3 % (95 % CI, 79.5-87.2 %), and the specificity values of 93.1 % (95 % CI, 90.5-95.7 %), 95.3 % (95 % CI, 93.1-97.5 %), and 93.1 % (95 % CI, 90.5-95.7 %) obtained for the three readers, respectively. Excellent intra-observer agreement and inter-observer agreement were observed with the k values of 0.883 (95 % CI, 0.814-0.952) and 0.848 (95 % CI, 0.770-0.926), respectively.

CONCLUSIONS

The O-RADS MRI risk stratification system based on enhanced non-DCE MRI scans exhibited high accuracy and reproducibility in the prediction of adnexal masses malignancy. Enhanced non-DCE MRI scan may offer an alternative diagnostic tool when DCE is not possible.

O-RADS MRI After Initial Ultrasound for Adnexal Lesions: AJR Expert Panel Narrative Review

The Ovarian-Adnexal Reporting and Data System (O-RADS) ultrasound (US) and MRI risk stratification systems were developed by an international group of experts in adnexal imaging to aid radiologists in assessing adnexal lesions. The goal of imaging is to appropriately triage patients with adnexal lesions. US is the first-line imaging modality for assessment, whereas MRI can be used as a problem-solving tool. Both US and MRI can accurately characterize benign lesions such as simple cysts, endometriomas, hemorrhagic cysts, and dermoid cysts, avoiding unnecessary or inappropriate surgery. In patients with a lesion that does not meet criteria for one of these benign diagnoses, MRI can further characterize the lesion with an improved specificity for cancer and the ability to provide a probable histologic subtype in the presence of certain MRI features. This allows personalized treatment, including avoiding overly extensive surgery or allowing fertility-sparing procedures for suspected benign, borderline, or low-grade tumors. When MRI findings indicate a risk of an invasive cancer, patients can be expeditiously referred to a gynecologic oncologic surgeon. This narrative review provides expert opinion on the utility of multiparametric MRI when using the O-RADS US and MRI management systems.

O-RADS MRI: A Systematic Review and Meta-Analysis of Diagnostic Performance and Category-wise Malignancy Rates

Background US-indeterminate adnexal lesions remain an important indication for gynecologic surgery. MRI can serve as a problem-solving tool through the use of the Ovarian-Adnexal Imaging Reporting and Data System (O-RADS) MRI lexicon, which is based on the ADNEX MR scoring system. Purpose To perform a systematic review and meta-analysis of the diagnostic performance of pelvic MRI interpreted using the ADNEX or O-RADS MRI stratification systems to characterize US-indeterminate adnexal lesions and of the category-wise malignancy rates. Materials and Methods A systematic literature search from May 2013 (publication of the ADNEX MR score) to September 2022 was performed. Studies reporting the use of pelvic MRI interpreted with the ADNEX or O-RADS MRI systems to characterize US-indeterminate adnexal lesions, with pathologic examination and/or follow-up as the reference standard, were included. Summary estimates of diagnostic performance were obtained with the bivariate random-effects model, while category-wise summary malignancy rates of O-RADS MRI 2, 3, 4, and 5 lesions were obtained with a random-effects model. Effects of covariates on heterogeneity and diagnostic performance were investigated through meta-regression. Results Thirteen study parts from 12 studies (3731 women, 4520 adnexal lesions) met the inclusion criteria. Diagnostic performance meta-analysis for 4012 lesions found a 92% summary sensitivity (95% CI: 88, 95) and a 91% summary specificity (95% CI: 89, 93). The meta-analysis of malignancy rates for 3641 lesions showed summary malignancy rates of 0.1% (95% CI: 0, 1) among O-RADS MRI 2 lesions, 6% (95% CI: 3, 9) among O-RADS MRI 3 lesions, 60% (95% CI: 52, 67) among O-RADS MRI 4 lesions, and 96% (95% CI: 92, 99) among O-RADS MRI 5 lesions. Conclusion Pelvic MRI interpreted with the Ovarian-Adnexal Reporting and Data System (O-RADS) MRI lexicon had high diagnostic performance for the characterization of US-indeterminate adnexal lesions. Summary estimates of malignancy rates in the O-RADS MRI 4 and O-RADS MRI 5 categories were higher than predicted ones. © RSNA, 2022 Supplemental material is available for this article. See also the editorial by Lee and Kang in this issue.

Discriminating Between Benign and Malignant Solid Ovarian Tumors Based on Clinical and Radiomic Features of MRI

RATIONALE AND OBJECTIVES

To develop and validate a combined model integrating clinical and radiomic features to non-invasive discriminate between the benign and malignant solid ovarian tumors.

MATERIALS AND METHODS

A total of 148 patients with 156 solid ovarian tumors (86 benign and 70 malignant tumors) were included in this study. The dataset was split into the training and the test set with a ratio of 8:2 using stratified random sampling. 12 clinical features and 1612 radiomic features were extracted from each tumor. These features were selected by least absolute shrinkage and selection operator (Lasso). Three classification models were built using extreme gradient boosting (XGB) algorithm: clinical model, radiomic model, combined model. The area under the receiver operating characteristic curve (AUC), accuracy, precision and sensitivity were analyzed to evaluate the performance of these models.

RESULTS

All of the three models obtained good performances in differentiating benign with malignant solid ovarian tumors in both training and test sets. The AUC, accuracy, precision, sensitivity of clinical model and radiomic model in test set were 0.847 (95% confidence interval (CI), 0.707-0.986, p <0.01), 0.774, 0.769, 0.714, and 0.807 (95%CI, 0.652-0.961, p <0.05), 0.677, 0.643, 0.643, respectively. Combined model had the best prediction results, the AUC, accuracy, precision and sensitivity were 0.954 (95%CI, 0.862-1.0, p <0.01), 0.839, 0.909 and 0.714 in test set.

CONCLUSION

Radiomics based on machine learning can be helpful for radiologists in differentiating the benign and malignant solid ovarian tumors.

O-RADS MRI Risk Stratification System: Guide for Assessing Adnexal Lesions from the ACR O-RADS Committee

MRI plays an important role as a secondary test or problem-solving modality in the evaluation of adnexal lesions depicted at US. MRI has increased specificity compared with US, decreasing the number of false-positive diagnoses for malignancy and thereby avoiding unnecessary or over-extensive surgery in patients with benign lesions or borderline tumors, while women with possible malignancies can be expeditiously referred for oncologic surgical evaluation. The Ovarian-Adnexal Reporting and Data System (O-RADS) MRI Committee is an international collaborative effort formed under the direction of the American College of Radiology and includes a diverse group of experts on adnexal imaging and management who developed the O-RADS MRI risk stratification system. This scoring system assigns a probability of malignancy based on the MRI features of an adnexal lesion and provides information to facilitate optimal patient management. The widespread implementation of a codified reporting system will lead to improved interpretation agreement and standardized communication between radiologists and referring physicians. In addition, it will allow for high-quality multi-institutional collaborations-an important unmet need that has hampered the performance of high-quality research in this area in the past. This article provides guidelines on using the O-RADS MRI risk stratification system in clinical practice, as well as in the educational and research settings.

Diagnostic accuracy and validity of the O-RADS MRI score based on a simplified MRI protocol: a single tertiary center retrospective study

BACKGROUND

Adnexal masses (AM) are a common gynecological problem. It is important to use a reliable imaging method in the differentiation of benign and malignant AMs.

PURPOSE

To assess the accuracy and validity of the O-RADS magnetic resonance imaging (MRI) score for characterizing AM using a simplified MRI protocol.

MATERIAL AND METHODS

The study population comprised 332 women who underwent MRI due to the detection of indeterminate AM on ultrasonography between January 2018 and June 2020. An experienced radiologist calculated the O-RADS MRI score into five categories, using an MRI protocol with a simplified dynamic study. Sensitivity, specificity, positive and negative predictive values, and area under the curve (AUC) were calculated (cutoff for malignancy, score ≥ 4). The reference standard was histopathologic diagnosis or imaging findings during >24 months of follow-up.

RESULTS

Of 237 AMs, 28 (11.9%) were malignant. The malignancy rates of AMs with scores of 1, 2, 3, 4, and 5 were 0% (0/12), 0% (0/111), 1.2% (1/77), 50% (10/20), and 100% (17/17), respectively. The O-RADS MRI score showed 96.3% sensitivity, 95.2% specificity, and 95.3% accuracy in malignancy prediction. The AUC for the differentiation of benign and malignant masses were 0.983. False positivity rate was high in cases with an O-RADS MRI score of 4 (50%).

CONCLUSION

The O-RADS MRI score, based on a simplified MRI protocol, has high accuracy and validity in distinguishing benign from malignant sonographically indeterminate AMs. Its use in clinical practice can classify the malignancy risks of masses and prevent unnecessary surgery in benign lesions.

[Adnexal Masses: Clinical Application of Multiparametric MR Imaging & O-RADS MRI]

Incidental adnexal masses considered indeterminate for malignancy are commonly observed on ultrasonography. Multiparametric MRI is the imaging modality of choice for the evaluation of sonographically indeterminate adnexal masses. Conventional MRI enables a confident pathologic diagnosis of various benign lesions due to accurate tissue characterization of fat, blood, fibrous tissue, and solid components. Additionally, functional imaging sequences, including perfusion- and diffusion-weighted imaging, improve the diagnostic efficacy of conventional MRI in differentiating benign from malignant adnexal masses. The ovarian-adnexal reporting and data system (O-RADS) MRI was recently designed to provide consistent interpretations in assigning risk of malignancy to ovarian and other adnexal masses, and to provide a management recommendation for each risk category. In this review, we describe the clinical application of multiparametric MRI for the evaluation of adnexal masses and introduce the O-RADS MRI risk stratification system.