The diagnosis of ovarian cancer: is color Doppler imaging reproducible and accurate in examiners with different degrees of experience?

Ultrasonographic diagnosis of adnexal masses: interobserver agreement in the interpretation of videos, using IOTA terminology

OBJECTIVES

Aim of this study is to estimate interobserver agreement in classifying adnexal tumors using IOTA terms, simple rules and subjective assessment. In addition, we related observers' accuracy with their experience in gynecological ultrasonography and the year of IOTA certification.

METHODS

Eleven observers with three different levels of experience evaluated videoclips of 70 adnexal masses, defining tumor type according to IOTA terms and definitions, classifying the mass using IOTA Simple rules and Subjective assessment as well as providing Color Score evaluation. Sensitivity, specificity and area under the ROC curve were calculated and the year of IOTA certification was related with operators' accuracy through Pearson correlation coefficient. Interobserver agreement was estimated calculating percentage of agreement, Fleiss kappa and Cohen's kappa.

RESULTS

We found a positive correlation between the year of IOTA certification and operators' accuracy (Pearson coefficient 0.694), especially among the observers with the least experience, the residents (p = 0.003). For tumor type classification, identification of papillary projections and classification of tumors using subjective assessment, agreement among all observers was moderate (Fleiss kappa 0.455, 0.552, and 0.476, respectively) and increased with the years of experience. Agreement in the application of Simple Rules was moderate in all examiners with IOTA certification, with Fleiss kappa in the range of (0.403, 0.498). For Color Score assignment interobserver agreement among all observers was fair (Cohen's kappa 0.380).

CONCLUSIONS

Even among expert examiners, the results of adnexal lesion assessment can be inconsistent. Experience impacts on accuracy and agreement in subjective assessment, while the application of Simple Rules can mitigate the role of experience in interobserver agreement. The knowledge of IOTA models among residents seams to improve their diagnostic accuracy, showing the benefits of IOTA terminology for in training sonographers.

Diagnostic Accuracy of the ADNEX Model for Ovarian Cancer at the 15% Cut-Off Value: A Systematic Review and Meta-Analysis

Objectives

To evaluate the diagnostic accuracy of the ADNEX model for ovarian cancer at the 15% cut-off value.

Methods

Studies on the identified diagnosis of the ADNEX model for ovarian cancer published in PubMed, Embase, the Cochrane Library and Web of Science databases from January 1st, 2014 to February 20th, 2021 were searched. Two researchers independently screened the retrieved studies and extracted the basic features and parameter data. The quality of the eligible studies was evaluated by Quality Assessment of Diagnostic Accuracy Studies-2, and the result was summarized by Review Manager 5.3. Meta-Disc 1.4 and STATA 16.0 were used in statistical analysis. Heterogeneity of this meta-analysis was calculated. Meta-regression was performed to investigate the potential sources of heterogeneity. Sensitivity analysis and Deek's funnel plot analysis were conducted to evaluate the stability and publication bias, respectively.

Results

280 studies were initially retrieved through the search strategy, and 10 eligible studies were ultimately included. The random-effects model was selected for data synthesis. The pooled sensitivity, specificity, positive likelihood ratio, negative likelihood ratio, diagnostic odds ratio and the area under the summary receiver operating characteristic curve were 0.92 (95% CI: 0.89-0.94), 0.82 (95% CI: 0.78-0.86), 5.2 (95% CI: 4.1-6.4), 0.10 (95% CI: 0.07-0.13), 54.0 (95% CI: 37.0-77.0) and 0.95 (95% CI: 0.91-0.95). Meta-regression based on study design, country, enrollment and blind method was not statistically significant. This meta-analysis was stable with no obvious publication bias.

Conclusions

The ADNEX model at the 15% cut-off had high diagnostic accuracy in identifying ovarian cancer.

IOTA Simple Rules in Differentiating between Benign and Malignant Adnexal Masses by Non-expert Examiners

OBJECTIVE

To evaluate the diagnostic performance of IOTA simple rules in predicting malignant adnexal tumors by non-expert examiners.

MATERIALS AND METHODS

Five obstetric/gynecologic residents, who had never performed gynecologic ultrasound examination by themselves before, were trained for IOTA simple rules by an experienced examiner. One trained resident performed ultrasound examinations including IOTA simple rules on 100 women, who were scheduled for surgery due to ovarian masses, within 24 hours of surgery. The gold standard diagnosis was based on pathological or operative findings. The five-trained residents performed IOTA simple rules on 30 patients for evaluation of inter-observer variability.

RESULTS

A total of 100 patients underwent ultrasound examination for the IOTA simple rules. Of them, IOTA simple rules could be applied in 94 (94%) masses including 71 (71.0%) benign masses and 29 (29.0%) malignant masses. The diagnostic performance of IOTA simple rules showed sensitivity of 89.3% (95%CI, 77.8%; 100.7%), specificity 83.3% (95%CI, 74.3%; 92.3%). Inter-observer variability was analyzed using Cohen's kappa coefficient. Kappa indices of the four pairs of raters are 0.713-0.884 (0.722, 0.827, 0.713, and 0.884).

CONCLUSIONS

IOTA simple rules have high diagnostic performance in discriminating adnexal masses even when are applied by non-expert sonographers, though a training course may be required. Nevertheless, they should be further tested by a greater number of general practitioners before widely use.

GyneScan: an improved online paradigm for screening of ovarian cancer via tissue characterization

Ovarian cancer is the fifth highest cause of cancer in women and the leading cause of death from gynecological cancers. Accurate diagnosis of ovarian cancer from acquired images is dependent on the expertise and experience of ultrasonographers or physicians, and is therefore, associated with inter observer variabilities. Computer Aided Diagnostic (CAD) techniques use a number of different data mining techniques to automatically predict the presence or absence of cancer, and therefore, are more reliable and accurate. A review of published literature in the field of CAD based ovarian cancer detection indicates that many studies use ultrasound images as the base for analysis. The key objective of this work is to propose an effective adjunct CAD technique called GyneScan for ovarian tumor detection in ultrasound images. In our proposed data mining framework, we extract several texture features based on first order statistics, Gray Level Co-occurrence Matrix and run length matrix. The significant features selected using t-test are then used to train and test several supervised learning based classifiers such as Probabilistic Neural Networks (PNN), Support Vector Machine (SVM), Decision Tree (DT), k-Nearest Neighbor (KNN), and Naive Bayes (NB). We evaluated the developed framework using 1300 benign and 1300 malignant images. Using 11 significant features in KNN/PNN classifiers, we were able to achieve 100% classification accuracy, sensitivity, specificity, and positive predictive value in detecting ovarian tumor. Even though more validation using larger databases would better establish the robustness of our technique, the preliminary results are promising. This technique could be used as a reliable adjunct method to existing imaging modalities to provide a more confident second opinion on the presence/absence of ovarian tumor.

IOTA simple rules for discriminating between benign and malignant adnexal masses: prospective external validation

OBJECTIVE

To determine the diagnostic performance of International Ovarian Tumor Analysis (IOTA) 'simple' rules for discriminating between benign and malignant adnexal masses.

METHODS

A prospective study was performed between January 2011 and June 2012. Eligible patients were women diagnosed with a persistent adnexal mass who presented to the participating centers. Four trainees evaluated the adnexal mass by transvaginal ultrasound under the supervision of an expert examiner. The trainee analyzed the mass according to IOTA simple rules and provided a diagnosis of benign, malignant or inconclusive. All women included in the study underwent surgery and tumor removal in the center of recruitment. Diagnostic performance was assessed by calculating sensitivity, specificity and positive (LR+) and negative (LR-) likelihood ratios.

RESULTS

A total of 340 women were included (mean patient age, 42.1 (range, 13-79) years). Of the tumors, 55 (16.2%) were malignant and 285 (83.8%) were benign. The IOTA simple rules could be applied in 270 (79.4%) cases. In these cases, sensitivity was 87.9% (95% CI, 72.4-95.2), specificity 97.5% (95% CI, 94.6-98.8), LR+ 34.7 (95% CI, 15.6-77.3) and LR- 0.12 (95% CI, 0.05-0.31).

CONCLUSIONS

Application of the IOTA simple rules yielded acceptable results in terms of specificity in the hands of non-expert examiners. However, with non-expert examiners there was a 12% false-negative rate, which is relatively high.

Intensive training program for ultrasound diagnosis of adnexal masses: protocol and preliminary results

OBJECTIVE

To assess the feasibility of a specific training program for ultrasound diagnosis of adnexal masses.

METHODS

A 2-month intensive training program was developed. The program protocol consisted of a 1-day intensive theoretical course focused on clinical and sonographic issues related to adnexal masses and ovarian cancer, followed by a 4-week real-time ultrasound training program in a tertiary center (25-30 adnexal masses evaluated per month) and a final 4-week period for offline assessment of three-dimensional (3D) volumes from adnexal masses. In this final period, each trainee evaluated five sets of 100 3D volumes. 3D volumes contained gray-scale and power Doppler information, and the trainee was provided with clinical data for each case (patient age, menopausal status and reported symptoms). 3D volumes were obtained from surgically removed masses that had undergone histological diagnosis or from masses that had been followed up until resolution. After assessment of each set, the trainee's diagnostic performance was calculated (sensitivity and specificity) and each incorrectly classified mass was evaluated with the trainer. The objective was to achieve a sensitivity of > 95% and a specificity of > 90%. Learning curve cumulative summation (LC-CUSUM) graphs were plotted to assess the learning curve for the trainees.

RESULTS

One trainer and two trainees with little experience in gynecological ultrasound (one gynecologist and one radiologist) participated in this study. LC-CUSUM graphs showed that competence was achieved after 170 or 185 examinations. The objectives for diagnostic performance were achieved after assessment of the second set of 3D volumes (200 cases) for each trainee.

CONCLUSIONS

The proposed training program appears to be feasible. High diagnostic performance can be achieved after analysis of 200 cases and maintained thereafter.

Doppler imaging in the diagnosis of ovarian disease

INTRODUCTION

Pelvic ultrasonography remains the non-invasive imaging modality of choice for detecting and characterising adnexal masses. The use of Doppler ultrasound has improved the differential diagnosis of ovarian disease and helped to widen the understanding of the aetiology and possible pathophysiology of the various underlying conditions with an aim of improved diagnosis and management.

AREAS COVERED

This review describes the basis of different Doppler techniques and their application in ovarian disease. The present evidence is critically analysed in an attempt to define the current role of Doppler ultrasound of the ovaries. The technique holds promise in certain ovarian diseases but remains controversial in others.

EXPERT OPINION

Doppler ultrasound has a role in both the clinical and research settings. Clinically, it can be used to differentiate benign and malignant disease and help confirm diagnoses suggested on grey-scale imaging giving the user more confidence. Its use here, however, is generally restricted to a subjective impression of colour flow rather than a quantitative analysis of vascularity or blood flow velocity. These measures are more relevant in the research setting where quantitative Doppler is being used to clarify the possible underlying aetiology of various ovarian diseases such as polycystic ovarian syndrome and benign cysts. Doppler ultrasound has also been used in various models to evaluate the response to ovarian stimulation and the effect of surgery on the ovary.