1
|
Massobrio R, Mariani LL, Conti D, De Grandis T, Buonomo F, Badellino E, Novara L, Bounous VE, Perotto S, Mancarella M, Ferrero A, Biglia N, Fuso L. Ultrasonographic diagnosis of adnexal masses: interobserver agreement in the interpretation of videos, using IOTA terminology. Arch Gynecol Obstet 2024; 309:211-218. [PMID: 37789207 PMCID: PMC10769985 DOI: 10.1007/s00404-023-07233-z] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/14/2023] [Accepted: 09/13/2023] [Indexed: 10/05/2023]
Abstract
OBJECTIVES Aim of this study is to estimate interobserver agreement in classifying adnexal tumors using IOTA terms, simple rules and subjective assessment. In addition, we related observers' accuracy with their experience in gynecological ultrasonography and the year of IOTA certification. METHODS Eleven observers with three different levels of experience evaluated videoclips of 70 adnexal masses, defining tumor type according to IOTA terms and definitions, classifying the mass using IOTA Simple rules and Subjective assessment as well as providing Color Score evaluation. Sensitivity, specificity and area under the ROC curve were calculated and the year of IOTA certification was related with operators' accuracy through Pearson correlation coefficient. Interobserver agreement was estimated calculating percentage of agreement, Fleiss kappa and Cohen's kappa. RESULTS We found a positive correlation between the year of IOTA certification and operators' accuracy (Pearson coefficient 0.694), especially among the observers with the least experience, the residents (p = 0.003). For tumor type classification, identification of papillary projections and classification of tumors using subjective assessment, agreement among all observers was moderate (Fleiss kappa 0.455, 0.552, and 0.476, respectively) and increased with the years of experience. Agreement in the application of Simple Rules was moderate in all examiners with IOTA certification, with Fleiss kappa in the range of (0.403, 0.498). For Color Score assignment interobserver agreement among all observers was fair (Cohen's kappa 0.380). CONCLUSIONS Even among expert examiners, the results of adnexal lesion assessment can be inconsistent. Experience impacts on accuracy and agreement in subjective assessment, while the application of Simple Rules can mitigate the role of experience in interobserver agreement. The knowledge of IOTA models among residents seams to improve their diagnostic accuracy, showing the benefits of IOTA terminology for in training sonographers.
Collapse
Affiliation(s)
- Roberta Massobrio
- Academic Division of Gynecology and Obstetrics, Mauriziano Hospital, University of Torino, 10128, Turin, Italy.
| | - Luca Liban Mariani
- Academic Division of Gynecology and Obstetrics, Mauriziano Hospital, University of Torino, 10128, Turin, Italy
| | - Daniele Conti
- SynDiag srl, c/o Innovative Enterprises Incubator of Polytechnic University of Turin, 10129, Turin, Italy
| | | | - Francesca Buonomo
- Institute for Maternal and Child Health - IRCCS ''Burlo Garofolo'', 34137, Trieste, Italy
| | - Enrico Badellino
- Academic Division of Gynecology and Obstetrics, Mauriziano Hospital, University of Torino, 10128, Turin, Italy
| | - Lorenzo Novara
- Academic Division of Gynecology and Obstetrics, Mauriziano Hospital, University of Torino, 10128, Turin, Italy
| | | | - Stefania Perotto
- Division of Gynecologic Oncology, Michele e Pietro Ferrero Hospital, 12060, Verduno, Italy
| | - Matteo Mancarella
- Academic Division of Gynecology and Obstetrics, Mauriziano Hospital, University of Torino, 10128, Turin, Italy
| | - Annamaria Ferrero
- Academic Division of Gynecology and Obstetrics, Mauriziano Hospital, University of Torino, 10128, Turin, Italy
| | - Nicoletta Biglia
- Academic Division of Gynecology and Obstetrics, Mauriziano Hospital, University of Torino, 10128, Turin, Italy
| | - Luca Fuso
- Academic Division of Gynecology and Obstetrics, Mauriziano Hospital, University of Torino, 10128, Turin, Italy
| |
Collapse
|
2
|
Huang X, Wang Z, Zhang M, Luo H. Diagnostic Accuracy of the ADNEX Model for Ovarian Cancer at the 15% Cut-Off Value: A Systematic Review and Meta-Analysis. Front Oncol 2021; 11:684257. [PMID: 34222006 PMCID: PMC8247918 DOI: 10.3389/fonc.2021.684257] [Citation(s) in RCA: 6] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/23/2021] [Accepted: 05/24/2021] [Indexed: 12/24/2022] Open
Abstract
Objectives To evaluate the diagnostic accuracy of the ADNEX model for ovarian cancer at the 15% cut-off value. Methods Studies on the identified diagnosis of the ADNEX model for ovarian cancer published in PubMed, Embase, the Cochrane Library and Web of Science databases from January 1st, 2014 to February 20th, 2021 were searched. Two researchers independently screened the retrieved studies and extracted the basic features and parameter data. The quality of the eligible studies was evaluated by Quality Assessment of Diagnostic Accuracy Studies-2, and the result was summarized by Review Manager 5.3. Meta-Disc 1.4 and STATA 16.0 were used in statistical analysis. Heterogeneity of this meta-analysis was calculated. Meta-regression was performed to investigate the potential sources of heterogeneity. Sensitivity analysis and Deek's funnel plot analysis were conducted to evaluate the stability and publication bias, respectively. Results 280 studies were initially retrieved through the search strategy, and 10 eligible studies were ultimately included. The random-effects model was selected for data synthesis. The pooled sensitivity, specificity, positive likelihood ratio, negative likelihood ratio, diagnostic odds ratio and the area under the summary receiver operating characteristic curve were 0.92 (95% CI: 0.89-0.94), 0.82 (95% CI: 0.78-0.86), 5.2 (95% CI: 4.1-6.4), 0.10 (95% CI: 0.07-0.13), 54.0 (95% CI: 37.0-77.0) and 0.95 (95% CI: 0.91-0.95). Meta-regression based on study design, country, enrollment and blind method was not statistically significant. This meta-analysis was stable with no obvious publication bias. Conclusions The ADNEX model at the 15% cut-off had high diagnostic accuracy in identifying ovarian cancer.
Collapse
Affiliation(s)
- Xiaotong Huang
- Department of Ultrasound, West China Second University Hospital, Sichuan University, Chengdu, China.,Key Laboratory of Birth Defects and Related Diseases of Women and Children, Sichuan University, Chengdu, China
| | - Ziwei Wang
- Department of Ultrasound, West China Second University Hospital, Sichuan University, Chengdu, China.,Key Laboratory of Birth Defects and Related Diseases of Women and Children, Sichuan University, Chengdu, China
| | - Meiqin Zhang
- Department of Ultrasound, West China Second University Hospital, Sichuan University, Chengdu, China.,Key Laboratory of Birth Defects and Related Diseases of Women and Children, Sichuan University, Chengdu, China
| | - Hong Luo
- Department of Ultrasound, West China Second University Hospital, Sichuan University, Chengdu, China.,Key Laboratory of Birth Defects and Related Diseases of Women and Children, Sichuan University, Chengdu, China
| |
Collapse
|
3
|
Tinnangwattana D, Vichak-Ururote L, Tontivuthikul P, Charoenratana C, Lerthiranwong T, Tongsong T. IOTA Simple Rules in Differentiating between Benign and Malignant Adnexal Masses by Non-expert Examiners. Asian Pac J Cancer Prev 2016; 16:3835-8. [PMID: 25987045 DOI: 10.7314/apjcp.2015.16.9.3835] [Citation(s) in RCA: 20] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/10/2022] Open
Abstract
OBJECTIVE To evaluate the diagnostic performance of IOTA simple rules in predicting malignant adnexal tumors by non-expert examiners. MATERIALS AND METHODS Five obstetric/gynecologic residents, who had never performed gynecologic ultrasound examination by themselves before, were trained for IOTA simple rules by an experienced examiner. One trained resident performed ultrasound examinations including IOTA simple rules on 100 women, who were scheduled for surgery due to ovarian masses, within 24 hours of surgery. The gold standard diagnosis was based on pathological or operative findings. The five-trained residents performed IOTA simple rules on 30 patients for evaluation of inter-observer variability. RESULTS A total of 100 patients underwent ultrasound examination for the IOTA simple rules. Of them, IOTA simple rules could be applied in 94 (94%) masses including 71 (71.0%) benign masses and 29 (29.0%) malignant masses. The diagnostic performance of IOTA simple rules showed sensitivity of 89.3% (95%CI, 77.8%; 100.7%), specificity 83.3% (95%CI, 74.3%; 92.3%). Inter-observer variability was analyzed using Cohen's kappa coefficient. Kappa indices of the four pairs of raters are 0.713-0.884 (0.722, 0.827, 0.713, and 0.884). CONCLUSIONS IOTA simple rules have high diagnostic performance in discriminating adnexal masses even when are applied by non-expert sonographers, though a training course may be required. Nevertheless, they should be further tested by a greater number of general practitioners before widely use.
Collapse
|
4
|
Ruiz de Gauna B, Rodriguez D, Olartecoechea B, Aubá M, Jurado M, Gómez Roig MD, Alcázar JL. Diagnostic performance of IOTA simple rules for adnexal masses classification: a comparison between two centers with different ovarian cancer prevalence. Eur J Obstet Gynecol Reprod Biol 2015; 191:10-4. [DOI: 10.1016/j.ejogrb.2015.05.024] [Citation(s) in RCA: 15] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/13/2015] [Revised: 04/27/2015] [Accepted: 05/19/2015] [Indexed: 10/23/2022]
|
5
|
The Diagnostic Value of Transvaginal Sonograph (TVS), Color Doppler, and Serum Tumor Marker CA125, CEA, and AFP in Ovarian Cancer. Cell Biochem Biophys 2015; 72:353-7. [DOI: 10.1007/s12013-014-0463-x] [Citation(s) in RCA: 11] [Impact Index Per Article: 1.2] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/21/2022]
|
6
|
Acharya UR, Sree SV, Kulshreshtha S, Molinari F, En Wei Koh J, Saba L, Suri JS. GyneScan: an improved online paradigm for screening of ovarian cancer via tissue characterization. Technol Cancer Res Treat 2014; 13:529-39. [PMID: 24325128 PMCID: PMC4527478 DOI: 10.7785/tcrtexpress.2013.600273] [Citation(s) in RCA: 48] [Impact Index Per Article: 4.8] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/08/2013] [Revised: 11/15/2013] [Accepted: 11/20/2013] [Indexed: 11/23/2022] Open
Abstract
Ovarian cancer is the fifth highest cause of cancer in women and the leading cause of death from gynecological cancers. Accurate diagnosis of ovarian cancer from acquired images is dependent on the expertise and experience of ultrasonographers or physicians, and is therefore, associated with inter observer variabilities. Computer Aided Diagnostic (CAD) techniques use a number of different data mining techniques to automatically predict the presence or absence of cancer, and therefore, are more reliable and accurate. A review of published literature in the field of CAD based ovarian cancer detection indicates that many studies use ultrasound images as the base for analysis. The key objective of this work is to propose an effective adjunct CAD technique called GyneScan for ovarian tumor detection in ultrasound images. In our proposed data mining framework, we extract several texture features based on first order statistics, Gray Level Co-occurrence Matrix and run length matrix. The significant features selected using t-test are then used to train and test several supervised learning based classifiers such as Probabilistic Neural Networks (PNN), Support Vector Machine (SVM), Decision Tree (DT), k-Nearest Neighbor (KNN), and Naive Bayes (NB). We evaluated the developed framework using 1300 benign and 1300 malignant images. Using 11 significant features in KNN/PNN classifiers, we were able to achieve 100% classification accuracy, sensitivity, specificity, and positive predictive value in detecting ovarian tumor. Even though more validation using larger databases would better establish the robustness of our technique, the preliminary results are promising. This technique could be used as a reliable adjunct method to existing imaging modalities to provide a more confident second opinion on the presence/absence of ovarian tumor.
Collapse
Affiliation(s)
- U Rajendra Acharya
- Department of Electronics and Computer Engineering, Ngee Ann Polytechnic, Singapore.
| | | | | | | | | | | | | |
Collapse
|
7
|
Tantipalakorn C, Wanapirak C, Khunamornpong S, Sukpan K, Tongsong T. IOTA Simple Rules in Differentiating between Benign and Malignant Ovarian Tumors. Asian Pac J Cancer Prev 2014; 15:5123-6. [DOI: 10.7314/apjcp.2014.15.13.5123] [Citation(s) in RCA: 27] [Impact Index Per Article: 2.7] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/10/2022] Open
|
8
|
Alcázar JL, Aubá M, Ruiz-Zambrana Á, Olartecoechea B, Diaz D, Hidalgo JJ, Pineda L, Utrilla-Layna J. Ultrasound assessment in adnexal masses: an update. ACTA ACUST UNITED AC 2014. [DOI: 10.1586/eog.12.49] [Citation(s) in RCA: 6] [Impact Index Per Article: 0.6] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/29/2022]
|
9
|
Guerriero S, Saba L, Ajossa S, Peddes C, Sedda F, Piras A, Olartecoechea B, Aubá M, Alcázar JL. Assessing the reproducibility of the IOTA simple ultrasound rules for classifying adnexal masses as benign or malignant using stored 3D volumes. Eur J Obstet Gynecol Reprod Biol 2013; 171:157-60. [DOI: 10.1016/j.ejogrb.2013.08.031] [Citation(s) in RCA: 9] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/28/2013] [Revised: 06/17/2013] [Accepted: 08/14/2013] [Indexed: 11/30/2022]
|
10
|
Alcázar JL, Pascual MÁ, Olartecoechea B, Graupera B, Aubá M, Ajossa S, Hereter L, Julve R, Gastón B, Peddes C, Sedda F, Piras A, Saba L, Guerriero S. IOTA simple rules for discriminating between benign and malignant adnexal masses: prospective external validation. ULTRASOUND IN OBSTETRICS & GYNECOLOGY : THE OFFICIAL JOURNAL OF THE INTERNATIONAL SOCIETY OF ULTRASOUND IN OBSTETRICS AND GYNECOLOGY 2013; 42:467-471. [PMID: 23576304 DOI: 10.1002/uog.12485] [Citation(s) in RCA: 23] [Impact Index Per Article: 2.1] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 01/03/2013] [Revised: 02/13/2013] [Accepted: 03/24/2013] [Indexed: 06/02/2023]
Abstract
OBJECTIVE To determine the diagnostic performance of International Ovarian Tumor Analysis (IOTA) 'simple' rules for discriminating between benign and malignant adnexal masses. METHODS A prospective study was performed between January 2011 and June 2012. Eligible patients were women diagnosed with a persistent adnexal mass who presented to the participating centers. Four trainees evaluated the adnexal mass by transvaginal ultrasound under the supervision of an expert examiner. The trainee analyzed the mass according to IOTA simple rules and provided a diagnosis of benign, malignant or inconclusive. All women included in the study underwent surgery and tumor removal in the center of recruitment. Diagnostic performance was assessed by calculating sensitivity, specificity and positive (LR+) and negative (LR-) likelihood ratios. RESULTS A total of 340 women were included (mean patient age, 42.1 (range, 13-79) years). Of the tumors, 55 (16.2%) were malignant and 285 (83.8%) were benign. The IOTA simple rules could be applied in 270 (79.4%) cases. In these cases, sensitivity was 87.9% (95% CI, 72.4-95.2), specificity 97.5% (95% CI, 94.6-98.8), LR+ 34.7 (95% CI, 15.6-77.3) and LR- 0.12 (95% CI, 0.05-0.31). CONCLUSIONS Application of the IOTA simple rules yielded acceptable results in terms of specificity in the hands of non-expert examiners. However, with non-expert examiners there was a 12% false-negative rate, which is relatively high.
Collapse
Affiliation(s)
- J L Alcázar
- Department of Obstetrics and Gynecology, Clínica Universidad de Navarra, University of Navarra, Pamplona, Spain
| | | | | | | | | | | | | | | | | | | | | | | | | | | |
Collapse
|
11
|
Alcázar JL, Díaz L, Flórez P, Guerriero S, Jurado M. Intensive training program for ultrasound diagnosis of adnexal masses: protocol and preliminary results. ULTRASOUND IN OBSTETRICS & GYNECOLOGY : THE OFFICIAL JOURNAL OF THE INTERNATIONAL SOCIETY OF ULTRASOUND IN OBSTETRICS AND GYNECOLOGY 2013; 42:218-223. [PMID: 23420783 DOI: 10.1002/uog.12440] [Citation(s) in RCA: 10] [Impact Index Per Article: 0.9] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 10/16/2012] [Revised: 12/28/2012] [Accepted: 01/04/2013] [Indexed: 06/01/2023]
Abstract
OBJECTIVE To assess the feasibility of a specific training program for ultrasound diagnosis of adnexal masses. METHODS A 2-month intensive training program was developed. The program protocol consisted of a 1-day intensive theoretical course focused on clinical and sonographic issues related to adnexal masses and ovarian cancer, followed by a 4-week real-time ultrasound training program in a tertiary center (25-30 adnexal masses evaluated per month) and a final 4-week period for offline assessment of three-dimensional (3D) volumes from adnexal masses. In this final period, each trainee evaluated five sets of 100 3D volumes. 3D volumes contained gray-scale and power Doppler information, and the trainee was provided with clinical data for each case (patient age, menopausal status and reported symptoms). 3D volumes were obtained from surgically removed masses that had undergone histological diagnosis or from masses that had been followed up until resolution. After assessment of each set, the trainee's diagnostic performance was calculated (sensitivity and specificity) and each incorrectly classified mass was evaluated with the trainer. The objective was to achieve a sensitivity of > 95% and a specificity of > 90%. Learning curve cumulative summation (LC-CUSUM) graphs were plotted to assess the learning curve for the trainees. RESULTS One trainer and two trainees with little experience in gynecological ultrasound (one gynecologist and one radiologist) participated in this study. LC-CUSUM graphs showed that competence was achieved after 170 or 185 examinations. The objectives for diagnostic performance were achieved after assessment of the second set of 3D volumes (200 cases) for each trainee. CONCLUSIONS The proposed training program appears to be feasible. High diagnostic performance can be achieved after analysis of 200 cases and maintained thereafter.
Collapse
Affiliation(s)
- J L Alcázar
- Department of Obstetrics and Gynecology, Clínica Universidad de Navarra, School of Medicine, University of Navarra, Pamplona, Spain.
| | | | | | | | | |
Collapse
|
12
|
Haoula Z, Deshpande R, Jayaprakasan K, Raine-Fenning N. Doppler imaging in the diagnosis of ovarian disease. EXPERT OPINION ON MEDICAL DIAGNOSTICS 2012; 6:59-73. [PMID: 23480621 DOI: 10.1517/17530059.2012.642366] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/01/2023]
Abstract
INTRODUCTION Pelvic ultrasonography remains the non-invasive imaging modality of choice for detecting and characterising adnexal masses. The use of Doppler ultrasound has improved the differential diagnosis of ovarian disease and helped to widen the understanding of the aetiology and possible pathophysiology of the various underlying conditions with an aim of improved diagnosis and management. AREAS COVERED This review describes the basis of different Doppler techniques and their application in ovarian disease. The present evidence is critically analysed in an attempt to define the current role of Doppler ultrasound of the ovaries. The technique holds promise in certain ovarian diseases but remains controversial in others. EXPERT OPINION Doppler ultrasound has a role in both the clinical and research settings. Clinically, it can be used to differentiate benign and malignant disease and help confirm diagnoses suggested on grey-scale imaging giving the user more confidence. Its use here, however, is generally restricted to a subjective impression of colour flow rather than a quantitative analysis of vascularity or blood flow velocity. These measures are more relevant in the research setting where quantitative Doppler is being used to clarify the possible underlying aetiology of various ovarian diseases such as polycystic ovarian syndrome and benign cysts. Doppler ultrasound has also been used in various models to evaluate the response to ovarian stimulation and the effect of surgery on the ovary.
Collapse
Affiliation(s)
- Zeina Haoula
- University of Nottingham, School of Clinical Sciences, Division of Obstetrics & Gynaecology , Nottingham , UK +44 0 115 823 1000 ; +44 0 115 823 0704 ;
| | | | | | | |
Collapse
|