1
|
Du Y, Guo W, Xiao Y, Chen H, Yao J, Wu J. Ultrasound-based deep learning radiomics model for differentiating benign, borderline, and malignant ovarian tumours: a multi-class classification exploratory study. BMC Med Imaging 2024; 24:89. [PMID: 38622546 PMCID: PMC11020982 DOI: 10.1186/s12880-024-01251-2] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/06/2023] [Accepted: 03/18/2024] [Indexed: 04/17/2024] Open
Abstract
BACKGROUND Accurate preoperative identification of ovarian tumour subtypes is imperative for patients as it enables physicians to custom-tailor precise and individualized management strategies. So, we have developed an ultrasound (US)-based multiclass prediction algorithm for differentiating between benign, borderline, and malignant ovarian tumours. METHODS We randomised data from 849 patients with ovarian tumours into training and testing sets in a ratio of 8:2. The regions of interest on the US images were segmented and handcrafted radiomics features were extracted and screened. We applied the one-versus-rest method in multiclass classification. We inputted the best features into machine learning (ML) models and constructed a radiomic signature (Rad_Sig). US images of the maximum trimmed ovarian tumour sections were inputted into a pre-trained convolutional neural network (CNN) model. After internal enhancement and complex algorithms, each sample's predicted probability, known as the deep transfer learning signature (DTL_Sig), was generated. Clinical baseline data were analysed. Statistically significant clinical parameters and US semantic features in the training set were used to construct clinical signatures (Clinic_Sig). The prediction results of Rad_Sig, DTL_Sig, and Clinic_Sig for each sample were fused as new feature sets, to build the combined model, namely, the deep learning radiomic signature (DLR_Sig). We used the receiver operating characteristic (ROC) curve and the area under the ROC curve (AUC) to estimate the performance of the multiclass classification model. RESULTS The training set included 440 benign, 44 borderline, and 196 malignant ovarian tumours. The testing set included 109 benign, 11 borderline, and 49 malignant ovarian tumours. DLR_Sig three-class prediction model had the best overall and class-specific classification performance, with micro- and macro-average AUC of 0.90 and 0.84, respectively, on the testing set. Categories of identification AUC were 0.84, 0.85, and 0.83 for benign, borderline, and malignant ovarian tumours, respectively. In the confusion matrix, the classifier models of Clinic_Sig and Rad_Sig could not recognise borderline ovarian tumours. However, the proportions of borderline and malignant ovarian tumours identified by DLR_Sig were the highest at 54.55% and 63.27%, respectively. CONCLUSIONS The three-class prediction model of US-based DLR_Sig can discriminate between benign, borderline, and malignant ovarian tumours. Therefore, it may guide clinicians in determining the differential management of patients with ovarian tumours.
Collapse
Affiliation(s)
- Yangchun Du
- Department of Ultrasound, The First Affiliated Hospital of Guangxi Medical University, No.6 Shuangyong Road, Qingxiu District, 530021, Nanning, China
- Department of Ultrasound, The People's Hospital of Guangxi Zhuang Autonomous Region & Guangxi Academy of Medical Sciences, No.6 Taoyuan Road, Qingxiu District, 530021, Nanning, China
| | - Wenwen Guo
- Department of Pathology, The People's Hospital of Guangxi Zhuang Autonomous Region & Guangxi Academy of Medical Sciences, No.6 Taoyuan Road, Qingxiu District, 530021, Nanning, China
| | - Yanju Xiao
- Department of Ultrasound, The People's Hospital of Guangxi Zhuang Autonomous Region & Guangxi Academy of Medical Sciences, No.6 Taoyuan Road, Qingxiu District, 530021, Nanning, China
| | - Haining Chen
- Department of Ultrasound, The People's Hospital of Guangxi Zhuang Autonomous Region & Guangxi Academy of Medical Sciences, No.6 Taoyuan Road, Qingxiu District, 530021, Nanning, China
| | - Jinxiu Yao
- Department of Ultrasound, The People's Hospital of Guangxi Zhuang Autonomous Region & Guangxi Academy of Medical Sciences, No.6 Taoyuan Road, Qingxiu District, 530021, Nanning, China
| | - Ji Wu
- Department of Ultrasound, The First Affiliated Hospital of Guangxi Medical University, No.6 Shuangyong Road, Qingxiu District, 530021, Nanning, China.
| |
Collapse
|
2
|
Du Y, Xiao Y, Guo W, Yao J, Lan T, Li S, Wen H, Zhu W, He G, Zheng H, Chen H. Development and validation of an ultrasound-based deep learning radiomics nomogram for predicting the malignant risk of ovarian tumours. Biomed Eng Online 2024; 23:41. [PMID: 38594729 PMCID: PMC11003110 DOI: 10.1186/s12938-024-01234-y] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/05/2024] [Accepted: 04/02/2024] [Indexed: 04/11/2024] Open
Abstract
BACKGROUND The timely identification and management of ovarian cancer are critical determinants of patient prognosis. In this study, we developed and validated a deep learning radiomics nomogram (DLR_Nomogram) based on ultrasound (US) imaging to accurately predict the malignant risk of ovarian tumours and compared the diagnostic performance of the DLR_Nomogram to that of the ovarian-adnexal reporting and data system (O-RADS). METHODS This study encompasses two research tasks. Patients were randomly divided into training and testing sets in an 8:2 ratio for both tasks. In task 1, we assessed the malignancy risk of 849 patients with ovarian tumours. In task 2, we evaluated the malignancy risk of 391 patients with O-RADS 4 and O-RADS 5 ovarian neoplasms. Three models were developed and validated to predict the risk of malignancy in ovarian tumours. The predicted outcomes of the models for each sample were merged to form a new feature set that was utilised as an input for the logistic regression (LR) model for constructing a combined model, visualised as the DLR_Nomogram. Then, the diagnostic performance of these models was evaluated by the receiver operating characteristic curve (ROC). RESULTS The DLR_Nomogram demonstrated superior predictive performance in predicting the malignant risk of ovarian tumours, as evidenced by area under the ROC curve (AUC) values of 0.985 and 0.928 for the training and testing sets of task 1, respectively. The AUC value of its testing set was lower than that of the O-RADS; however, the difference was not statistically significant. The DLR_Nomogram exhibited the highest AUC values of 0.955 and 0.869 in the training and testing sets of task 2, respectively. The DLR_Nomogram showed satisfactory fitting performance for both tasks in Hosmer-Lemeshow testing. Decision curve analysis demonstrated that the DLR_Nomogram yielded greater net clinical benefits for predicting malignant ovarian tumours within a specific range of threshold values. CONCLUSIONS The US-based DLR_Nomogram has shown the capability to accurately predict the malignant risk of ovarian tumours, exhibiting a predictive efficacy comparable to that of O-RADS.
Collapse
Affiliation(s)
- Yangchun Du
- Department of Ultrasound, The People's Hospital of Guangxi Zhuang Autonomous Region and Guangxi Academy of Medical Sciences, No. 6 Taoyuan Road, Qingxiu District, Nanning, 530021, China
| | - Yanju Xiao
- Department of Ultrasound, The People's Hospital of Guangxi Zhuang Autonomous Region and Guangxi Academy of Medical Sciences, No. 6 Taoyuan Road, Qingxiu District, Nanning, 530021, China
| | - Wenwen Guo
- Department of Pathology, The People's Hospital of Guangxi Zhuang Autonomous Region and Guangxi Academy of Medical Sciences, No. 6 Taoyuan Road, Qingxiu District, Nanning, 530021, China
| | - Jinxiu Yao
- Department of Ultrasound, The People's Hospital of Guangxi Zhuang Autonomous Region and Guangxi Academy of Medical Sciences, No. 6 Taoyuan Road, Qingxiu District, Nanning, 530021, China
| | - Tongliu Lan
- Department of Ultrasound, The People's Hospital of Guangxi Zhuang Autonomous Region and Guangxi Academy of Medical Sciences, No. 6 Taoyuan Road, Qingxiu District, Nanning, 530021, China
| | - Sijin Li
- Department of Ultrasound, The People's Hospital of Guangxi Zhuang Autonomous Region and Guangxi Academy of Medical Sciences, No. 6 Taoyuan Road, Qingxiu District, Nanning, 530021, China
| | - Huoyue Wen
- Department of Ultrasound, The People's Hospital of Guangxi Zhuang Autonomous Region and Guangxi Academy of Medical Sciences, No. 6 Taoyuan Road, Qingxiu District, Nanning, 530021, China
| | - Wenying Zhu
- Department of Ultrasound, The People's Hospital of Guangxi Zhuang Autonomous Region and Guangxi Academy of Medical Sciences, No. 6 Taoyuan Road, Qingxiu District, Nanning, 530021, China
| | - Guangling He
- Department of Ultrasound, The People's Hospital of Guangxi Zhuang Autonomous Region and Guangxi Academy of Medical Sciences, No. 6 Taoyuan Road, Qingxiu District, Nanning, 530021, China
| | - Hongyu Zheng
- Department of Ultrasound, The People's Hospital of Guangxi Zhuang Autonomous Region and Guangxi Academy of Medical Sciences, No. 6 Taoyuan Road, Qingxiu District, Nanning, 530021, China.
| | - Haining Chen
- Department of Ultrasound, The People's Hospital of Guangxi Zhuang Autonomous Region and Guangxi Academy of Medical Sciences, No. 6 Taoyuan Road, Qingxiu District, Nanning, 530021, China.
| |
Collapse
|
3
|
Huang ML, Ren J, Jin ZY, Liu XY, He YL, Li Y, Xue HD. A systematic review and meta-analysis of CT and MRI radiomics in ovarian cancer: methodological issues and clinical utility. Insights Imaging 2023; 14:117. [PMID: 37395888 DOI: 10.1186/s13244-023-01464-z] [Citation(s) in RCA: 4] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/12/2023] [Accepted: 06/11/2023] [Indexed: 07/04/2023] Open
Abstract
OBJECTIVES We aimed to present the state of the art of CT- and MRI-based radiomics in the context of ovarian cancer (OC), with a focus on the methodological quality of these studies and the clinical utility of these proposed radiomics models. METHODS Original articles investigating radiomics in OC published in PubMed, Embase, Web of Science, and the Cochrane Library between January 1, 2002, and January 6, 2023, were extracted. The methodological quality was evaluated using the radiomics quality score (RQS) and Quality Assessment of Diagnostic Accuracy Studies 2 (QUADAS-2). Pairwise correlation analyses were performed to compare the methodological quality, baseline information, and performance metrics. Additional meta-analyses of studies exploring differential diagnoses and prognostic prediction in patients with OC were performed separately. RESULTS Fifty-seven studies encompassing 11,693 patients were included. The mean RQS was 30.7% (range - 4 to 22); less than 25% of studies had a high risk of bias and applicability concerns in each domain of QUADAS-2. A high RQS was significantly associated with a low QUADAS-2 risk and recent publication year. Significantly higher performance metrics were observed in studies examining differential diagnosis; 16 such studies as well as 13 exploring prognostic prediction were included in a separate meta-analysis, which revealed diagnostic odds ratios of 25.76 (95% confidence interval (CI) 13.50-49.13) and 12.55 (95% CI 8.38-18.77), respectively. CONCLUSION Current evidence suggests that the methodological quality of OC-related radiomics studies is unsatisfactory. Radiomics analysis based on CT and MRI showed promising results in terms of differential diagnosis and prognostic prediction. CRITICAL RELEVANCE STATEMENT Radiomics analysis has potential clinical utility; however, shortcomings persist in existing studies in terms of reproducibility. We suggest that future radiomics studies should be more standardized to better bridge the gap between concepts and clinical applications.
Collapse
Affiliation(s)
- Meng-Lin Huang
- Department of Radiology, Peking Union Medical College Hospital, Chinese Academy of Medical Sciences & Peking Union Medical College, Beijing, People's Republic of China
| | - Jing Ren
- Department of Radiology, Peking Union Medical College Hospital, Chinese Academy of Medical Sciences & Peking Union Medical College, Beijing, People's Republic of China
| | - Zheng-Yu Jin
- Department of Radiology, Peking Union Medical College Hospital, Chinese Academy of Medical Sciences & Peking Union Medical College, Beijing, People's Republic of China
| | - Xin-Yu Liu
- Department of Radiology, Peking Union Medical College Hospital, Chinese Academy of Medical Sciences & Peking Union Medical College, Beijing, People's Republic of China
| | - Yong-Lan He
- Department of Radiology, Peking Union Medical College Hospital, Chinese Academy of Medical Sciences & Peking Union Medical College, Beijing, People's Republic of China.
| | - Yuan Li
- Department of Obstetrics and Gynecology, Peking Union Medical College Hospital, Chinese Academy of Medical Sciences & Peking Union Medical College, National Clinical Research Center for Obstetric & Gynecologic Diseases, Beijing, People's Republic of China.
| | - Hua-Dan Xue
- Department of Radiology, Peking Union Medical College Hospital, Chinese Academy of Medical Sciences & Peking Union Medical College, Beijing, People's Republic of China.
| |
Collapse
|
4
|
Hsu ST, Su YJ, Hung CH, Chen MJ, Lu CH, Kuo CE. Automatic ovarian tumors recognition system based on ensemble convolutional neural network with ultrasound imaging. BMC Med Inform Decis Mak 2022; 22:298. [PMID: 36397100 PMCID: PMC9673368 DOI: 10.1186/s12911-022-02047-6] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/24/2022] [Accepted: 11/14/2022] [Indexed: 11/18/2022] Open
Abstract
Background Upon the discovery of ovarian cysts, obstetricians, gynecologists, and ultrasound examiners must address the common clinical challenge of distinguishing between benign and malignant ovarian tumors. Numerous types of ovarian tumors exist, many of which exhibit similar characteristics that increase the ambiguity in clinical diagnosis. Using deep learning technology, we aimed to develop a method that rapidly and accurately assists the different diagnosis of ovarian tumors in ultrasound images. Methods Based on deep learning method, we used ten well-known convolutional neural network models (e.g., Alexnet, GoogleNet, and ResNet) for training of transfer learning. To ensure method stability and robustness, we repeated the random sampling of the training and validation data ten times. The mean of the ten test results was set as the final assessment data. After the training process was completed, the three models with the highest ratio of calculation accuracy to time required for classification were used for ensemble learning pertaining. Finally, the interpretation results of the ensemble classifier were used as the final results. We also applied ensemble gradient-weighted class activation mapping (Grad-CAM) technology to visualize the decision-making results of the models. Results The highest mean accuracy, mean sensitivity, and mean specificity of ten single CNN models were 90.51 ± 4.36%, 89.77 ± 4.16%, and 92.00 ± 5.95%, respectively. The mean accuracy, mean sensitivity, and mean specificity of the ensemble classifier method were 92.15 ± 2.84%, 91.37 ± 3.60%, and 92.92 ± 4.00%, respectively. The performance of the ensemble classifier is better than that of a single classifier in three evaluation metrics. Moreover, the standard deviation is also better which means the ensemble classifier is more stable and robust. Conclusion From the comprehensive perspective of data quantity, data diversity, robustness of validation strategy, and overall accuracy, the proposed method outperformed the methods used in previous studies. In future studies, we will continue to increase the number of authenticated images and apply our proposed method in clinical settings to increase its robustness and reliability.
Collapse
|