Li D, Hu R, Li H, Cai Y, Zhang PJ, Wu J, Zhu C, Bai HX. Performance of automatic machine learning versus radiologists in the evaluation of endometrium on computed tomography.
Abdom Radiol (NY) 2021;
46:5316-5324. [PMID:
34286371 DOI:
10.1007/s00261-021-03210-9]
[Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/29/2021] [Revised: 06/21/2021] [Accepted: 06/22/2021] [Indexed: 12/17/2022]
Abstract
PURPOSE
In this study, we developed radiomic models that utilize a combination of imaging features and clinical variables to distinguish endometrial cancer (EC) from normal endometrium on routine computed tomography (CT).
METHODS
A total of 926 patients consisting of 416 endometrial cancer (EC) and 510 normal endometrium were included. The CT images of these patients were segmented manually, and divided into training, validation, testing and external testing sets. Non-texture and texture features of these images with endometrium or uterus as region of interest were extracted. The clinical feature "age" was also included in the feature set. Feature selection and machine learning classifier were applied to normalized feature set. This manual optimized combination was then compared with the best pipeline exported by Tree-Based Pipeline Optimization Tool (TPOT) on testing and external testing set. The performances of these machine learning pipelines were compared to that of radiologists.
RESULTS
The manual expert optimized pipeline using the "reliefF" feature selection method and "Bagging" classifier on the external testing set achieved a test ROC AUC of 0.73, accuracy of 0.73 (95% CI 0.62-0.82), sensitivity of 0.64 (95% CI 0.45-0.79), and specificity of 0.78 (95% CI 0.65-0.87), while TPOT achieved a test ROC AUC of 0.79, accuracy of 0.80 (95% CI 0.70-0.87), sensitivity of 0.61 (95% CI 0.43-0.77), and specificity of 0.90 (95% CI 0.78-0.96). When compared to average radiologist performance, the TPOT achieved higher test accuracy (0.80 vs. 0.49, p < 0.001) and specificity (0.90 vs. 0.51, p < 0.001), with comparable sensitivity (0.61 vs. 0.46, p = 0.130).
CONCLUSION
Our results demonstrate that automatic machine learning can distinguish EC from normal endometrium on routine CT imaging with higher accuracy and specificity than radiologists.
Collapse