Application of deep learning to the classification of images from colposcopy

Artificial intelligence for cervical cancer screening: Scoping review, 2009-2022

BACKGROUND

The intersection of artificial intelligence (AI) with cancer research is increasing, and many of the advances have focused on the analysis of cancer images.

OBJECTIVES

To describe and synthesize the literature on the diagnostic accuracy of AI in early imaging diagnosis of cervical cancer following Preferred Reporting Items for Systematic Reviews and Meta-Analyses Extension for Scoping Reviews (PRISMA-ScR).

SEARCH STRATEGY

Arksey and O'Malley methodology was used and PubMed, Scopus, and Google Scholar databases were searched using a combination of English and Spanish keywords.

SELECTION CRITERIA

Identified titles and abstracts were screened to select original reports and cross-checked for overlap of cases.

DATA COLLECTION AND ANALYSIS

A descriptive summary was organized by the AI algorithm used, total of images analyzed, data source, clinical comparison criteria, and diagnosis performance.

MAIN RESULTS

We identified 32 studies published between 2009 and 2022. The primary sources of images were digital colposcopy, cervicography, and mobile devices. The machine learning/deep learning (DL) algorithms applied in the articles included support vector machine (SVM), random forest classifier, k-nearest neighbors, multilayer perceptron, C4.5, Naïve Bayes, AdaBoost, XGboots, conditional random fields, Bayes classifier, convolutional neural network (CNN; and variations), ResNet (several versions), YOLO+EfficientNetB0, and visual geometry group (VGG; several versions). SVM and DL methods (CNN, ResNet, VGG) showed the best diagnostic performances, with an accuracy of over 97%.

CONCLUSION

We concluded that the use of AI for cervical cancer screening has increased over the years, and some results (mainly from DL) are very promising. However, further research is necessary to validate these findings.

Diagnostic Efficacy of Enhanced Visual Assessment [Visual Check] for Triaging Cervical Cancer Screen Positive Women

Introduction

Colposcopy is important for triaging any abnormal cervical screening test. Scarcity of trained colposcopists and colposcopy centers is a big hurdle to screening programs in low- and middle-income countries.

Objectives of the Study

The objective was to assess the performance of the artificial intelligence incorporated into the mobile optical device technologies (ODT) Enhanced Visual Assessment (EVA visual check) against physician colposcopic diagnosis and the gold standard of histopathology.

Materials and Methods

It was a cross-sectional observational study conducted on women referred to a colposcopy clinic following an abnormal screening test. Colposcopic examination was performed by colposcopists using the MobileODT EVA system. Physician's impression and Visual Check analysis were compared with the final histopathological analysis or cytology. Cases with normal cytology and normal colposcopy did not undergo biopsy, and these were considered normal.

Results

A total of 2050 women were screened, and 147 screen-positive women were recruited in the study. EVA Visual Check had a sensitivity of 86.8% (75-95), specificity of 28.7% (20-39), positive predictive value (PPV) of 40.7% (32-50), negative predictive value (NPV) of 79.4% (62-91), and diagnostic accuracy of 49.7% (41-58) for diagnosing cervical intraepithelial neoplasia (CIN) 1+ lesions. EVA Visual Check has a sensitivity of 89.3% (72-98), specificity of 26.1% (18-35), PPV of 22.1% (15-31), NPV of 91.2% (76-98), and diagnostic accuracy of 38.1% (30-46) for CIN 2+ lesions.

Conclusion

MobileODT EVA colposcope with AI has sensitivity comparable to physician's diagnosis, whereas specificity, PPV, and NPV were less than that of physician's diagnosis. It could prove valuable for triage of screen-positive women for further management.

Predicting cervical intraepithelial neoplasia and determining the follow-up period in high-risk human papillomavirus patients

Purpose

Despite strong efforts to promote human papillomavirus (HPV) vaccine and cervical cancer screening, cervical cancer remains a threat to women's reproductive health. Some high-risk HPV types play a crucial role in the progression of cervical cancer and precancerous lesions. Therefore, HPV screening has become an important means to prevent, diagnose, and triage cervical cancer. This study aims to leverage artificial intelligence to predict individual risks of cervical intraepithelial neoplasia (CIN) in women with high-risk HPV infection and to recommend the appropriate triage strategy and follow-up period according to the risk level.

Materials and methods

A total of 475 cases were collected in this study. The sources were from the Department of Gynecology and Obstetrics in a tertiary hospital, a case report on HPV from the PubMed website, and clinical data of cervical cancer patients from The Cancer Genome Atlas (TCGA) database. Through in-depth study of the interaction between high-risk HPV and its risk factors, the risk factor relationship diagram structure was constructed. A Classification of Lesion Stages (CLS) algorithm was designed to predict cervical lesion stages. The risk levels of patients were analyzed based on all risk factors, and follow-up periods were formulated for each risk level.

Results

Our proposed CLS algorithm predicted the probability of occurrence of CIN3-the precancerous lesion stage of cervical cancer. This prediction was based on patients' HPV-16 and -18 infection status, age, presence of persistent infection, and HPV type. Follow-up periods of 3-6 months, 6-12 months, and 3- to 5-year intervals were suggested for high-risk, medium-risk, and low-risk patients, respectively.

Conclusion

A lesion prediction model was constructed to determine the probabilities of occurrence of CIN by analyzing individual data, such as patient lifestyle, physical assessments, and patient complaints, in order to identify high-risk patients. Furthermore, the potential implications of the calculated features were mined to devise prevention strategies.

CervicoXNet: an automated cervicogram interpretation network

Visual inspection with acetic acid (VIA) is a pre-cancerous screening program for low-middle-income countries (LMICs). Due to the limited number of oncology-gynecologist clinicians in LMICs, VIA examinations are performed mainly by medical workers. However, the inability of the medical workers to recognize a significant pattern based on cervicograms, VIA examination produces high inter-observer variance and high false-positive rate. This study proposed an automated cervicogram interpretation using explainable convolutional neural networks named "CervicoXNet" to support medical workers decision. The total number of 779 cervicograms was used for the learning process: 487 with VIA ( +) and 292 with VIA ( -). We performed data augmentation process under a geometric transformation scenario, such process produces 7325 cervicogram with VIA ( -) and 7242 cervicogram with VIA ( +). The proposed model outperformed other deep learning models, with 99.22% accuracy, 100% sensitivity, and 98.28% specificity. Moreover, to test the robustness of the proposed model, colposcope images used to validate the model's generalization ability. The results showed that the proposed architecture still produced satisfactory performance, with 98.11% accuracy, 98.33% sensitivity, and 98% specificity. It can be proven that the proposed model has been achieved satisfactory results. To make the prediction results visually interpretable, the results are localized with a heat map in fine-grained pixels using a combination of Grad-CAM and guided backpropagation. CervicoXNet can be used an alternative early screening tool with VIA alone.

Computer-aided diagnosis of cervical dysplasia using colposcopic images

Background

computer-aided diagnosis of medical images is becoming more significant in intelligent medicine. Colposcopy-guided biopsy with pathological diagnosis is the gold standard in diagnosing CIN and invasive cervical cancer. However, it struggles with its low sensitivity in differentiating cancer/HSIL from LSIL/normal, particularly in areas with a lack of skilled colposcopists and access to adequate medical resources.

Methods

the model used the auto-segmented colposcopic images to extract color and texture features using the T-test method. It then augmented minority data using the SMOTE method to balance the skewed class distribution. Finally, it used an RBF-SVM to generate a preliminary output. The results, integrating the TCT, HPV tests, and age, were combined into a naïve Bayes classifier for cervical lesion diagnosis.

Results

the multimodal machine learning model achieved physician-level performance (sensitivity: 51.2%, specificity: 86.9%, accuracy: 81.8%), and it could be interpreted by feature extraction and visualization. With the aid of the model, colposcopists improved the sensitivity from 53.7% to 70.7% with an acceptable specificity of 81.1% and accuracy of 79.6%.

Conclusion

using a computer-aided diagnosis system, physicians could identify cancer/HSIL with greater sensitivity, which guided biopsy to take timely treatment.

Radiomics Diagnostic Tool Based on Deep Learning for Colposcopy Image Classification

Background: Colposcopy imaging is widely used to diagnose, treat and follow-up on premalignant and malignant lesions in the vulva, vagina, and cervix. Thus, deep learning algorithms are being used widely in cervical cancer diagnosis tools. In this study, we developed and preliminarily validated a model based on the Unet network plus SVM to classify cervical lesions on colposcopy images. Methodology: Two sets of images were used: the Intel & Mobile ODT Cervical Cancer Screening public dataset, and a private dataset from a public hospital in Ecuador during a routine colposcopy, after the application of acetic acid and lugol. For the latter, the corresponding clinical information was collected, specifically cytology on the PAP smear and the screening of human papillomavirus testing, prior to colposcopy. The lesions of the cervix or regions of interest were segmented and classified by the Unet and the SVM model, respectively. Results: The CAD system was evaluated for the ability to predict the risk of cervical cancer. The lesion segmentation metric results indicate a DICE of 50%, a precision of 65%, and an accuracy of 80%. The classification results’ sensitivity, specificity, and accuracy were 70%, 48.8%, and 58%, respectively. Randomly, 20 images were selected and sent to 13 expert colposcopists for a statistical comparison between visual evaluation experts and the CAD tool (p-value of 0.597). Conclusion: The CAD system needs to improve but could be acceptable in an environment where women have limited access to clinicians for the diagnosis, follow-up, and treatment of cervical cancer; better performance is possible through the exploration of other deep learning methods with larger datasets.

Colposcopic multimodal fusion for the classification of cervical lesions

Objective: Cervical cancer is one of the two biggest killers of women and early detection of cervical precancerous lesions can effectively improve the survival rate of patients. Manual diagnosis by combining colposcopic images and clinical examination results is the main clinical diagnosis method at present. Developing an intelligent diagnosis algorithm based on artificial intelligence is an inevitable trend to solve the objectification of diagnosis and improve the quality and efficiency of diagnosis.Approach: A colposcopic multimodal fusion convolutional neural network (CMF-CNN) was proposed for the classification of cervical lesions. Mask region convolutional neural network was used to detect the cervical region while the encoding network EfficientNet-B3 was introduced to extract the multimodal image features from the acetic image and iodine image. Finally, Squeeze-and-Excitation, Atrous Spatial Pyramid Pooling, and convolution block were also adopted to encode and fuse the patient's clinical text information.Main results: The experimental results showed that in 7106 cases of colposcopy, the accuracy, macro F1-score, macro-areas under the curve of the proposed model were 92.70%, 92.74%, 98.56%, respectively. They are superior to the mainstream unimodal image classification models.Significance: CMF-CNN proposed in this paper combines multimodal information, which has high performance in the classification of cervical lesions in colposcopy, so it can provide comprehensive diagnostic aid.

Simplified Convolutional Neural Network Application for Cervix Type Classification via Colposcopic Images

The inner parts of the human body are usually inspected endoscopically using special equipment. For instance, each part of the female reproductive system can be examined endoscopically (laparoscopy, hysteroscopy, and colposcopy). The primary purpose of colposcopy is the early detection of malignant lesions of the cervix. Cervical cancer (CC) is one of the most common cancers in women worldwide, especially in middle- and low-income countries. Therefore, there is a growing demand for approaches that aim to detect precancerous lesions, ideally without quality loss. Despite its high efficiency, this method has some disadvantages, including subjectivity and pronounced dependence on the operator’s experience. The objective of the current work is to propose an alternative to overcoming these limitations by utilizing the neural network approach. The classifier is trained to recognize and classify lesions. The classifier has a high recognition accuracy and a low computational complexity. The classification accuracies for the classes normal, LSIL, HSIL, and suspicious for invasion were 95.46%, 79.78%, 94.16%, and 97.09%, respectively. We argue that the proposed architecture is simpler than those discussed in other articles due to the use of the global averaging level of the pool. Therefore, the classifier can be implemented on low-power computing platforms at a reasonable cost.

Chronic Cervicitis and Cervical Cancer Detection Based on Deep Learning of Colposcopy Images Toward Translational Pharmacology

With the rapid development of deep learning, automatic image recognition is widely used in medical development. In this study, a deep learning convolutional neural network model was developed to recognize and classify chronic cervicitis and cervical cancer. A total of 10,012 colposcopy images of 1,081 patients from Hunan Provincial People’s Hospital in China were recorded. Five different colposcopy image features of the cervix including chronic cervicitis, intraepithelial lesions, cancer, polypus, and free hyperplastic squamous epithelial tissue were extracted to be applied in our deep learning network convolutional neural network model. However, the result showed a low accuracy (42.16%) due to computer misrecognition of chronic cervicitis, intraepithelial lesions, and free hyperplastic squamous epithelial tissue with high similarity. To optimize this model, we selected two significant feature images: chronic cervicitis and cervical cancer to input into a deep learning network. The result indicates high accuracy and robustness with an accuracy of 95.19%, which can be applied to detect whether the patient has chronic cervicitis or cervical cancer based on the patient’s colposcopy images.

A Weakly Supervised Deep Learning Method for Guiding Ovarian Cancer Treatment and Identifying an Effective Biomarker

Ovarian cancer is a common malignant gynecological disease. Molecular target therapy, i.e., antiangiogenesis with bevacizumab, was found to be effective in some patients of epithelial ovarian cancer (EOC). Although careful patient selection is essential, there are currently no biomarkers available for routine therapeutic usage. To the authors’ best knowledge, this is the first automated precision oncology framework to effectively identify and select EOC and peritoneal serous papillary carcinoma (PSPC) patients with positive therapeutic effect. From March 2013 to January 2021, we have a database, containing four kinds of immunohistochemical tissue samples, including AIM2, c3, C5 and NLRP3, from patients diagnosed with EOC and PSPC and treated with bevacizumab in a hospital-based retrospective study. We developed a hybrid deep learning framework and weakly supervised deep learning models for each potential biomarker, and the experimental results show that the proposed model in combination with AIM2 achieves high accuracy 0.92, recall 0.97, F-measure 0.93 and AUC 0.97 for the first experiment (66% training and 34%testing) and high accuracy 0.86 ± 0.07, precision 0.9 ± 0.07, recall 0.85 ± 0.06, F-measure 0.87 ± 0.06 and AUC 0.91 ± 0.05 for the second experiment using five-fold cross validation, respectively. Both Kaplan-Meier PFS analysis and Cox proportional hazards model analysis further confirmed that the proposed AIM2-DL model is able to distinguish patients gaining positive therapeutic effects with low cancer recurrence from patients with disease progression after treatment (p < 0.005).

Role of Artificial Intelligence Interpretation of Colposcopic Images in Cervical Cancer Screening

The accuracy of colposcopic diagnosis depends on the skill and proficiency of physicians. This study evaluated the feasibility of interpreting colposcopic images with the assistance of artificial intelligence (AI) for the diagnosis of high-grade cervical intraepithelial lesions. This study included female patients who underwent colposcopy-guided biopsy in 2020 at two institutions in the Republic of Korea. Two experienced colposcopists reviewed all images separately. The Cerviray AI® system (AIDOT, Seoul, Korea) was used to interpret the cervical images. AI demonstrated improved sensitivity with comparable specificity and positive predictive value when compared with the colposcopic impressions of each clinician. The areas under the curve were greater with combined impressions (both AI and that of the two colposcopists) of high-grade lesions, when compared with the individual impressions of each colposcopist. This study highlights the feasibility of the application of an AI system in cervical cancer screening. AI interpretation can be utilized as an assisting tool in combination with human colposcopic evaluation of exocervix.

Deep learning based cervical screening by the cross-modal integration of colposcopy, cytology, and HPV test

PURPOSE

To develop and evaluate the colposcopy based deep learning model using all kinds of cervical images for cervical screening, and investigate the synergetic benefits of the colposcopy, the cytology test, and the HPV test for improving cervical screening performance.

METHODS

This study consisted of 2160 women who underwent cervical screening, there were 442 cases with the histopathological confirmed high-grade squamous intraepithelial lesion (HSIL) or cancer, and the remained 1718 women were controls. Three kinds of cervical images were acquired from colposcopy including the saline image of cervix after saline irrigation, the acetic acid image of cervix after applying acetic acid solution, and the iodine image of cervix after applying Lugol's iodine solution. Each kind of image was used to build a single-image based deep learning model by the VGG-16 convolutional neural network, respectively. A multiple-images based deep learning model was built using multivariable logistic regression (MLR) by combining the single-image based models. The performance of the visual inspection was also obtained. The results of the cytology test and HPV test were used to build a Cytology-HPV joint diagnostic model by MLR. Finally, a cross-modal integrated model was built using MLR by combining the multiple-images based deep learning model, the cytology test results, and the HPV test results. The performances of models were tested in an independent test set using the area under the receiver operating characteristic curve (AUC).

RESULTS

The saline image, acetic acid image, and iodine image based deep learning models had AUC of 0.760, 0.791, and 0.840. The multiple-images based deep learning model achieved an improved AUC of 0.845. The AUC of the visual inspection was 0.751. The Cytology-HPV joint diagnostic model had an AUC of 0.837, which was higher than the cytology test (AUC = 0.749) and the HPV test (AUC = 0.742). The cross-modal integrated model achieved the best performance with AUC of 0.921.

CONCLUSIONS

Combining all kinds of cervical images were benefit for improving the performance of the colposcopy based deep learning model, and more accurate cervical screening could be achieved by incorporating the colposcopy based deep learning model, the cytology test results, and the HPV test results.

Machine Learning in Modeling of Mouse Behavior

Mouse behavior is a primary outcome in evaluations of therapeutic efficacy. Exhaustive, continuous, multiparametric behavioral phenotyping is a valuable tool for understanding the pathophysiological status of mouse brain diseases. Automated home cage behavior analysis produces highly granulated data both in terms of number of features and sampling frequency. Previously, we demonstrated several ways to reduce feature dimensionality. In this study, we propose novel approaches for analyzing 33-Hz data generated by CleverSys software. We hypothesized that behavioral patterns within short time windows are reflective of physiological state, and that computer modeling of mouse behavioral routines can serve as a predictive tool in classification tasks. To remove bias due to researcher decisions, our data flow is indifferent to the quality, value, and importance of any given feature in isolation. To classify day and night behavior, as an example application, we developed a data preprocessing flow and utilized logistic regression (LG), support vector machines (SVM), random forest (RF), and one-dimensional convolutional neural networks paired with long short-term memory deep neural networks (1DConvBiLSTM). We determined that a 5-min video clip is sufficient to classify mouse behavior with high accuracy. LG, SVM, and RF performed similarly, predicting mouse behavior with 85% accuracy, and combining the three algorithms in an ensemble procedure increased accuracy to 90%. The best performance was achieved by combining the 1DConv and BiLSTM algorithms yielding 96% accuracy. Our findings demonstrate that computer modeling of the home-cage ethome can clearly define mouse physiological state. Furthermore, we showed that continuous behavioral data can be analyzed using approaches similar to natural language processing. These data provide proof of concept for future research in diagnostics of complex pathophysiological changes that are accompanied by changes in behavioral profile.

A pilot study of mobile digital colposcopy in Japanese patients with cervical intraepithelial neoplasm

Digital colposcopy built around a smartphone is becoming common, and this has advantages for telemedicine and data sharing by taking advantage of smartphone characteristics. However, digital colposcopy itself is not allowed in clinical practice in Japan. The aim of the present study was to investigate the feasibility of mobile digital colposcopy incorporating a smartphone for management of cervical screening in Japanese patients. Patients who underwent colposcopy at Saitama Medical University International Medical Center between July 2019 and February 2020 were enrolled in the present study. The inclusion criteria were women aged 21-65 years old referred for colposcopy following the Japanese standard of care. Written informed consent was obtained from all patients. A total of 40 patients (52 tests) were included in the study. Following the standard of care, acetic acid was applied to the cervix, which was then visualized using a traditional colposcope, with biopsies collected as necessary. The cervix was then visualized and an imaged was captured using a mobile digital colposcope incorporating a smartphone (EVA System; Mobile ODT). All images were collected before biopsy. Images were stored on a secure cloud portal for subsequent evaluation by the provider who performed the conventional colposcopy, and the diagnoses were compared. The present study was approved by the Institutional Review Board of Saitama Medical University International Medical Center (Hidaka, Japan). The match rates for diagnoses were 75%. The match rates for the actual (from conventional colposcopy) and assumed (from digital colposcopy) biopsy sites were 61, 16 and 23%, based on definitions of the ‘same’, ‘almost the same’ and ‘different’, respectively. The present results indicated that ≥75% cases were equivalent in digital colposcopy and conventional colposcopy. This suggests that digital colposcopy may not be inferior to conventional colposcopy.

Comparison of machine and deep learning for the classification of cervical cancer based on cervicography images

Cervical cancer is the second most common cancer in women worldwide with a mortality rate of 60%. Cervical cancer begins with no overt signs and has a long latent period, making early detection through regular checkups vitally immportant. In this study, we compare the performance of two different models, machine learning and deep learning, for the purpose of identifying signs of cervical cancer using cervicography images. Using the deep learning model ResNet-50 and the machine learning models XGB, SVM, and RF, we classified 4119 Cervicography images as positive or negative for cervical cancer using square images in which the vaginal wall regions were removed. The machine learning models extracted 10 major features from a total of 300 features. All tests were validated by fivefold cross-validation and receiver operating characteristics (ROC) analysis yielded the following AUCs: ResNet-50 0.97(CI 95% 0.949-0.976), XGB 0.82(CI 95% 0.797-0.851), SVM 0.84(CI 95% 0.801-0.854), RF 0.79(CI 95% 0.804-0.856). The ResNet-50 model showed a 0.15 point improvement (p < 0.05) over the average (0.82) of the three machine learning methods. Our data suggest that the ResNet-50 deep learning algorithm could offer greater performance than current machine learning models for the purpose of identifying cervical cancer using cervicography images.

Cervical Cancer Diagnostics Healthcare System Using Hybrid Object Detection Adversarial Networks

Cervical cancer is one of the common cancers among women and it causes significant mortality in many developing countries. Diagnosis of cervical lesions is done using pap smear test or visual inspection using acetic acid (staining). Digital colposcopy, an inexpensive methodology, provides painless and efficient screening results. Therefore, automating cervical cancer screening using colposcopy images will be highly useful in saving many lives. Nowadays, many automation techniques using computer vision and machine learning in cervical screening gained attention, paving the way for diagnosing cervical cancer. However, most of the methods rely entirely on the annotation of cervical spotting and segmentation. This paper aims to introduce the Faster Small-Object Detection Neural Networks (FSOD-GAN) to address the cervical screening and diagnosis of cervical cancer and the type of cancer using digital colposcopy images. The proposed approach automatically detects the cervical spot using Faster Region-Based Convolutional Neural Network (FR-CNN) and performs the hierarchical multiclass classification of three types of cervical cancer lesions. Experimentation was done with colposcopy data collected from available open sources consisting of 1,993 patients with three cervical categories, and the proposed approach shows 99% accuracy in diagnosing the stages of cervical cancer.

Deep Metric Learning for Cervical Image Classification

Cervical cancer is caused by the persistent infection of certain types of the Human Papillomavirus (HPV) and is a leading cause of female mortality particularly in low and middle-income countries (LMIC). Visual inspection of the cervix with acetic acid (VIA) is a commonly used technique in cervical screening. While this technique is inexpensive, clinical assessment is highly subjective, and relatively poor reproducibility has been reported. A deep learning-based algorithm for automatic visual evaluation (AVE) of aceto-whitened cervical images was shown to be effective in detecting confirmed precancer (i.e. direct precursor to invasive cervical cancer). The images were selected from a large longitudinal study conducted by the National Cancer Institute in the Guanacaste province of Costa Rica. The training of AVE used annotation for cervix boundary, and the data scarcity challenge was dealt with manually optimized data augmentation. In contrast, we present a novel approach for cervical precancer detection using a deep metric learning-based (DML) framework which does not incorporate any effort for cervix boundary marking. The DML is an advanced learning strategy that can deal with data scarcity and bias training due to class imbalance data in a better way. Three different widely-used state-of-the-art DML techniques are evaluated- (a) Contrastive loss minimization, (b) N-pair embedding loss minimization, and, (c) Batch-hard loss minimization. Three popular Deep Convolutional Neural Networks (ResNet-50, MobileNet, NasNet) are configured for training with DML to produce class-separated (i.e. linearly separable) image feature descriptors. Finally, a K-Nearest Neighbor (KNN) classifier is trained with the extracted deep features. Both the feature quality and classification performance are quantitatively evaluated on the same data set as used in AVE. It shows that, unlike AVE, without using any data augmentation, the best model produced from our research improves specificity in disease detection without compromising sensitivity. The present research thus paves the way for new research directions for the related field.

A Machine Learning Tool to Predict the Response to Neoadjuvant Chemotherapy in Patients with Locally Advanced Cervical Cancer

Despite several studies having identified factors associated with successful treatment outcomes in locally advanced cervical cancer, there is the lack of accurate predictive modeling for progression-free survival (PFS) in patients who undergo radical hysterectomy after neoadjuvant chemotherapy (NACT). Here we investigated whether machine learning (ML) may have the potential to provide a tool to predict neoadjuvant treatment response as PFS. In this retrospective observational study, we analyzed patients with locally advanced cervical cancer (FIGO stages IB2, IB3, IIA1, IIA2, IIB, and IIIC1) who were followed in a tertiary center from 2010 to 2018. Demographic and clinical characteristics were collected at either treatment baseline or at 24-month follow-up. Furthermore, we recorded data about magnetic resonance imaging (MRI) examinations and post-surgery histopathology. Proper feature selection was used to determine an attribute core set. Three different machine learning algorithms, namely Logistic Regression (LR), Random Forest (RFF), and K-nearest neighbors (KNN), were then trained and validated with 10-fold cross-validation to predict 24-month PFS. Our analysis included n. 92 patients. The attribute core set used to train machine learning algorithms included the presence/absence of fornix infiltration at pre-treatment MRI as well as of either parametrium invasion and lymph nodes involvement at post-surgery histopathology. RFF showed the best performance (accuracy 82.4%, precision 83.4%, recall 96.2%, area under receiver operating characteristic curve (AUROC) 0.82). We developed an accurate ML model to predict 24-month PFS.

Computer-Aided Cervical Cancer Diagnosis Using Time-Lapsed Colposcopic Images

Cervical cancer causes the fourth most cancer-related deaths of women worldwide. Early detection of cervical intraepithelial neoplasia (CIN) can significantly increase the survival rate of patients. In this paper, we propose a deep learning framework for the accurate identification of LSIL+ (including CIN and cervical cancer) using time-lapsed colposcopic images. The proposed framework involves two main components, i.e., key-frame feature encoding networks and feature fusion network. The features of the original (pre-acetic-acid) image and the colposcopic images captured at around 60s, 90s, 120s and 150s during the acetic acid test are encoded by the feature encoding networks. Several fusion approaches are compared, all of which outperform the existing automated cervical cancer diagnosis systems using a single time slot. A graph convolutional network with edge features (E-GCN) is found to be the most suitable fusion approach in our study, due to its excellent explainability consistent with the clinical practice. A large-scale dataset, containing time-lapsed colposcopic images from 7,668 patients, is collected from the collaborative hospital to train and validate our deep learning framework. Colposcopists are invited to compete with our computer-aided diagnosis system. The proposed deep learning framework achieves a classification accuracy of 78.33%-comparable to that of an in-service colposcopist-which demonstrates its potential to provide assistance in the realistic clinical scenario.

Predicting a live birth by artificial intelligence incorporating both the blastocyst image and conventional embryo evaluation parameters

BACKGROUND

The achievement of live birth is the goal of assisted reproductive technology in reproductive medicine. When the selected blastocyst is transferred to the uterus, the degree of implantation of the blastocyst is evaluated by microscopic inspection, and the result is only about 30%-40%, and the method of predicting live birth from the blastocyst image is unknown. Live births correlate with several clinical conventional embryo evaluation parameters (CEE), such as maternal age. Therefore, it is necessary to develop artificial intelligence (AI) that combines blastocyst images and CEE to predict live births.

AIM

To develop an AI classifier for blastocyst images and CEE to predict the probability of achieving a live birth.

METHODS

A total of 5691 images of blastocysts on the fifth day after oocyte retrieval obtained from consecutive patients from January 2009 to April 2017 with fully deidentified data were retrospectively enrolled with explanations to patients and a website containing additional information with an opt-out option. We have developed a system in which the original architecture of the deep learning neural network is used to predict the probability of live birth from a blastocyst image and CEE.

RESULTS

The live birth rate was 0.387 (= 1587/4104 cases). The number of independent clinical information for predicting live birth is 10, which significantly avoids multicollinearity. A single AI classifier is composed of ten layers of convolutional neural networks, and each elementwise layer of ten factors is developed and obtained with 42792 as the number of training data points and 0.001 as the L2 regularization value. The accuracy, sensitivity, specificity, negative predictive value, positive predictive value, Youden J index, and area under the curve values for predicting live birth are 0.743, 0.638, 0.789, 0.831, 0.573, 0.427, and 0.740, respectively. The optimal cut-off point of the receiver operator characteristic curve is 0.207.

CONCLUSION

AI classifiers have the potential of predicting live births that humans cannot predict. Artificial intelligence may make progress in assisted reproductive technology.

Classification of cervical neoplasms on colposcopic photography using deep learning

Colposcopy is widely used to detect cervical cancers, but experienced physicians who are needed for an accurate diagnosis are lacking in developing countries. Artificial intelligence (AI) has been recently used in computer-aided diagnosis showing remarkable promise. In this study, we developed and validated deep learning models to automatically classify cervical neoplasms on colposcopic photographs. Pre-trained convolutional neural networks were fine-tuned for two grading systems: the cervical intraepithelial neoplasia (CIN) system and the lower anogenital squamous terminology (LAST) system. The multi-class classification accuracies of the networks for the CIN system in the test dataset were 48.6 ± 1.3% by Inception-Resnet-v2 and 51.7 ± 5.2% by Resnet-152. The accuracies for the LAST system were 71.8 ± 1.8% and 74.7 ± 1.8%, respectively. The area under the curve (AUC) for discriminating high-risk lesions from low-risk lesions by Resnet-152 was 0.781 ± 0.020 for the CIN system and 0.708 ± 0.024 for the LAST system. The lesions requiring biopsy were also detected efficiently (AUC, 0.947 ± 0.030 by Resnet-152), and presented meaningfully on attention maps. These results may indicate the potential of the application of AI for automated reading of colposcopic photographs.

Automated Measurement of Lumbar Lordosis on Radiographs Using Machine Learning and Computer Vision

STUDY DESIGN

Cross sectional database study.

OBJECTIVE

To develop a fully automated artificial intelligence and computer vision pipeline for assisted evaluation of lumbar lordosis.

METHODS

Lateral lumbar radiographs were used to develop a segmentation neural network (n = 629). After synthetic augmentation, 70% of these radiographs were used for network training, while the remaining 30% were used for hyperparameter optimization. A computer vision algorithm was deployed on the segmented radiographs to calculate lumbar lordosis angles. A test set of radiographs was used to evaluate the validity of the entire pipeline (n = 151).

RESULTS

The U-Net segmentation achieved a test dataset dice score of 0.821, an area under the receiver operating curve of 0.914, and an accuracy of 0.862. The computer vision algorithm identified the L1 and S1 vertebrae on 84.1% of the test set with an average speed of 0.14 seconds/radiograph. From the 151 test set radiographs, 50 were randomly chosen for surgeon measurement. When compared with those measurements, our algorithm achieved a mean absolute error of 8.055° and a median absolute error of 6.965° (not statistically significant, P > .05).

CONCLUSION

This study is the first to use artificial intelligence and computer vision in a combined pipeline to rapidly measure a sagittal spinopelvic parameter without prior manual surgeon input. The pipeline measures angles with no statistically significant differences from manual measurements by surgeons. This pipeline offers clinical utility in an assistive capacity, and future work should focus on improving segmentation network performance.

The artificial intelligence-assisted cytology diagnostic system in large-scale cervical cancer screening: A population-based cohort study of 0.7 million women

Background

Adequate cytology is limited by insufficient cytologists in a large‐scale cervical cancer screening. We aimed to develop an artificial intelligence (AI)‐assisted cytology system in cervical cancer screening program.

Methods

We conducted a perspective cohort study within a population‐based cervical cancer screening program for 0.7 million women, using a validated AI‐assisted cytology system. For comparison, cytologists examined all slides classified by AI as abnormal and a randomly selected 10% of normal slides. Each woman with slides classified as abnormal by either AI‐assisted or manual reading was diagnosed by colposcopy and biopsy. The outcomes were histologically confirmed cervical intraepithelial neoplasia grade 2 or worse (CIN2+).

Results

Finally, we recruited 703 103 women, of whom 98 549 were independently screened by AI and manual reading. The overall agreement rate between AI and manual reading was 94.7% (95% confidential interval [CI], 94.5%‐94.8%), and kappa was 0.92 (0.91‐0.92). The detection rates of CIN2+ increased with the severity of cytology abnormality performed by both AI and manual reading (P_trend < 0.001). General estimated equations showed that detection of CIN2+ among women with ASC‐H or HSIL by AI were significantly higher than corresponding groups classified by cytologists (for ASC‐H: odds ratio [OR] = 1.22, 95%CI 1.11‐1.34, P < .001; for HSIL: OR = 1.41, 1.28‐1.55, P < .001). AI‐assisted cytology was 5.8% (3.0%‐8.6%) more sensitive for detection of CIN2+ than manual reading with a slight reduction in specificity.

Conclusions

AI‐assisted cytology system could exclude most of normal cytology, and improve sensitivity with clinically equivalent specificity for detection of CIN2+ compared with manual cytology reading. Overall, the results support AI‐based cytology system for the primary cervical cancer screening in large‐scale population.

The application of deep learning based diagnostic system to cervical squamous intraepithelial lesions recognition in colposcopy images

Background Deep learning has presented considerable potential and is gaining more importance in computer assisted diagnosis. As the gold standard for pathologically diagnosing cervical intraepithelial lesions and invasive cervical cancer, colposcopy-guided biopsy faces challenges in improving accuracy and efficiency worldwide, especially in developing countries. To ease the heavy burden of cervical cancer screening, it is urgent to establish a scientific, accurate and efficient method for assisting diagnosis and biopsy. Methods The data were collected to establish three deep-learning-based models. For every case, one saline image, one acetic image, one iodine image and the corresponding clinical information, including age, the results of human papillomavirus testing and cytology, type of transformation zone, and pathologic diagnosis, were collected. The dataset was proportionally divided into three subsets including the training set, the test set and the validation set, at a ratio of 8:1:1. The validation set was used to evaluate model performance. After model establishment, an independent dataset of high-definition images was collected to further evaluate the model performance. In addition, the comparison of diagnostic accuracy between colposcopists and models weas performed. Results The sensitivity, specificity and accuracy of the classification model to differentiate negative cases from positive cases were 85.38%, 82.62% and 84.10% respectively, with an AUC of 0.93. The recall and DICE of the segmentation model to segment suspicious lesions in acetic images were 84.73% and 61.64%, with an average accuracy of 95.59%. Furthermore, 84.67% of high-grade lesions were detected by the acetic detection model. Compared to colposcopists, the diagnostic system performed better in ordinary colposcopy images but slightly unsatisfactory in high-definition images. Implications The deep learning-based diagnostic system could help assist colposcopy diagnosis and biopsy for HSILs.

Comparative Analysis of Rhino-Cytological Specimens with Image Analysis and Deep Learning Techniques

Cytological study of the nasal mucosa (also known as rhino-cytology) represents an important diagnostic aid that allows highlighting of the presence of some types of rhinitis through the analysis of cellular features visible under a microscope. Nowadays, the automated detection and classification of cells benefit from the capacity of deep learning techniques in processing digital images of the cytological preparation. Even though the results of such automatic systems need to be validated by a specialized rhino-cytologist, this technology represents a valid support that aims at increasing the accuracy of the analysis while reducing the required time and effort. The quality of the rhino-cytological preparation, which is clearly important for the microscope observation phase, is also fundamental for the automatic classification process. In fact, the slide-preparing technique turns out to be a crucial factor among the multiple ones that may modify the morphological and chromatic characteristics of the cells. This paper aims to investigate the possible differences between direct smear (SM) and cytological centrifugation (CYT) slide-preparation techniques, in order to preserve image quality during the observation and cell classification phases in rhino-cytology. Firstly, a comparative study based on image analysis techniques has been put forward. The extraction of densitometric and morphometric features has made it possible to quantify and describe the spatial distribution of the cells in the field images observed under the microscope. Statistical analysis of the distribution of these features has been used to evaluate the degree of similarity between images acquired from SM and CYT slides. The results prove an important difference in the observation process of the cells prepared with the above-mentioned techniques, with reference to cell density and spatial distribution: the analysis of CYT slides has been more difficult than of the SM ones due to the spatial distribution of the cells, which results in a lower cell density than the SM slides. As a marginal part of this study, a performance assessment of the computer-aided diagnosis (CAD) system called Rhino-cyt has also been carried out on both groups of image slide types.

A Novel Approach for Biofilm Detection Based on a Convolutional Neural Network

Rhinology studies anatomy, physiology and diseases affecting the nasal region: one of the most modern techniques to diagnose these diseases is nasal cytology or rhinocytology, which involves analyzing the cells contained in the nasal mucosa under a microscope and researching of other elements such as bacteria, to suspect a pathology. During the microscopic observation, bacteria can be detected in the form of biofilm, that is, a bacterial colony surrounded by an organic extracellular matrix, with a protective function, made of polysaccharides. In the field of nasal cytology, the presence of biofilm in microscopic samples denotes the presence of an infection. In this paper, we describe the design and testing of interesting diagnostic support, for the automatic detection of biofilm, based on a convolutional neural network (CNN). To demonstrate the reliability of the system, alternative solutions based on isolation forest and deep random forest techniques were also tested. Texture analysis is used, with Haralick feature extraction and dominant color. The CNN-based biofilm detection system shows an accuracy of about 98%, an average accuracy of about 100% on the test set and about 99% on the validation set. The CNN-based system designed in this study is confirmed as the most reliable among the best automatic image recognition technologies, in the specific context of this study. The developed system allows the specialist to obtain a rapid and accurate identification of the biofilm in the slide images.

MobileNetV2 Ensemble for Cervical Precancerous Lesions Classification

Women’s cancers remain a major challenge for many health systems. Between 1991 and 2017, the death rate for all major cancers fell continuously in the United States, excluding uterine cervix and uterine corpus cancers. Together with HPV (Human Papillomavirus) testing and cytology, colposcopy has played a central role in cervical cancer screening. This medical procedure allows physicians to view the cervix at a magnification of up to 10%. This paper presents an automated colposcopy image analysis framework for the classification of precancerous and cancerous lesions of the uterine cervix. This framework is based on an ensemble of MobileNetV2 networks. Our experimental results show that this method achieves accuracies of 83.33% and 91.66% on the four-class and binary classification tasks, respectively. These results are promising for the future use of automatic classification methods based on deep learning as tools to support medical doctors.

Application of deep learning to the classification of uterine cervical squamous epithelial lesion from colposcopy images combined with HPV types

The aim of the present study was to explore the feasibility of using deep learning, such as artificial intelligence (AI), to classify cervical squamous epithelial lesions (SILs) from colposcopy images combined with human papilloma virus (HPV) types. Among 330 patients who underwent colposcopy and biopsy performed by gynecological oncologists, a total of 253 patients with confirmed HPV typing tests were enrolled in the present study. Of these patients, 210 were diagnosed with high-grade SIL (HSIL) and 43 were diagnosed with low-grade SIL (LSIL). An original AI classifier with a convolutional neural network catenating with an HPV tensor was developed and trained. The accuracy of the AI classifier and gynecological oncologists was 0.941 and 0.843, respectively. The AI classifier performed better compared with the oncologists, although not significantly. The sensitivity, specificity, positive predictive value, negative predictive value, Youden's J index and the area under the receiver-operating characteristic curve ± standard error for AI colposcopy combined with HPV types and pathological results were 0.956 (43/45), 0.833 (5/6), 0.977 (43/44), 0.714 (5/7), 0.789 and 0.963±0.026, respectively. Although further study is required, the clinical use of AI for the classification of HSIL/LSIL by both colposcopy and HPV type may be feasible.

Towards clinical application of image mining: a systematic review on artificial intelligence and radiomics

PURPOSE

The aim of this systematic review was to analyse literature on artificial intelligence (AI) and radiomics, including all medical imaging modalities, for oncological and non-oncological applications, in order to assess how far the image mining research stands from routine medical application. To do this, we applied a trial phases classification inspired from the drug development process.

METHODS

Among the articles we considered for inclusion from PubMed were multimodality AI and radiomics investigations, with a validation analysis aimed at relevant clinical objectives. Quality assessment of selected papers was performed according to the QUADAS-2 criteria. We developed the phases classification criteria for image mining studies.

RESULTS

Overall 34,626 articles were retrieved, 300 were selected applying the inclusion/exclusion criteria, and 171 high-quality papers (QUADAS-2 ≥ 7) were identified and analysed. In 27/171 (16%), 141/171 (82%), and 3/171 (2%) studies the development of an AI-based algorithm, radiomics model, and a combined radiomics/AI approach, respectively, was described. A total of 26/27(96%) and 1/27 (4%) AI studies were classified as phase II and III, respectively. Consequently, 13/141 (9%), 10/141 (7%), 111/141 (79%), and 7/141 (5%) radiomics studies were classified as phase 0, I, II, and III, respectively. All three radiomics/AI studies were categorised as phase II trials.

CONCLUSIONS

The results of the studies are promising but still not mature enough for image mining tools to be implemented in the clinical setting and be widely used. The transfer learning from the well-known drug development process, with some specific adaptations to the image mining discipline could represent the most effective way for radiomics and AI algorithms to become the standard of care tools.

Application of deep learning to the classification of uterine cervical squamous epithelial lesion from colposcopy images

The aim of the present study was to explore the feasibility of using deep learning as artificial intelligence (AI) to classify cervical squamous epithelial lesions (SIL) from colposcopy images. A total of 330 patients who underwent colposcopy and biopsy by gynecologic oncologists were enrolled in the current study. A total of 97 patients received a pathological diagnosis of low-grade SIL (LSIL) and 213 of high-grade SIL (HSIL). An original AI-classifier with 11 layers of the convolutional neural network was developed and trained. The accuracy, sensitivity, specificity and Youden's J index of the AI-classifier and oncologists for diagnosing HSIL were 0.823 and 0.797, 0.800 and 0.831, 0.882 and 0.773, and 0.682 and 0.604, respectively. The area under the receiver-operating characteristic curve was 0.826±0.052 (mean ± standard error), and the 95% confidence interval 0.721–0.928. The optimal cut-off point was 0.692. Cohen's Kappa coefficient for AI and colposcopy was 0.437 (P<0.0005). The AI-classifier performed better than oncologists, although not significantly. Although further study is required, the clinical use of AI for the classification of HSIL/LSIL from by colposcopy may be feasible.

Feasibility of predicting live birth by combining conventional embryo evaluation with artificial intelligence applied to a blastocyst image in patients classified by age

PURPOSE

To identify the multivariate logistic regression in a combination (combination method) involving artificial intelligence (AI) classifiers in images of blastocysts along with a conventional embryo evaluation (CEE) to predict the probability of accomplishing a live birth in patients classified by maternal age.

METHODS

Retrospectively, a total of 5691 blastocysts were enrolled. Images captured 115 hours or 139 hours if not yet sufficiently large after insemination were classified according to age as follows: <35, 35-37, 38-39, 40-41, and ≥42 years old. The classifiers for each category were created by using convolutional neural networks associated with deep learning. Next, the feasibility of a method combining AI with multivariate logistic model functions by CEE was investigated.

RESULTS

The values of the area under the curve (AUC) and the accuracies to predict live birth achieved by the CEE/AI/combination methods were 0.651/0.634/0.655, 0.697/0.688/0.723, 0.771/0.728/0.791, 0.788/0.743/0.806 and 0.820/0.837/0.888, and 0.631/0.647/0.616, 0.687/0.675/0.671, 0.725/0.697/0.732, 0.714/0.776/0.801, and 0.910/0.866/0.784 for age categories of <35, 35-37, 38-39, 40-41, and ≥42 years old, respectively.

CONCLUSIONS

Though there were mostly no significant differences regarding the AUC and the sensitivity plus specificity in all age categories, the combination method seemed to be the best.

Protein functional annotation of simultaneously improved stability, accuracy and false discovery rate achieved by a sequence-based deep learning

Functional annotation of protein sequence with high accuracy has become one of the most important issues in modern biomedical studies, and computational approaches of significantly accelerated analysis process and enhanced accuracy are greatly desired. Although a variety of methods have been developed to elevate protein annotation accuracy, their ability in controlling false annotation rates remains either limited or not systematically evaluated. In this study, a protein encoding strategy, together with a deep learning algorithm, was proposed to control the false discovery rate in protein function annotation, and its performances were systematically compared with that of the traditional similarity-based and de novo approaches. Based on a comprehensive assessment from multiple perspectives, the proposed strategy and algorithm were found to perform better in both prediction stability and annotation accuracy compared with other de novo methods. Moreover, an in-depth assessment revealed that it possessed an improved capacity of controlling the false discovery rate compared with traditional methods. All in all, this study not only provided a comprehensive analysis on the performances of the newly proposed strategy but also provided a tool for the researcher in the fields of protein function annotation.

Recognition and Clinical Diagnosis of Cervical Cancer Cells Based on our Improved Lightweight Deep Network for Pathological Image

Accurate recognition of cervical cancer cells is of great significance to clinical diagnosis, but these existing algorithms are designed by low-level manual feature, and their performance improvements are limited an improved algorithm based on residual neural network is proposed to improve the accuracy of diagnosis. Firstly, momentum parameters are introduced into the training model; secondly, by changing the number of training samples, the recognition rate of the algorithm can be improved. Therefore, aiming at the task of object recognition under resource constrained condition, we optimize the design method of the network structure such as convolution operation, model parameter compression and enhancement of feature expression depth, and design and implement the lightweight network model structure for embedded platform. Our proposed deep network model can reduce the parameters of the model and the resources needed for operation under the condition of guaranteeing the precision. The experimental results show that the lightweight deep model has better performance than that of the existing comparison models, and it can achieve the model accuracy of 94.1% under the condition that the model with fewer parameters on cervical cells data set.

Feasibility of artificial intelligence for predicting live birth without aneuploidy from a blastocyst image

PURPOSE

To make the artificial intelligence (AI) classifiers of the image of the blastocyst implanted later in order to predict the probability of achieving live birth.

METHODS

A system for using the machine learning approaches, which are logistic regression, naive Bayes, nearest neighbors, random forest, neural network, and support vector machine, of artificial intelligence to predict the probability of live birth from a blastocyst image was developed. Eighty images of blastocysts that led to live births and 80 images of blastocysts that led to aneuploid miscarriages were used to create an AI-based method with 5-fold cross-validation retrospectively for classifying embryos.

RESULTS

The logistic regression method showed the best results. The accuracy, sensitivity, specificity, positive predictive value, and negative predictive value were 0.65, 0.60, 0.70, 0.67, and 0.64, respectively. Area under the curve was 0.65 ± 0.04 (mean ± SE). Estimated probability of belonging to the live birth category was found significantly related to the probability of live birth (P < 0.005).

CONCLUSIONS

Classifiers using artificial intelligence applied toward a blastocyst image have a potential to show the probability of live birth being the outcome.

Deep learning based classification of ultrasound images for thyroid nodules: a large scale of pilot study

Background

To explore the ability of the deep learning network Inception-v3 to differentiate between papillary thyroid carcinomas (PTCs) and benign nodules in ultrasound images.

Methods

A total of 2,836 thyroid ultrasound images from 2,235 patients were divided into a training dataset and a test dataset. Inception-v3 was trained and tested to crop the margin of the images of nodules and provide a differential diagnosis. The sizes and sonographic features of nodules were further analysed to identify the factors that may influence diagnostic efficiency. Statistical analyses included χ² and Fisher's exact tests and univariate and multivariate analyses.

Results

There were 1,275 PTCs and 1,162 benign nodules in the training group and 209 PTCs and 190 benign nodules in the test group. A margin size of 50 pixels and an input size of 384×384 showed the best outcome after training, and these parameters were selected for the test group. In the test group, the sensitivity and specificity for Inception-v3 were 93.3% (195/209) and 87.4% (166/190), respectively. Inception-v3 displayed the highest accuracy for 0.5-1.0 cm nodules. The accuracy differed according to the margin description (P=0.024). Taller nodules were more accurately diagnosed than were wider nodules (P=0.015). Microcalcification [odds ratio (OR) =0.254, 95% confidence interval (CI): 0.076-0.847, P=0.026] and taller shape (OR =0.243, 95% CI: 0.073-0.810, P=0.021) were negatively associated with misdiagnosis rate.

Conclusions

Inception-v3 can achieve an excellent diagnostic efficiency. Nodules that are 0.5-1.0 cm in size and have microcalcification and a taller shape can be more accurately diagnosed by Inception-v3.

Feasibility of deep learning for predicting live birth from a blastocyst image in patients classified by age

PURPOSE

To identify artificial intelligence (AI) classifiers in images of blastocysts to predict the probability of achieving a live birth in patients classified by age. Results are compared to those obtained by conventional embryo (CE) evaluation.

METHODS

A total of 5691 blastocysts were retrospectively enrolled. Images captured 115 hours after insemination (or 139 hours if not yet large enough) were classified according to maternal age as follows: <35, 35-37, 38-39, 40-41, and ≥42 years. The classifiers for each category and a classifier for all ages were related to convolutional neural networks associated with deep learning. Then, the live birth functions predicted by the AI and the multivariate logistic model functions predicted by CE were tested. The feasibility of the AI was investigated.

RESULTS

The accuracies of AI/CE for predicting live birth were 0.64/0.61, 0.71/0.70, 0.78/0.77, 0.81/0.83, 0.88/0.94, and 0.72/0.74 for the age categories <35, 35-37, 38-39, 40-41, and ≥42 years and all ages, respectively. The sum value of the sensitivity and specificity revealed that AI performed better than CE (P = 0.01).

CONCLUSIONS

AI classifiers categorized by age can predict the probability of live birth from an image of the blastocyst and produced better results than were achieved using CE.

Survival outcome prediction in cervical cancer: Cox models vs deep-learning model

BACKGROUND

Historically, the Cox proportional hazard regression model has been the mainstay for survival analyses in oncologic research. The Cox proportional hazard regression model generally is used based on an assumption of linear association. However, it is likely that, in reality, there are many clinicopathologic features that exhibit a nonlinear association in biomedicine.

OBJECTIVE

The purpose of this study was to compare the deep-learning neural network model and the Cox proportional hazard regression model in the prediction of survival in women with cervical cancer.

STUDY DESIGN

This was a retrospective pilot study of consecutive cases of newly diagnosed stage I-IV cervical cancer from 2000-2014. A total of 40 features that included patient demographics, vital signs, laboratory test results, tumor characteristics, and treatment types were assessed for analysis and grouped into 3 feature sets. The deep-learning neural network model was compared with the Cox proportional hazard regression model and 3 other survival analysis models for progression-free survival and overall survival. Mean absolute error and concordance index were used to assess the performance of these 5 models.

RESULTS

There were 768 women included in the analysis. The median age was 49 years, and the majority were Hispanic (71.7%). The majority of tumors were squamous (75.3%) and stage I (48.7%). The median follow-up time was 40.2 months; there were 241 events for recurrence and progression and 170 deaths during the follow-up period. The deep-learning model showed promising results in the prediction of progression-free survival when compared with the Cox proportional hazard regression model (mean absolute error, 29.3 vs 316.2). The deep-learning model also outperformed all the other models, including the Cox proportional hazard regression model, for overall survival (mean absolute error, Cox proportional hazard regression vs deep-learning, 43.6 vs 30.7). The performance of the deep-learning model further improved when more features were included (concordance index for progression-free survival: 0.695 for 20 features, 0.787 for 36 features, and 0.795 for 40 features). There were 10 features for progression-free survival and 3 features for overall survival that demonstrated significance only in the deep-learning model, but not in the Cox proportional hazard regression model. There were no features for progression-free survival and 3 features for overall survival that demonstrated significance only in the Cox proportional hazard regression model, but not in the deep-learning model.

CONCLUSION

Our study suggests that the deep-learning neural network model may be a useful analytic tool for survival prediction in women with cervical cancer because it exhibited superior performance compared with the Cox proportional hazard regression model. This novel analytic approach may provide clinicians with meaningful survival information that potentially could be integrated into treatment decision-making and planning. Further validation studies are necessary to support this pilot study.

Optimization of a Deep-Learning Method Based on the Classification of Images Generated by Parameterized Deep Snap a Novel Molecular-Image-Input Technique for Quantitative Structure-Activity Relationship (QSAR) Analysis

Numerous chemical compounds are distributed around the world and may affect the homeostasis of the endocrine system by disrupting the normal functions of hormone receptors. Although the risks associated with these compounds have been evaluated by acute toxicity testing in mammalian models, the chronic toxicity of many chemicals remains due to high cost of the compounds and the testing, etc. However, computational approaches may be promising alternatives and reduce these evaluations. Recently, deep learning (DL) has been shown to be promising prediction models with high accuracy for recognition of images, speech, signals, and videos since it greatly benefits from large datasets. Recently, a novel DL-based technique called DeepSnap was developed to conduct QSAR analysis using three-dimensional images of chemical structures. It can be used to predict the potential toxicity of many different chemicals to various receptors without extraction of descriptors. DeepSnap has been shown to have a very high capacity in tests using Tox21 quantitative qHTP datasets. Numerous parameters must be adjusted to use the DeepSnap method but they have not been optimized. In this study, the effects of these parameters on the performance of the DL prediction model were evaluated in terms of the loss in validation as an indicator for evaluating the performance of the DL using the toxicity information in the Tox21 qHTP database. The relations of the parameters of DeepSnap such as (1) number of molecules per SDF split into (2) zoom factor percentage, (3) atom size for van der waals percentage, (4) bond radius, (5) minimum bond distance, and (6) bond tolerance, with the validation loss following quadratic function curves, which suggests that optimal thresholds exist to attain the best performance with these prediction models. Using the parameter values set with the best performance, the prediction model of chemical compounds for CAR agonist was built using 64 images, at 105° angle, with AUC of 0.791. Thus, based on these parameters, the proposed DeepSnap-DL approach will be highly reliable and beneficial to establish models to assess the risk associated with various chemicals.