1
|
Xue Y, Zhang J, Li C, Liu X, Kuang W, Deng J, Wang J, Tan X, Li S, Li C. Machine learning for screening and predicting the risk of anti-MDA5 antibody in juvenile dermatomyositis children. Front Immunol 2023; 13:940802. [PMID: 36703989 PMCID: PMC9872019 DOI: 10.3389/fimmu.2022.940802] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/10/2022] [Accepted: 11/14/2022] [Indexed: 01/11/2023] Open
Abstract
Objective The anti-MDA5 (anti-melanoma differentiation associated gene 5) antibody is often associated with a poor prognosis in juvenile dermatomyositis (JDM) patients. In many developing countries, there is limited ability to access myositis- specific antibodies due to financial and technological issues, especially in remote regions. This study was performed to develop a prediction model for screening anti-MDA5 antibodies in JDM patients with commonly available clinical findings. Methods A cross-sectional study was undertaken with 152 patients enrolled from the inpatient wards of Beijing Children's Hospital between June 2018 and September 2021. Stepwise logistic regression, least absolute shrinkage and selection operator (LASSO) regression, and the random forest (RF) method were used to fit the model. Model discrimination, calibration, and decision curve analysis were performed for validation. Results The final prediction model included eight clinical variables (gender, fever, alopecia, periungual telangiectasia, digital ulcer, interstitial lung disease, arthritis/arthralgia, and Gottron sign) and four auxiliary results (WBC, CK, CKMB, and ALB). An anti-MDA5 antibody risk probability-predictive nomogram was established with an AUC of 0.975 predicted by the random forest algorithm. The model was internally validated by Harrell's concordance index (0.904), the Brier score (0.052), and a 500 bootstrapped satisfactory calibration curve. According to the net benefit and predicted probability thresholds of decision curve analysis, the established model showed a significantly higher net benefit than the traditional logistic regression model. Conclusion We developed a prediction model using routine clinical assessments to screen for JDM patients likely to be anti-MDA5 positive. This new tool may effectively predict the detection of anti-MDA5 in these patients using a non-invasive and efficient way.
Collapse
Affiliation(s)
| | | | | | | | | | | | | | | | | | - Caifeng Li
- Department of Rheumatology, Beijing Children's Hospital, Capital Medical Universtity, National Centre for Children's Health, Beijing, China
| |
Collapse
|
2
|
Supervised Classification of Healthcare Text Data Based on Context-Defined Categories. MATHEMATICS 2022. [DOI: 10.3390/math10122005] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 12/10/2022]
Abstract
Achieving a good success rate in supervised classification analysis of a text dataset, where the relationship between the text and its label can be extracted from the context, but not from isolated words in the text, is still an important challenge facing the fields of statistics and machine learning. For this purpose, we present a novel mathematical framework. We then conduct a comparative study between established classification methods for the case where the relationship between the text and the corresponding label is clearly depicted by specific words in the text. In particular, we use logistic LASSO, artificial neural networks, support vector machines, and decision-tree-like procedures. This methodology is applied to a real case study involving mapping Consolidated Framework for Implementation and Research (CFIR) constructs to health-related text data and achieves a prediction success rate of over 80% when just the first 55% of the text, or more, is used for training and the remaining for testing. The results indicate that the methodology can be useful to accelerate the CFIR coding process.
Collapse
|
3
|
Pfob A, Sidey-Gibbons C, Barr RG, Duda V, Alwafai Z, Balleyguier C, Clevert DA, Fastner S, Gomez C, Goncalo M, Gruber I, Hahn M, Hennigs A, Kapetas P, Lu SC, Nees J, Ohlinger R, Riedel F, Rutten M, Schaefgen B, Schuessler M, Stieber A, Togawa R, Tozaki M, Wojcinski S, Xu C, Rauch G, Heil J, Golatta M. The importance of multi-modal imaging and clinical information for humans and AI-based algorithms to classify breast masses (INSPiRED 003): an international, multicenter analysis. Eur Radiol 2022; 32:4101-4115. [PMID: 35175381 PMCID: PMC9123064 DOI: 10.1007/s00330-021-08519-z] [Citation(s) in RCA: 7] [Impact Index Per Article: 3.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/08/2021] [Revised: 09/14/2021] [Accepted: 10/17/2021] [Indexed: 01/23/2023]
Abstract
OBJECTIVES AI-based algorithms for medical image analysis showed comparable performance to human image readers. However, in practice, diagnoses are made using multiple imaging modalities alongside other data sources. We determined the importance of this multi-modal information and compared the diagnostic performance of routine breast cancer diagnosis to breast ultrasound interpretations by humans or AI-based algorithms. METHODS Patients were recruited as part of a multicenter trial (NCT02638935). The trial enrolled 1288 women undergoing routine breast cancer diagnosis (multi-modal imaging, demographic, and clinical information). Three physicians specialized in ultrasound diagnosis performed a second read of all ultrasound images. We used data from 11 of 12 study sites to develop two machine learning (ML) algorithms using unimodal information (ultrasound features generated by the ultrasound experts) to classify breast masses which were validated on the remaining study site. The same ML algorithms were subsequently developed and validated on multi-modal information (clinical and demographic information plus ultrasound features). We assessed performance using area under the curve (AUC). RESULTS Of 1288 breast masses, 368 (28.6%) were histopathologically malignant. In the external validation set (n = 373), the performance of the two unimodal ultrasound ML algorithms (AUC 0.83 and 0.82) was commensurate with performance of the human ultrasound experts (AUC 0.82 to 0.84; p for all comparisons > 0.05). The multi-modal ultrasound ML algorithms performed significantly better (AUC 0.90 and 0.89) but were statistically inferior to routine breast cancer diagnosis (AUC 0.95, p for all comparisons ≤ 0.05). CONCLUSIONS The performance of humans and AI-based algorithms improves with multi-modal information. KEY POINTS • The performance of humans and AI-based algorithms improves with multi-modal information. • Multimodal AI-based algorithms do not necessarily outperform expert humans. • Unimodal AI-based algorithms do not represent optimal performance to classify breast masses.
Collapse
Affiliation(s)
- André Pfob
- grid.5253.10000 0001 0328 4908University Breast Unit, Department of Obstetrics and Gynecology, Heidelberg University Hospital, Im Neuenheimer Feld 440, 69120 Heidelberg, Germany ,grid.240145.60000 0001 2291 4776MD Anderson Center for INSPiRED Cancer Care (Integrated Systems for Patient-Reported Data), The University of Texas MD Anderson Cancer Center, Houston, TX USA
| | - Chris Sidey-Gibbons
- grid.240145.60000 0001 2291 4776MD Anderson Center for INSPiRED Cancer Care (Integrated Systems for Patient-Reported Data), The University of Texas MD Anderson Cancer Center, Houston, TX USA ,grid.240145.60000 0001 2291 4776Department of Symptom Research, The University of Texas MD Anderson Cancer Center, Houston, TX USA
| | - Richard G. Barr
- grid.261103.70000 0004 0459 7529Department of Radiology, Northeast Ohio Medical University, Ravenna, OH USA
| | - Volker Duda
- grid.10253.350000 0004 1936 9756Department of Gynecology and Obstetrics, University of Marburg, Marburg, Germany
| | - Zaher Alwafai
- grid.5603.0Department of Gynecology and Obstetrics, University of Greifswald, Greifswald, Germany
| | - Corinne Balleyguier
- grid.14925.3b0000 0001 2284 9388Department of Radiology, Institut Gustave Roussy, Villejuif Cedex, France
| | - Dirk-André Clevert
- grid.411095.80000 0004 0477 2585Department of Radiology, University Hospital Munich-Grosshadern, Munich, Germany
| | - Sarah Fastner
- grid.5253.10000 0001 0328 4908University Breast Unit, Department of Obstetrics and Gynecology, Heidelberg University Hospital, Im Neuenheimer Feld 440, 69120 Heidelberg, Germany
| | - Christina Gomez
- grid.5253.10000 0001 0328 4908University Breast Unit, Department of Obstetrics and Gynecology, Heidelberg University Hospital, Im Neuenheimer Feld 440, 69120 Heidelberg, Germany
| | - Manuela Goncalo
- grid.8051.c0000 0000 9511 4342Department of Radiology, University of Coimbra, Coimbra, Portugal
| | - Ines Gruber
- grid.10392.390000 0001 2190 1447Department of Gynecology and Obstetrics, University of Tuebingen, Tuebingen, Germany
| | - Markus Hahn
- grid.10392.390000 0001 2190 1447Department of Gynecology and Obstetrics, University of Tuebingen, Tuebingen, Germany
| | - André Hennigs
- grid.5253.10000 0001 0328 4908University Breast Unit, Department of Obstetrics and Gynecology, Heidelberg University Hospital, Im Neuenheimer Feld 440, 69120 Heidelberg, Germany
| | - Panagiotis Kapetas
- grid.22937.3d0000 0000 9259 8492Department of Biomedical Imaging and Image-Guided Therapy, Medical University of Vienna, Vienna, Austria
| | - Sheng-Chieh Lu
- grid.240145.60000 0001 2291 4776MD Anderson Center for INSPiRED Cancer Care (Integrated Systems for Patient-Reported Data), The University of Texas MD Anderson Cancer Center, Houston, TX USA ,grid.240145.60000 0001 2291 4776Department of Symptom Research, The University of Texas MD Anderson Cancer Center, Houston, TX USA
| | - Juliane Nees
- grid.5253.10000 0001 0328 4908University Breast Unit, Department of Obstetrics and Gynecology, Heidelberg University Hospital, Im Neuenheimer Feld 440, 69120 Heidelberg, Germany
| | - Ralf Ohlinger
- grid.5603.0Department of Gynecology and Obstetrics, University of Greifswald, Greifswald, Germany
| | - Fabian Riedel
- grid.5253.10000 0001 0328 4908University Breast Unit, Department of Obstetrics and Gynecology, Heidelberg University Hospital, Im Neuenheimer Feld 440, 69120 Heidelberg, Germany
| | - Matthieu Rutten
- grid.413508.b0000 0004 0501 9798Department of Radiology, Jeroen Bosch Hospital, ‘s-Hertogenbosch, The Netherlands ,grid.10417.330000 0004 0444 9382Radboud University Medical Center, Nijmegen, The Netherlands
| | - Benedikt Schaefgen
- grid.5253.10000 0001 0328 4908University Breast Unit, Department of Obstetrics and Gynecology, Heidelberg University Hospital, Im Neuenheimer Feld 440, 69120 Heidelberg, Germany
| | - Maximilian Schuessler
- grid.5253.10000 0001 0328 4908National Center for Tumor Diseases, Heidelberg University Hospital, Heidelberg, Germany
| | - Anne Stieber
- grid.5253.10000 0001 0328 4908University Breast Unit, Department of Obstetrics and Gynecology, Heidelberg University Hospital, Im Neuenheimer Feld 440, 69120 Heidelberg, Germany
| | - Riku Togawa
- grid.5253.10000 0001 0328 4908University Breast Unit, Department of Obstetrics and Gynecology, Heidelberg University Hospital, Im Neuenheimer Feld 440, 69120 Heidelberg, Germany
| | | | - Sebastian Wojcinski
- grid.461805.e0000 0000 9323 0964Department of Gynecology and Obstetrics, Breast Cancer Center, Klinikum Bielefeld Mitte GmbH, Bielefeld, Germany
| | - Cai Xu
- grid.240145.60000 0001 2291 4776MD Anderson Center for INSPiRED Cancer Care (Integrated Systems for Patient-Reported Data), The University of Texas MD Anderson Cancer Center, Houston, TX USA ,grid.240145.60000 0001 2291 4776Department of Symptom Research, The University of Texas MD Anderson Cancer Center, Houston, TX USA
| | - Geraldine Rauch
- grid.7468.d0000 0001 2248 7639Institute of Biometry and Clinical Epidemiology, Charité – Universitätsmedizin Berlin, Freie Universität Berlin, Humboldt-Universität Zu Berlin, Berlin , Germany
| | - Joerg Heil
- grid.5253.10000 0001 0328 4908University Breast Unit, Department of Obstetrics and Gynecology, Heidelberg University Hospital, Im Neuenheimer Feld 440, 69120 Heidelberg, Germany
| | - Michael Golatta
- grid.5253.10000 0001 0328 4908University Breast Unit, Department of Obstetrics and Gynecology, Heidelberg University Hospital, Im Neuenheimer Feld 440, 69120 Heidelberg, Germany
| |
Collapse
|
4
|
Aggarwal R, Sounderajah V, Martin G, Ting DSW, Karthikesalingam A, King D, Ashrafian H, Darzi A. Diagnostic accuracy of deep learning in medical imaging: a systematic review and meta-analysis. NPJ Digit Med 2021; 4:65. [PMID: 33828217 PMCID: PMC8027892 DOI: 10.1038/s41746-021-00438-z] [Citation(s) in RCA: 252] [Impact Index Per Article: 84.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/06/2020] [Accepted: 02/25/2021] [Indexed: 12/19/2022] Open
Abstract
Deep learning (DL) has the potential to transform medical diagnostics. However, the diagnostic accuracy of DL is uncertain. Our aim was to evaluate the diagnostic accuracy of DL algorithms to identify pathology in medical imaging. Searches were conducted in Medline and EMBASE up to January 2020. We identified 11,921 studies, of which 503 were included in the systematic review. Eighty-two studies in ophthalmology, 82 in breast disease and 115 in respiratory disease were included for meta-analysis. Two hundred twenty-four studies in other specialities were included for qualitative review. Peer-reviewed studies that reported on the diagnostic accuracy of DL algorithms to identify pathology using medical imaging were included. Primary outcomes were measures of diagnostic accuracy, study design and reporting standards in the literature. Estimates were pooled using random-effects meta-analysis. In ophthalmology, AUC's ranged between 0.933 and 1 for diagnosing diabetic retinopathy, age-related macular degeneration and glaucoma on retinal fundus photographs and optical coherence tomography. In respiratory imaging, AUC's ranged between 0.864 and 0.937 for diagnosing lung nodules or lung cancer on chest X-ray or CT scan. For breast imaging, AUC's ranged between 0.868 and 0.909 for diagnosing breast cancer on mammogram, ultrasound, MRI and digital breast tomosynthesis. Heterogeneity was high between studies and extensive variation in methodology, terminology and outcome measures was noted. This can lead to an overestimation of the diagnostic accuracy of DL algorithms on medical imaging. There is an immediate need for the development of artificial intelligence-specific EQUATOR guidelines, particularly STARD, in order to provide guidance around key issues in this field.
Collapse
Affiliation(s)
- Ravi Aggarwal
- Institute of Global Health Innovation, Imperial College London, London, UK
| | | | - Guy Martin
- Institute of Global Health Innovation, Imperial College London, London, UK
| | - Daniel S W Ting
- Singapore Eye Research Institute, Singapore National Eye Center, Singapore, Singapore
| | | | - Dominic King
- Institute of Global Health Innovation, Imperial College London, London, UK
| | - Hutan Ashrafian
- Institute of Global Health Innovation, Imperial College London, London, UK.
| | - Ara Darzi
- Institute of Global Health Innovation, Imperial College London, London, UK
| |
Collapse
|
5
|
Calas MJG, Pereira FPA, Gonçalves LP, Lopes FPPL. Preliminary study of the technical limitations of automated breast ultrasound: from procedure to diagnosis. Radiol Bras 2020; 53:293-300. [PMID: 33071372 PMCID: PMC7545727 DOI: 10.1590/0100-3984.2019.0079] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/22/2022] Open
Abstract
Objective To evaluate the main technical limitations of automated breast ultrasound and to determine the proportion of examinations excluded. Materials and Methods We evaluated 440 automated breast ultrasound examinations performed, over a 12-month period, by technicians using an established protocol. Results In five cases (1.1%), the examination was deemed unacceptable for diagnostic purposes, those examinations therefore being excluded. Conclusion Automated breast ultrasound is expected to overcome some of the major limitations of conventional ultrasound in breast cancer screening. In Brazil, this new method can be accepted for inclusion in routine clinical practice only after its advantages have been validated in the national context.
Collapse
|
6
|
Liu X, Faes L, Kale AU, Wagner SK, Fu DJ, Bruynseels A, Mahendiran T, Moraes G, Shamdas M, Kern C, Ledsam JR, Schmid MK, Balaskas K, Topol EJ, Bachmann LM, Keane PA, Denniston AK. A comparison of deep learning performance against health-care professionals in detecting diseases from medical imaging: a systematic review and meta-analysis. Lancet Digit Health 2019; 1:e271-e297. [PMID: 33323251 DOI: 10.1016/s2589-7500(19)30123-2] [Citation(s) in RCA: 724] [Impact Index Per Article: 144.8] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/05/2019] [Revised: 08/06/2019] [Accepted: 08/14/2019] [Indexed: 02/06/2023]
Abstract
BACKGROUND Deep learning offers considerable promise for medical diagnostics. We aimed to evaluate the diagnostic accuracy of deep learning algorithms versus health-care professionals in classifying diseases using medical imaging. METHODS In this systematic review and meta-analysis, we searched Ovid-MEDLINE, Embase, Science Citation Index, and Conference Proceedings Citation Index for studies published from Jan 1, 2012, to June 6, 2019. Studies comparing the diagnostic performance of deep learning models and health-care professionals based on medical imaging, for any disease, were included. We excluded studies that used medical waveform data graphics material or investigated the accuracy of image segmentation rather than disease classification. We extracted binary diagnostic accuracy data and constructed contingency tables to derive the outcomes of interest: sensitivity and specificity. Studies undertaking an out-of-sample external validation were included in a meta-analysis, using a unified hierarchical model. This study is registered with PROSPERO, CRD42018091176. FINDINGS Our search identified 31 587 studies, of which 82 (describing 147 patient cohorts) were included. 69 studies provided enough data to construct contingency tables, enabling calculation of test accuracy, with sensitivity ranging from 9·7% to 100·0% (mean 79·1%, SD 0·2) and specificity ranging from 38·9% to 100·0% (mean 88·3%, SD 0·1). An out-of-sample external validation was done in 25 studies, of which 14 made the comparison between deep learning models and health-care professionals in the same sample. Comparison of the performance between health-care professionals in these 14 studies, when restricting the analysis to the contingency table for each study reporting the highest accuracy, found a pooled sensitivity of 87·0% (95% CI 83·0-90·2) for deep learning models and 86·4% (79·9-91·0) for health-care professionals, and a pooled specificity of 92·5% (95% CI 85·1-96·4) for deep learning models and 90·5% (80·6-95·7) for health-care professionals. INTERPRETATION Our review found the diagnostic performance of deep learning models to be equivalent to that of health-care professionals. However, a major finding of the review is that few studies presented externally validated results or compared the performance of deep learning models and health-care professionals using the same sample. Additionally, poor reporting is prevalent in deep learning studies, which limits reliable interpretation of the reported diagnostic accuracy. New reporting standards that address specific challenges of deep learning could improve future studies, enabling greater confidence in the results of future evaluations of this promising technology. FUNDING None.
Collapse
Affiliation(s)
- Xiaoxuan Liu
- Department of Ophthalmology, University Hospitals Birmingham NHS Foundation Trust, Birmingham, UK; Academic Unit of Ophthalmology, Institute of Inflammation & Ageing, College of Medical and Dental Sciences, University of Birmingham, Birmingham, UK; Medical Retina Department, Moorfields Eye Hospital NHS Foundation Trust, London, UK; Health Data Research UK, London, UK
| | - Livia Faes
- Medical Retina Department, Moorfields Eye Hospital NHS Foundation Trust, London, UK; Eye Clinic, Cantonal Hospital of Lucerne, Lucerne, Switzerland
| | - Aditya U Kale
- Department of Ophthalmology, University Hospitals Birmingham NHS Foundation Trust, Birmingham, UK
| | - Siegfried K Wagner
- NIHR Biomedical Research Centre for Ophthalmology, Moorfields Eye Hospital NHS Foundation Trust and UCL Institute of Ophthalmology, London, UK
| | - Dun Jack Fu
- Medical Retina Department, Moorfields Eye Hospital NHS Foundation Trust, London, UK
| | - Alice Bruynseels
- Department of Ophthalmology, University Hospitals Birmingham NHS Foundation Trust, Birmingham, UK
| | - Thushika Mahendiran
- Department of Ophthalmology, University Hospitals Birmingham NHS Foundation Trust, Birmingham, UK
| | - Gabriella Moraes
- Medical Retina Department, Moorfields Eye Hospital NHS Foundation Trust, London, UK
| | - Mohith Shamdas
- Academic Unit of Ophthalmology, Institute of Inflammation & Ageing, College of Medical and Dental Sciences, University of Birmingham, Birmingham, UK
| | - Christoph Kern
- Medical Retina Department, Moorfields Eye Hospital NHS Foundation Trust, London, UK; University Eye Hospital, Ludwig Maximilian University of Munich, Munich, Germany
| | | | - Martin K Schmid
- Eye Clinic, Cantonal Hospital of Lucerne, Lucerne, Switzerland
| | - Konstantinos Balaskas
- Medical Retina Department, Moorfields Eye Hospital NHS Foundation Trust, London, UK; NIHR Biomedical Research Centre for Ophthalmology, Moorfields Eye Hospital NHS Foundation Trust and UCL Institute of Ophthalmology, London, UK
| | - Eric J Topol
- Scripps Research Translational Institute, La Jolla, California
| | | | - Pearse A Keane
- NIHR Biomedical Research Centre for Ophthalmology, Moorfields Eye Hospital NHS Foundation Trust and UCL Institute of Ophthalmology, London, UK; Health Data Research UK, London, UK
| | - Alastair K Denniston
- Department of Ophthalmology, University Hospitals Birmingham NHS Foundation Trust, Birmingham, UK; Academic Unit of Ophthalmology, Institute of Inflammation & Ageing, College of Medical and Dental Sciences, University of Birmingham, Birmingham, UK; Centre for Patient Reported Outcome Research, Institute of Applied Health Research, University of Birmingham, Birmingham, UK; NIHR Biomedical Research Centre for Ophthalmology, Moorfields Eye Hospital NHS Foundation Trust and UCL Institute of Ophthalmology, London, UK; Health Data Research UK, London, UK.
| |
Collapse
|
7
|
Development of a computer-aided tool for the pattern recognition of facial features in diagnosing Turner syndrome: comparison of diagnostic accuracy with clinical workers. Sci Rep 2018; 8:9317. [PMID: 29915349 PMCID: PMC6006259 DOI: 10.1038/s41598-018-27586-9] [Citation(s) in RCA: 13] [Impact Index Per Article: 2.2] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/27/2017] [Accepted: 06/04/2018] [Indexed: 12/20/2022] Open
Abstract
Technologies applied for the recognition of facial features in diagnosing certain disorders seem to be promising in reducing the medical burden and improve the efficiency. This pilot study aimed to develop a computer-assisted tool for the pattern recognition of facial features for diagnosing Turner syndrome (TS). Photographs of 54 patients with TS and 158 female controls were collected from July 2016 to May 2017. Finally, photographs of 32 patients with TS and 96 age-matched controls were included in the study that were further divided equally into training and testing groups. The process of automatic classification consisted of image preprocessing, facial feature extraction, feature reduction and fusion, automatic classification, and result presentation. A total of 27 physicians and 21 medical students completed a web-based test including the same photographs used in computer testing. After training, the automatic facial classification system for diagnosing TS achieved a 68.8% sensitivity and 87.5% specificity (and a 67.6% average sensitivity and 87.9% average specificity after resampling), which was significantly higher than the average sensitivity (57.4%, P < 0.001) and specificity (75.4%, P < 0.001) of 48 participants, respectively. The accuracy of this system was satisfactory and better than the diagnosis by clinicians. However, the system necessitates further improvement for achieving a high diagnostic accuracy in clinical practice.
Collapse
|
8
|
Li G, Zhou X, Liu J, Chen Y, Zhang H, Chen Y, Liu J, Jiang H, Yang J, Nie S. Comparison of three data mining models for prediction of advanced schistosomiasis prognosis in the Hubei province. PLoS Negl Trop Dis 2018; 12:e0006262. [PMID: 29447165 PMCID: PMC5831639 DOI: 10.1371/journal.pntd.0006262] [Citation(s) in RCA: 14] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/06/2017] [Revised: 02/28/2018] [Accepted: 01/23/2018] [Indexed: 01/27/2023] Open
Abstract
BACKGROUND In order to better assist medical professionals, this study aimed to develop and compare the performance of three models-a multivariate logistic regression (LR) model, an artificial neural network (ANN) model, and a decision tree (DT) model-to predict the prognosis of patients with advanced schistosomiasis residing in the Hubei province. METHODOLOGY/PRINCIPAL FINDINGS Schistosomiasis surveillance data were collected from a previous study based on a Hubei population sample including 4136 advanced schistosomiasis cases. The predictive models use LR, ANN, and DT methods. From each of the three groups, 70% of the cases (2896 cases) were used as training data for the predictive models. The remaining 30% of the cases (1240 cases) were used as validation groups for performance comparisons between the three models. Prediction performance was evaluated using area under the receiver operating characteristic curve (AUC), sensitivity, specificity, and accuracy. Univariate analysis indicated that 16 risk factors were significantly associated with a patient's outcome of prognosis. In the training group, the mean AUC was 0.8276 for LR, 0.9267 for ANN, and 0.8229 for DT. In the validation group, the mean AUC was 0.8349 for LR, 0.8318 for ANN, and 0.8148 for DT. The three models yielded similar results in terms of accuracy, sensitivity, and specificity. CONCLUSIONS/SIGNIFICANCE Predictive models for advanced schistosomiasis prognosis, respectively using LR, ANN and DT models were proved to be effective approaches based on our dataset. The ANN model outperformed the LR and DT models in terms of AUC.
Collapse
Affiliation(s)
- Guo Li
- Department of Epidemiology and Health Statistics, School of Public Health, Tongji Medical College, Huazhong University of Science and Technology, Wuhan, Hubei, China
- Hubei Provincial Center for Disease Control and Prevention, Wuhan, Hubei, China
| | - Xiaorong Zhou
- Hubei Provincial Center for Disease Control and Prevention, Wuhan, Hubei, China
| | - Jianbing Liu
- Hubei Provincial Center for Disease Control and Prevention, Wuhan, Hubei, China
| | - Yuanqi Chen
- Department of Mathematics, Wuhan University, Wuhan, Hubei, China
| | - Hengtao Zhang
- Department of Mathematics, Wuhan University, Wuhan, Hubei, China
| | - Yanyan Chen
- Hubei Provincial Center for Disease Control and Prevention, Wuhan, Hubei, China
| | - Jianhua Liu
- Yichang Center for Disease Control and Prevention, Yichang, Hubei, China
| | - Hongbo Jiang
- Department of Epidemiology and Biostatistics, School of Public Health, Guangdong Pharmaceutical University, Guangzhou, China
| | - Junjing Yang
- Hubei Provincial Center for Disease Control and Prevention, Wuhan, Hubei, China
| | - Shaofa Nie
- Department of Epidemiology and Health Statistics, School of Public Health, Tongji Medical College, Huazhong University of Science and Technology, Wuhan, Hubei, China
| |
Collapse
|
9
|
Becker AS, Mueller M, Stoffel E, Marcon M, Ghafoor S, Boss A. Classification of breast cancer in ultrasound imaging using a generic deep learning analysis software: a pilot study. Br J Radiol 2018; 91:20170576. [PMID: 29215311 DOI: 10.1259/bjr.20170576] [Citation(s) in RCA: 48] [Impact Index Per Article: 8.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/25/2022] Open
Abstract
OBJECTIVE To train a generic deep learning software (DLS) to classify breast cancer on ultrasound images and to compare its performance to human readers with variable breast imaging experience. METHODS In this retrospective study, all breast ultrasound examinations from January 1, 2014 to December 31, 2014 at our institution were reviewed. Patients with post-surgical scars, initially indeterminate, or malignant lesions with histological diagnoses or 2-year follow-up were included. The DLS was trained with 70% of the images, and the remaining 30% were used to validate the performance. Three readers with variable expertise also evaluated the validation set (radiologist, resident, medical student). Diagnostic accuracy was assessed with a receiver operating characteristic analysis. RESULTS 82 patients with malignant and 550 with benign lesions were included. Time needed for training was 7 min (DLS). Evaluation time for the test data set were 3.7 s (DLS) and 28, 22 and 25 min for human readers (decreasing experience). Receiver operating characteristic analysis revealed non-significant differences (p-values 0.45-0.47) in the area under the curve of 0.84 (DLS), 0.88 (experienced and intermediate readers) and 0.79 (inexperienced reader). CONCLUSION DLS may aid diagnosing cancer on breast ultrasound images with an accuracy comparable to radiologists, and learns better and faster than a human reader with no prior experience. Further clinical trials with dedicated algorithms are warranted. Advances in knowledge: DLS can be trained classify cancer on breast ultrasound images high accuracy even with comparably few training cases. The fast evaluation speed makes real-time image analysis feasible.
Collapse
Affiliation(s)
- Anton S Becker
- 1 Institute of Diagnostic and Interventional Radiology, University Hospital of Zurich , Zurich , Switzerland
| | - Michael Mueller
- 1 Institute of Diagnostic and Interventional Radiology, University Hospital of Zurich , Zurich , Switzerland
| | - Elina Stoffel
- 1 Institute of Diagnostic and Interventional Radiology, University Hospital of Zurich , Zurich , Switzerland
| | - Magda Marcon
- 1 Institute of Diagnostic and Interventional Radiology, University Hospital of Zurich , Zurich , Switzerland
| | - Soleen Ghafoor
- 1 Institute of Diagnostic and Interventional Radiology, University Hospital of Zurich , Zurich , Switzerland
| | - Andreas Boss
- 1 Institute of Diagnostic and Interventional Radiology, University Hospital of Zurich , Zurich , Switzerland
| |
Collapse
|
10
|
Kim SM, Kim Y, Jeong K, Jeong H, Kim J. Logistic LASSO regression for the diagnosis of breast cancer using clinical demographic data and the BI-RADS lexicon for ultrasonography. Ultrasonography 2017; 37:36-42. [PMID: 28618771 PMCID: PMC5769953 DOI: 10.14366/usg.16045] [Citation(s) in RCA: 47] [Impact Index Per Article: 6.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/17/2016] [Revised: 04/14/2017] [Accepted: 04/14/2017] [Indexed: 02/03/2023] Open
Abstract
Purpose The aim of this study was to compare the performance of image analysis for predicting breast cancer using two distinct regression models and to evaluate the usefulness of incorporating clinical and demographic data (CDD) into the image analysis in order to improve the diagnosis of breast cancer. Methods This study included 139 solid masses from 139 patients who underwent a ultrasonography-guided core biopsy and had available CDD between June 2009 and April 2010. Three breast radiologists retrospectively reviewed 139 breast masses and described each lesion using the Breast Imaging Reporting and Data System (BI-RADS) lexicon. We applied and compared two regression methods-stepwise logistic (SL) regression and logistic least absolute shrinkage and selection operator (LASSO) regression-in which the BI-RADS descriptors and CDD were used as covariates. We investigated the performances of these regression methods and the agreement of radiologists in terms of test misclassification error and the area under the curve (AUC) of the tests. Results Logistic LASSO regression was superior (P<0.05) to SL regression, regardless of whether CDD was included in the covariates, in terms of test misclassification errors (0.234 vs. 0.253, without CDD; 0.196 vs. 0.258, with CDD) and AUC (0.785 vs. 0.759, without CDD; 0.873 vs. 0.735, with CDD). However, it was inferior (P<0.05) to the agreement of three radiologists in terms of test misclassification errors (0.234 vs. 0.168, without CDD; 0.196 vs. 0.088, with CDD) and the AUC without CDD (0.785 vs. 0.844, P<0.001), but was comparable to the AUC with CDD (0.873 vs. 0.880, P=0.141). Conclusion Logistic LASSO regression based on BI-RADS descriptors and CDD showed better performance than SL in predicting the presence of breast cancer. The use of CDD as a supplement to the BI-RADS descriptors significantly improved the prediction of breast cancer using logistic LASSO regression.
Collapse
Affiliation(s)
- Sun Mi Kim
- Department of Radiology, Seoul National University Bundang Hospital, Seoul National University, Seongnam, Korea
| | - Yongdai Kim
- Department of Statistics, Seoul National University, Seoul, Korea
| | - Kuhwan Jeong
- Department of Statistics, Seoul National University, Seoul, Korea
| | - Heeyeong Jeong
- Department of Health Promotion, Seoul National University Bundang Hospital, Seongnam, Korea
| | - Jiyoung Kim
- Department of Radiology, Seoul National University Bundang Hospital, Seoul National University, Seongnam, Korea
| |
Collapse
|
11
|
Faradmal J, Soltanian AR, Roshanaei G, Khodabakhshi R, Kasaeian A. Comparison of the performance of log-logistic regression and artificial neural networks for predicting breast cancer relapse. Asian Pac J Cancer Prev 2015; 15:5883-8. [PMID: 25081718 DOI: 10.7314/apjcp.2014.15.14.5883] [Citation(s) in RCA: 17] [Impact Index Per Article: 1.9] [Reference Citation Analysis] [Abstract] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/10/2022] Open
Abstract
BACKGROUND Breast cancer is the most common cancers in female populations. The exact cause is not known, but is most likely to be a combination of genetic and environmental factors. Log-logistic model (LLM) is applied as a statistical method for predicting survival and it influencing factors. In recent decades, artificial neural network (ANN) models have been increasingly applied to predict survival data. The present research was conducted to compare log-logistic regression and artificial neural network models in prediction of breast cancer (BC) survival. MATERIALS AND METHODS A historical cohort study was established with 104 patients suffering from BC from 1997 to 2005. To compare the ANN and LLM in our setting, we used the estimated areas under the receiver-operating characteristic (ROC) curve (AUC) and integrated AUC (iAUC). The data were analyzed using R statistical software. RESULTS The AUC for the first, second and third years after diagnosis are 0.918, 0.780 and 0.800 in ANN, and 0.834, 0.733 and 0.616 in LLM, respectively. The mean AUC for ANN was statistically higher than that of the LLM (0.845 vs. 0.744). Hence, this study showed a significant difference between the performance in terms of prediction by ANN and LLM. CONCLUSIONS This study demonstrated that the ability of prediction with ANN was higher than with the LLM model. Thus, the use of ANN method for prediction of survival in field of breast cancer is suggested.
Collapse
Affiliation(s)
- Javad Faradmal
- Modeling of Noncommunicable Diseases Research Center, Department of Biostatistics and Epidemiology, School of Public Health, Hamadan University of Medical Sciences, Hamadan, Iran E-mail :
| | | | | | | | | |
Collapse
|
12
|
Lee T. Comparison of Breast Cancer Screening Results in Korean Middle-Aged Women: A Hospital-based Prospective Cohort Study. Osong Public Health Res Perspect 2013; 4:197-202. [PMID: 24159556 PMCID: PMC3767103 DOI: 10.1016/j.phrp.2013.06.002] [Citation(s) in RCA: 4] [Impact Index Per Article: 0.4] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/10/2013] [Revised: 06/17/2013] [Accepted: 06/18/2013] [Indexed: 12/14/2022] Open
Abstract
Objectives The aim of this hospital-based prospective study was to evaluate the diagnostic ability of breast cancer screening in Korean middle-aged women using age, ultrasonography, mammography, and magnification mammography, which are commonly used in most hospitals. Methods A total of 21 patents were examined using ultrasonography, mammography, and magnification mammography, and their data were prospectively analyzed from August 2011 to March 2013. All patients were divided into benign and malignant groups and the screening results were classified using the American College of Radiology Breast Imaging Reporting and Data System (BI-RADS). The final pathology report was used as the reference standard and the sensitivity and specificity of ultrasonography, mammography, and magnification mammography were evaluated using receiver-operating characteristics (ROC) analysis. Results The analysis included 21 patients who underwent biopsy. Among them, three (14.3%) were positive and 18 (85.7%) negative for breast cancer. The average age was 50.5 years (range = 38–61 years). The sensitivity was the same for ultrasonography and magnification mammography and the specificity of magnification mammography was higher than that of ultrasonography. The highest area under the ROC curve (AUC) was observed in the combination of age and magnification mammography (1.000) and the decreasing order of AUC in others was magnification mammography (0.833), ultrasonography (0.787), mammography (0.667), and age (0.648). Conclusions In Korean women, the diagnostic accuracy of magnification mammography was better than that of ultrasonography and mammography. The combination of age and magnification mammography increased the sensitivity and diagnostic accuracy.
Collapse
Affiliation(s)
- Taebum Lee
- Advanced Medical Device Research Center, Korea Electrotechnology Research Institute, Ansan, Korea
| |
Collapse
|