1
|
A New Computer-Aided Diagnosis System with Modified Genetic Feature Selection for BI-RADS Classification of Breast Masses in Mammograms. BIOMED RESEARCH INTERNATIONAL 2020; 2020:7695207. [PMID: 32462017 PMCID: PMC7238352 DOI: 10.1155/2020/7695207] [Citation(s) in RCA: 16] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 01/13/2020] [Revised: 03/19/2020] [Accepted: 04/02/2020] [Indexed: 11/17/2022]
Abstract
Mammography remains the most prevalent imaging tool for early breast cancer screening. The language used to describe abnormalities in mammographic reports is based on the Breast Imaging Reporting and Data System (BI-RADS). Assigning a correct BI-RADS category to each examined mammogram is a strenuous and challenging task for even experts. This paper proposes a new and effective computer-aided diagnosis (CAD) system to classify mammographic masses into four assessment categories in BI-RADS. The mass regions are first enhanced by means of histogram equalization and then semiautomatically segmented based on the region growing technique. A total of 130 handcrafted BI-RADS features are then extracted from the shape, margin, and density of each mass, together with the mass size and the patient's age, as mentioned in BI-RADS mammography. Then, a modified feature selection method based on the genetic algorithm (GA) is proposed to select the most clinically significant BI-RADS features. Finally, a back-propagation neural network (BPN) is employed for classification, and its accuracy is used as the fitness in GA. A set of 500 mammogram images from the digital database for screening mammography (DDSM) is used for evaluation. Our system achieves classification accuracy, positive predictive value, negative predictive value, and Matthews correlation coefficient of 84.5%, 84.4%, 94.8%, and 79.3%, respectively. To our best knowledge, this is the best current result for BI-RADS classification of breast masses in mammography, which makes the proposed system promising to support radiologists for deciding proper patient management based on the automatically assigned BI-RADS categories.
Collapse
|
2
|
Ahsen ME, Ayvaci MUS, Raghunathan S. When Algorithmic Predictions Use Human-Generated Data: A Bias-Aware Classification Algorithm for Breast Cancer Diagnosis. INFORMATION SYSTEMS RESEARCH 2019. [DOI: 10.1287/isre.2018.0789] [Citation(s) in RCA: 21] [Impact Index Per Article: 4.2] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/20/2022]
Affiliation(s)
- Mehmet Eren Ahsen
- Genetics and Genomic Sciences, Icahn School of Medicine at Mount Sinai, New York, New York 10029
| | | | | |
Collapse
|
3
|
Rezaianzadeh A, Sepandi M, Rahimikazerooni S. Assessment of Breast Cancer Risk in an Iranian Female Population Using Bayesian Networks with Varying Node Number. Asian Pac J Cancer Prev 2016; 17:4913-4916. [PMID: 28032495 PMCID: PMC5454695 DOI: 10.22034/apjcp.2016.17.11.4913] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.1] [Reference Citation Analysis] [Abstract] [Key Words] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/27/2022] Open
Abstract
Objective: As a source of information, medical data can feature hidden relationships. However, the high volume of datasets and complexity of decision-making in medicine introduce difficulties for analysis and interpretation and processing steps may be needed before the data can be used by clinicians in their work. This study focused on the use of Bayesian models with different numbers of nodes to aid clinicians in breast cancer risk estimation. Methods: Bayesian networks (BNs) with a retrospectively collected dataset including mammographic details, risk factor exposure, and clinical findings was assessed for prediction of the probability of breast cancer in individual patients. Area under the receiver-operating characteristic curve (AUC), accuracy, sensitivity, specificity, and positive and negative predictive values were used to evaluate discriminative performance. Result: A network incorporating selected features performed better (AUC = 0.94) than that incorporating all the features (AUC = 0.93). The results revealed no significant difference among 3 models regarding performance indices at the 5% significance level. Conclusion: BNs could effectively discriminate malignant from benign abnormalities and accurately predict the risk of breast cancer in individuals. Moreover, the overall performance of the 9-node BN was better, and due to the lower number of nodes it might be more readily be applied in clinical settings.
Collapse
Affiliation(s)
- Abbas Rezaianzadeh
- Colorectal Research Center, Shiraz University of Medical Sciences. Shiraz, Iran.
| | | | | |
Collapse
|
4
|
Ayer T, Chen Q, Burnside ES. Artificial neural networks in mammography interpretation and diagnostic decision making. COMPUTATIONAL AND MATHEMATICAL METHODS IN MEDICINE 2013; 2013:832509. [PMID: 23781276 PMCID: PMC3677609 DOI: 10.1155/2013/832509] [Citation(s) in RCA: 26] [Impact Index Per Article: 2.4] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 01/18/2013] [Accepted: 04/22/2013] [Indexed: 11/27/2022]
Abstract
Screening mammography is the most effective means for early detection of breast cancer. Although general rules for discriminating malignant and benign lesions exist, radiologists are unable to perfectly detect and classify all lesions as malignant and benign, for many reasons which include, but are not limited to, overlap of features that distinguish malignancy, difficulty in estimating disease risk, and variability in recommended management. When predictive variables are numerous and interact, ad hoc decision making strategies based on experience and memory may lead to systematic errors and variability in practice. The integration of computer models to help radiologists increase the accuracy of mammography examinations in diagnostic decision making has gained increasing attention in the last two decades. In this study, we provide an overview of one of the most commonly used models, artificial neural networks (ANNs), in mammography interpretation and diagnostic decision making and discuss important features in mammography interpretation. We conclude by discussing several common limitations of existing research on ANN-based detection and diagnostic models and provide possible future research directions.
Collapse
Affiliation(s)
- Turgay Ayer
- H. Milton Stewart School of Industrial and Systems Engineering, Georgia Institute of Technology, 765 Ferst Dr., Atlanta, GA 30332, USA.
| | | | | |
Collapse
|
5
|
Torres-Tabanera M, Cárdenas-Rebollo J, Villar-Castaño P, Sánchez-Gómez S, Cobo-Soler J, Montoro-Martos E, Sainz-Miranda M. Análisis del valor predictivo positivo de las subcategorías BI-RADS®4: resultados preliminares en 880 lesiones. RADIOLOGIA 2012; 54:520-31. [DOI: 10.1016/j.rx.2011.04.004] [Citation(s) in RCA: 14] [Impact Index Per Article: 1.2] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/19/2011] [Revised: 04/06/2011] [Accepted: 04/11/2011] [Indexed: 10/17/2022]
|
6
|
Torres-Tabanera M, Cárdenas-Rebollo J, Villar-Castaño P, Sánchez-Gómez S, Cobo-Soler J, Montoro-Martos E, Sainz-Miranda M. Analysis of the positive predictive value of the subcategories of BI-RADS® 4 lesions: Preliminary results in 880 lesions. ACTA ACUST UNITED AC 2012. [DOI: 10.1016/j.rxeng.2011.04.002] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.2] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/17/2022]
|
7
|
Carney PA, Cook AJ, Miglioretti DL, Feig SA, Bowles EA, Geller BM, Kerlikowske K, Kettler M, Onega T, Elmore JG. Use of clinical history affects accuracy of interpretive performance of screening mammography. J Clin Epidemiol 2012; 65:219-30. [PMID: 22000816 PMCID: PMC3253253 DOI: 10.1016/j.jclinepi.2011.06.010] [Citation(s) in RCA: 13] [Impact Index Per Article: 1.1] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/09/2010] [Revised: 06/15/2011] [Accepted: 06/18/2011] [Indexed: 10/16/2022]
Abstract
OBJECTIVE To examine how use of clinical history affects radiologist's interpretation of screening mammography. STUDY DESIGN AND SETTING Using a self-administered survey and actual interpretive performance, we examined associations between use of clinical history and sensitivity, false-positive rate, recall rate, and positive predictive value, after adjusting for relevant covariates using conditional logistic regression. RESULTS Of the 216 radiologists surveyed (63.4%), most radiologists reported usually or always using clinical history when interpreting screening mammography. Compared with radiologists who rarely use clinical history, radiologists who usually or always use it had a higher false-positive rate with younger women (10.7 vs. 9.7), denser breast tissue (10.1 for heterogeneously dense to 10.9 for extremely dense vs. 8.9 for fatty tissue), or longer screening intervals (> prior 5 years) (12.5 vs. 10.5). Effect of current hormone therapy (HT) use on false-positive rate was weaker among radiologists who use clinical history compared with those who did not (P=0.01), resulting in fewer false-positive examinations and a nonsignificant lower sensitivity (79.2 vs. 85.2) among HT users. CONCLUSION Interpretive performance appears to be influenced by patient age, breast density, screening interval, and HT use. This influence does not always result in improved interpretive performance.
Collapse
Affiliation(s)
- Patricia A Carney
- Department of Family Medicine, Oregon Health & Science University, Portland, OR 97239-3098, USA.
| | | | | | | | | | | | | | | | | | | |
Collapse
|
8
|
Ayer T, Ayvaci MUS, Liu ZX, Alagoz O, Burnside ES. Computer-aided diagnostic models in breast cancer screening. IMAGING IN MEDICINE 2010; 2:313-323. [PMID: 20835372 PMCID: PMC2936490 DOI: 10.2217/iim.10.24] [Citation(s) in RCA: 24] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 01/11/2023]
Abstract
Mammography is the most common modality for breast cancer detection and diagnosis and is often complemented by ultrasound and MRI. However, similarities between early signs of breast cancer and normal structures in these images make detection and diagnosis of breast cancer a difficult task. To aid physicians in detection and diagnosis, computer-aided detection and computer-aided diagnostic (CADx) models have been proposed. A large number of studies have been published for both computer-aided detection and CADx models in the last 20 years. The purpose of this article is to provide a comprehensive survey of the CADx models that have been proposed to aid in mammography, ultrasound and MRI interpretation. We summarize the noteworthy studies according to the screening modality they consider and describe the type of computer model, input data size, feature selection method, input feature type, reference standard and performance measures for each study. We also list the limitations of the existing CADx models and provide several possible future research directions.
Collapse
Affiliation(s)
- Turgay Ayer
- Industrial & Systems Engineering Department, University of Wisconsin, Madison, WI, USA
| | - Mehmet US Ayvaci
- Industrial & Systems Engineering Department, University of Wisconsin, Madison, WI, USA
| | - Ze Xiu Liu
- Industrial & Systems Engineering Department, University of Wisconsin, Madison, WI, USA
| | - Oguzhan Alagoz
- Industrial & Systems Engineering Department, University of Wisconsin, Madison, WI, USA
- Department of Population Health Sciences, University of Wisconsin, Madison, WI, USA
| | - Elizabeth S Burnside
- Industrial & Systems Engineering Department, University of Wisconsin, Madison, WI, USA
- Department of Biostatistics & Medical Informatics, University of Wisconsin, Madison, WI, USA
| |
Collapse
|
9
|
Cook AJ, Elmore JG, Miglioretti DL, Sickles EA, Aiello Bowles EJ, Cutter GR, Carney PA. Decreased accuracy in interpretation of community-based screening mammography for women with multiple clinical risk factors. J Clin Epidemiol 2009; 63:441-51. [PMID: 19744825 DOI: 10.1016/j.jclinepi.2009.06.008] [Citation(s) in RCA: 14] [Impact Index Per Article: 0.9] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/27/2008] [Revised: 06/10/2009] [Accepted: 06/27/2009] [Indexed: 11/28/2022]
Abstract
OBJECTIVE To assess the impact of women's breast cancer risk factors (use of hormone therapy, family history of breast cancer, previous breast biopsy) on radiologists' mammographic interpretive performance and whether the influence of risk factors varies according to radiologist characteristics. STUDY DESIGN AND SETTING Screening mammograms (n=638,947) performed from 1996 to 2005 by 134 radiologists from three Breast Cancer Surveillance Consortium registries was linked to cancer outcomes, radiologist surveys, and patient questionnaires. Interpretive performance measures were modeled using marginal and conditional logistic regression. RESULTS Having one or more clinical risk factors was associated with higher recall rates (1 vs. 0 risk factors: odds ratio [OR]=1.17, 95% confidence interval [CI]=1.15-1.19; > or = 2 vs. 0: OR=1.43, 95% CI=1.40-1.47) and lower specificity (1 vs. 0: OR=0.86 [95% CI=0.84-0.88]; > or = 2 vs. 0: OR=0.70 [95% CI=0.68-0.72]) without a corresponding improvement in sensitivity and only a small increase in positive predictive value (1 vs. 0: OR=1.08 [95% CI=0.99-1.19]; > or = 2 vs. 0: OR=1.12 [95% CI=0.99-1.26]). There was no indication that influence of risk factors varied by radiologist characteristics. CONCLUSION Women with clinical risk factors who undergo screening mammography are more likely recalled for false-positive evaluation without an associated increase in cancer detection. Radiologists and patients with risk factors should be aware of this increased risk of adverse screening events.
Collapse
Affiliation(s)
- Andrea J Cook
- Biostatistics Unit, Group Health Research Institute, Seattle, WA 98101, USA.
| | | | | | | | | | | | | |
Collapse
|
10
|
Elter M, Horsch A. CADx of mammographic masses and clustered microcalcifications: A review. Med Phys 2009; 36:2052-68. [PMID: 19610294 DOI: 10.1118/1.3121511] [Citation(s) in RCA: 141] [Impact Index Per Article: 9.4] [Reference Citation Analysis] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/07/2022] Open
Affiliation(s)
- Matthias Elter
- Fraunhofer Institute for Integrated Circuits, Am Wolfsmantel 33, 91058 Erlangen, Germany.
| | | |
Collapse
|
11
|
Jesneck JL, Lo JY, Baker JA. Breast mass lesions: computer-aided diagnosis models with mammographic and sonographic descriptors. Radiology 2007; 244:390-8. [PMID: 17562812 DOI: 10.1148/radiol.2442060712] [Citation(s) in RCA: 73] [Impact Index Per Article: 4.3] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/11/2022]
Abstract
PURPOSE To retrospectively develop and evaluate computer-aided diagnosis (CAD) models that include both mammographic and sonographic descriptors. MATERIALS AND METHODS Institutional review board approval was obtained for this HIPAA-compliant study. A waiver of informed consent was obtained. Mammographic and sonographic examinations were performed in 737 patients (age range, 17-87 years), which yielded 803 breast mass lesions (296 malignant, 507 benign). Radiologist-interpreted features from mammograms and sonograms were used as input features for linear discriminant analysis (LDA) and artificial neural network (ANN) models to differentiate benign from malignant lesions. An LDA with all the features was compared with an LDA with only stepwise-selected features. Classification performances were quantified by using receiver operating characteristic (ROC) analysis and were evaluated in a train, validate, and retest scheme. On the retest set, both LDAs were compared with radiologist assessment score of malignancy. RESULTS Both the LDA and ANN achieved high classification performance with cross validation (area under the ROC curve [A(z)] = 0.92 +/- 0.01 [standard deviation] and (0.90)A(z) = 0.54 +/- 0.08 for LDA, A(z) = 0.92 +/- 0.01 and (0.90)A(z) = 0.55 +/- 0.08 for ANN). Results of both models generalized well to the retest set, with no significant performance differences between the validate and retest sets (P > .1). On the retest set, there were no significant performance differences between LDA with all features and LDA with only the stepwise-selected features (P > .3) and between either LDA and radiologist assessment score (P > .2). CONCLUSION Results showed that combining mammographic and sonographic descriptors in a CAD model can result in high classification and generalization performance. On the retest set, LDA performance matched radiologist classification performance.
Collapse
Affiliation(s)
- Jonathan L Jesneck
- Department of Biomedical Engineering, Duke University Medical Center, 2424 Erwin Rd, Suite 302, Durham, NC 27705, USA.
| | | | | |
Collapse
|
12
|
Fischer EA, Lo JY, Markey MK. Bayesian networks of BI-RADStrade mark descriptors for breast lesion classification. CONFERENCE PROCEEDINGS : ... ANNUAL INTERNATIONAL CONFERENCE OF THE IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. ANNUAL CONFERENCE 2007; 2004:3031-4. [PMID: 17270917 DOI: 10.1109/iembs.2004.1403858] [Citation(s) in RCA: 14] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/06/2022]
Abstract
We investigated Bayesian network structure learning and probability estimation from mammographic feature data in order to classify breast lesions into different pathological categories. We compared the learned networks to naive Bayes classifiers, which are similar to the expert systems previously investigated for breast lesion classification. The learned network structures reflect the difference in the classification of biopsy outcome and the invasiveness of malignant lesions for breast masses and microcalcifications. The difference between masses and microcalcifications should be taken into consideration when interpreting systems for automatic pathological classification of breast lesions. The difference may also affect use of these systems for tasks such as estimating the sampling error of biopsy.
Collapse
Affiliation(s)
- E A Fischer
- Dept. of Biomed. Eng., Texas Univ., Austin, TX, USA
| | | | | |
Collapse
|
13
|
Markey MK, Tourassi GD, Margolis M, DeLong DM. Impact of missing data in evaluating artificial neural networks trained on complete data. Comput Biol Med 2006; 36:516-25. [PMID: 15893745 DOI: 10.1016/j.compbiomed.2005.02.001] [Citation(s) in RCA: 21] [Impact Index Per Article: 1.2] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/24/2004] [Accepted: 02/17/2005] [Indexed: 11/30/2022]
Abstract
This study investigated the impact of missing data in the evaluation of artificial neural network (ANN) models trained on complete data for the task of predicting whether breast lesions are benign or malignant from their mammographic Breast Imaging and Reporting Data System (BI-RADS) descriptors. A feed-forward, back-propagation ANN was tested with three methods for estimating the missing values. Similar results were achieved with a constraint satisfaction ANN, which can accommodate missing values without a separate estimation step. This empirical study highlights the need for additional research on developing robust clinical decision support systems for realistic environments in which key information may be unknown or inaccessible.
Collapse
Affiliation(s)
- Mia K Markey
- Biomedical Engineering Department, The University of Texas at Austin, 1 University Station, C0800, ENS617B, Austin, TX 78712, USA.
| | | | | | | |
Collapse
|
14
|
Gupta S, Chyn PF, Markey MK. Breast cancer CADx based on BI-RAds descriptors from two mammographic views. Med Phys 2006; 33:1810-7. [PMID: 16872088 DOI: 10.1118/1.2188080] [Citation(s) in RCA: 32] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/07/2022] Open
Abstract
In this study we compared the performance of computer aided diagnosis (CADx) algorithms based on Breast Imaging Reporting And Data System (BI-RADS) descriptors from one or two views. To select cases for the study with different mediolateral (MLO) and craniocaudal (CC) view descriptors, we assessed the agreement in BI-RADS lesion descriptors, BI-RADS assessment, and subtlety ratings for 1626 cases from the Digital Database for Screening Mammogrpahy (DDSM) using kappa statistics. We used 115 mass caseswith different descriptors for the two views to design linear discriminant analysis (LDA) based CADx algorithms. The CADx algorithms used BI-RADS descriptors and patient age as features. Thealgorithms based on BI-RADS descriptors from both the views performed marginally betterthan algorithms based on BI-RADS descriptors from a single view. A system that averaged theresults of two classifiers trained separately on the MLO and CC views displayed the best performance (Az=0.920 +/- 0.027). Thus, some improvement in performance of BI-RADS based CADx algorithms may be achieved by combining information from two mammographic views.
Collapse
Affiliation(s)
- Shalini Gupta
- Department of Electrical and Computer Engineering, The University of Texas at Austin, Austin, Texas 78712, USA
| | | | | |
Collapse
|
15
|
Bilska-Wolak AO, Floyd CE, Lo JY, Baker JA. Computer aid for decision to biopsy breast masses on mammography: validation on new cases. Acad Radiol 2005; 12:671-80. [PMID: 15935965 DOI: 10.1016/j.acra.2005.02.011] [Citation(s) in RCA: 17] [Impact Index Per Article: 0.9] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/10/2004] [Revised: 02/07/2005] [Accepted: 02/08/2005] [Indexed: 11/20/2022]
Abstract
RATIONALE AND OBJECTIVES The purpose of this study was to validate the performance of a previously developed computer aid for breast mass classification for mammography on a new, independent database of cases not used for algorithm development. MATERIALS AND METHODS A computer aid (classifier) based on the likelihood ratio (LRb) was previously developed on a database of 670 mass cases. The 670 cases (245 malignant) from one medical institution were described using 16 features from the American College of Radiology Breast Imaging-Reporting and Data System lexicon and patient history findings. A separate database of 151 (43 malignant) validation cases were collected that were previously unseen by the classifier. These new validation cases were evaluated by the classifier without retraining. Performance evaluation methods included Receiver Operating Characteristic (ROC), round-robin, and leave-one-out bootstrap sampling. RESULTS The performance of the classifier on the training data yielded an average ROC area of 0.90 +/- 0.02 and partial ROC area (0.90AUC) of 0.60 +/- 0.06. The exact nonparametric performance on the validation set of 151 cases yielded a ROC area of 0.88 and 0.90AUC of 0.57. Using a 100% sensitivity cutoff threshold established on the training data (100% negative predictive value), the classifier correctly identified 100% of the malignant masses in the validation test set, while potentially obviating 26% of the biopsies performed on benign masses. CONCLUSION The LRb classifier performed consistently on new data that was not used for classifier development. The LRb classifier shows promise as a potential aid in reducing the number of biopsies performed on benign masses.
Collapse
Affiliation(s)
- Anna O Bilska-Wolak
- Duke Advanced Imaging Laboratories, Department of Radiology, Duke University Medical Center, DUMC 2623, Durham, NC 27710, USA.
| | | | | | | |
Collapse
|
16
|
Bilska-Wolak AO, Floyd CE. Tolerance to missing data using a likelihood ratio based classifier for computer-aided classification of breast cancer. Phys Med Biol 2004; 49:4219-37. [PMID: 15509062 DOI: 10.1088/0031-9155/49/18/003] [Citation(s) in RCA: 6] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/11/2022]
Abstract
While mammography is a highly sensitive method for detecting breast tumours, its ability to differentiate between malignant and benign lesions is low, which may result in as many as 70% of unnecessary biopsies. The purpose of this study was to develop a highly specific computer-aided diagnosis algorithm to improve classification of mammographic masses. A classifier based on the likelihood ratio was developed to accommodate cases with missing data. Data for development included 671 biopsy cases (245 malignant), with biopsy-proved outcome. Sixteen features based on the BI-RADS lexicon and patient history had been recorded for the cases, with 1.3 +/- 1.1 missing feature values per case. Classifier evaluation methods included receiver operating characteristic and leave-one-out bootstrap sampling. The classifier achieved 32% specificity at 100% sensitivity on the 671 cases with 16 features that had missing values. Utilizing just the seven features present for all cases resulted in decreased performance at 100% sensitivity with average 19% specificity. No cases and no feature data were omitted during classifier development, showing that it is more beneficial to utilize cases with missing values than to discard incomplete cases that cannot be handled by many algorithms. Classification of mammographic masses was commendable at high sensitivity levels, indicating that benign cases could be potentially spared from biopsy.
Collapse
Affiliation(s)
- Anna O Bilska-Wolak
- Department of Biomedical Engineering, Duke University, 2623 DUMC, Durham, NC 27708, USA.
| | | |
Collapse
|
17
|
Bilska-Wolak AO, Floyd CE, Nolte LW, Lo JY. Application of likelihood ratio to classification of mammographic masses; performance comparison to case-based reasoning. Med Phys 2003; 30:949-58. [PMID: 12773004 DOI: 10.1118/1.1565339] [Citation(s) in RCA: 7] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/07/2022] Open
Abstract
The likelihood ratio (LR) is an optimal approach for deciding which of two alternate hypotheses best describes a given situation. We adopted this formalism for predicting whether biopsy results of mammographic masses will be benign or malignant, aiming to reduce the number of biopsies performed on benign lesions. We compared the performance of this LR-based algorithm (LRb) to a case-based reasoning (CBR) classifier, which provides a solution to a new problem using past similiar cases. Each classifier used mammographers' BI-RADS descriptions of mammographic masses as input. The database consisted of 646 biopsy-proven mammography cases. Performance was evaluated using Receiver Operating Characteristic (ROC) analysis, Round Robin sampling, and bootstrap. The ROC areas (AUC) for the LRb and CBR were 0.91+/- 0.01 and 0.92 +/- 0.01, respectively. The partial ROC area index (0.90AUC) was the same for both classifiers, 0.59 +/- 0.05. At a sensitivity of 98%, the CBR would spare 204 (49%) of benign lesions from biopsy; the LRb would spare 209 (51%) benign lesions. The performance of the two classifiers was very similar, with no statistical differences in AUC or 0.90AUC. Although the CBR and LRb originate from different fields of study, their implementations differ only in the estimation of the probability density functions (PDFs) of the feature distributions. The CBR performs this estimation implicitly, while using various similarity metrics. On the other hand, the estimation of the PDFs is specified explicitly in the LRb implementation. This difference in the estimation of the PDFs results in the very small difference in performance, and at 98% sensitivity, both classifiers would spare about half of the benign mammographic masses from biopsy. The CBR and LRb are equivalent methods in implementation and performance.
Collapse
Affiliation(s)
- Anna O Bilska-Wolak
- Department of Biomedical Engineering, Duke University, Durham, North Carolina 27708, USA.
| | | | | | | |
Collapse
|
18
|
Abstract
The purpose of this study was to identify and characterize clusters in a heterogeneous breast cancer computer-aided diagnosis database. Identification of subgroups within the database could help elucidate clinical trends and facilitate future model building. A self-organizing map (SOM) was used to identify clusters in a large (2258 cases), heterogeneous computer-aided diagnosis database based on mammographic findings (BI-RADS) and patient age. The resulting clusters were then characterized by their prototypes determined using a constraint satisfaction neural network (CSNN). The clusters showed logical separation of clinical subtypes such as architectural distortions, masses, and calcifications. Moreover, the broad categories of masses and calcifications were stratified into several clusters (seven for masses and three for calcifications). The percent of the cases that were malignant was notably different among the clusters (ranging from 6 to 83%). A feed-forward back-propagation artificial neural network (BP-ANN) was used to identify likely benign lesions that may be candidates for follow up rather than biopsy. The performance of the BP-ANN varied considerably across the clusters identified by the SOM. In particular, a cluster (#6) of mass cases (6% malignant) was identified that accounted for 79% of the recommendations for follow up that would have been made by the BP-ANN. A classification rule based on the profile of cluster #6 performed comparably to the BP-ANN, providing approximately 25% specificity at 98% sensitivity. This performance was demonstrated to generalize to a large (2177) set of cases held-out for model validation.
Collapse
|
19
|
Bilska-Wolak AO, Floyd CE. Development and evaluation of a case-based reasoning classifier for prediction of breast biopsy outcome with BI-RADS lexicon. Med Phys 2002; 29:2090-100. [PMID: 12349930 DOI: 10.1118/1.1501140] [Citation(s) in RCA: 28] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/07/2022] Open
Abstract
Approximately 70-85% of breast biopsies are performed on benign lesions. To reduce this high number of biopsies performed on benign lesions, a case-based reasoning (CBR) classifier was developed to predict biopsy results from BI-RADS findings. We used 1433 (931 benign) biopsy-proven mammographic cases. CBR similarity was defined using either the Hamming or Euclidean distance measure over case features. Ten features represented each case: calcification distribution, calcification morphology, calcification number, mass margin, mass shape, mass density, mass size, associated findings, special cases, and age. Performance was evaluated using Round Robin sampling, Receiver Operating Characteristic (ROC) analysis, and bootstrap. To determine the most influential features for the CBR, an exhaustive feature search was performed over all possible feature combinations (1022) and similarity thresholds. Influential features were defined as the most frequently occurring features in the feature subsets with the highest partial ROC areas (0.90AUC). For CBR with Hamming distance, the most influential features were found to be mass margin, calcification morphology, age, calcification distribution, calcification number, and mass shape, resulting in an 0.90AUC of 0.33. At 95% sensitivity, the Hamming CBR would spare from biopsy 34% of the benign lesions. At 98% sensitivity, the Hamming CBR would spare 27% benign lesions. For the CBR with Euclidean distance, the most influential feature subset consisted of mass margin, calcification morphology, age, mass density, and associated findings, resulting in 0.90AUC of 0.37. At 95% sensitivity, the Euclidean CBR would spare from biopsy 41% benign lesions. At 98% sensitivity, the Euclidean CBR would spare 27% benign lesions. The profile of cases spared by both distance measures at 98% sensitivity indicates that the CBR is a potentially useful diagnostic tool for the classification of mammographic lesions, by recommending short-term follow-up for likely benign lesions that is in agreement with final biopsy results and mammographer's intuition.
Collapse
Affiliation(s)
- Anna O Bilska-Wolak
- Department of Biomedical Engineering, Duke University, Durham, North Carolina 27710, USA.
| | | |
Collapse
|
20
|
Markey MK, Lo JY, Floyd CE. Differences between computer-aided diagnosis of breast masses and that of calcifications. Radiology 2002; 223:489-93. [PMID: 11997558 DOI: 10.1148/radiol.2232011257] [Citation(s) in RCA: 32] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/11/2023]
Abstract
PURPOSE To compare the performance of a computer-aided diagnosis (CAD) system for diagnosis of previously detected lesions, based on radiologist-extracted findings on masses and calcifications. MATERIALS AND METHODS A feed-forward, back-propagation artificial neural network (BP-ANN) was trained in a round-robin (leave-one-out) manner to predict biopsy outcome from mammographic findings (according to the Breast Imaging Reporting and Data System) and patient age. The BP-ANN was trained by using a large (>1,000 cases) heterogeneous data set containing masses and microcalcifications. The performances of the BP-ANN on masses and microcalcifications were compared with use of receiver operating characteristic analysis and a z test for uncorrelated samples. RESULTS The BP-ANN performed significantly better on masses than microcalcifications in terms of both the area under the receiver operating characteristic curve and the partial receiver operating characteristic area index. A similar difference in performance was observed with a second model (linear discriminant analysis) and also with a second data set from a similar institution. CONCLUSION Masses and calcifications should be considered separately when evaluating CAD systems for breast cancer diagnosis.
Collapse
Affiliation(s)
- Mia K Markey
- Department of Biomedical Engineering and Radiology, Digital Imaging Research Division, Duke University Medical Center, DUMC 3302, Durham, NC 27710, USA.
| | | | | |
Collapse
|
21
|
Markey MK, Lo JY, Vargas-Voracek R, Tourassi GD, Floyd CE. Perceptron error surface analysis: a case study in breast cancer diagnosis. Comput Biol Med 2002; 32:99-109. [PMID: 11879823 DOI: 10.1016/s0010-4825(01)00035-x] [Citation(s) in RCA: 9] [Impact Index Per Article: 0.4] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/29/2022]
Abstract
Perceptrons are typically trained to minimize mean square error (MSE). In computer-aided diagnosis (CAD), model performance is usually evaluated according to other more clinically relevant measures. The purpose of this study was to investigate the relationship between MSE and the area (A(z)) under the receiver operating characteristic (ROC) curve and the high-sensitivity partial ROC area ((0.90)A'(z)). A perceptron was used to predict lesion malignancy based on two mammographic findings and patient age. For each performance measure, the error surface in weight space was visualized. Comparison of the surfaces indicated that minimizing MSE tended to maximize A(z), but not (0.90)A'(z).
Collapse
Affiliation(s)
- Mia K Markey
- Department of Biomedical Engineering, Duke University, Durham, NC 27708, USA.
| | | | | | | | | |
Collapse
|
22
|
Lo JY, Markey MK, Baker JA, Floyd CE. Cross-institutional evaluation of BI-RADS predictive model for mammographic diagnosis of breast cancer. AJR Am J Roentgenol 2002; 178:457-63. [PMID: 11804918 DOI: 10.2214/ajr.178.2.1780457] [Citation(s) in RCA: 30] [Impact Index Per Article: 1.4] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/18/2022]
Abstract
OBJECTIVE Given a predictive model for identifying very likely benign breast lesions on the basis of Breast Imaging Reporting and Data System (BI-RADS) mammographic findings, this study evaluated the model's ability to generalize to a patient data set from a different institution. MATERIALS AND METHODS The artificial neural network model underwent three trials: it was optimized over 500 biopsy-proven lesions from Duke University Medical Center or "Duke," evaluated on 1,000 similar cases from the University of Pennsylvania Health System or "Penn," and reoptimized for Penn. RESULTS Trial A's Duke-only model yielded 98% sensitivity, 36% specificity, area index (A(z)) of 0.86, and partial A(z) of 0.51. The cross-institutional trial B yielded 96% sensitivity, 28% specificity, A(z) of 0.79, and partial A(z) of 0.28. The decreases were significant for both A(z) (p = 0.017) and partial A(z) (p < 0.001). In trial C, the model reoptimized for the Penn data yielded 96% sensitivity, 35% specificity, A(z) of 0.83, and partial A(z) of 0.32. There were no significant differences compared with trial B for specificity (p = 0.44) or partial A(z) (p = 0.46), suggesting that the Penn data were inherently more difficult to characterize. CONCLUSION The BI-RADS lexicon facilitated the cross-institutional test of a breast cancer prediction model. The model generalized reasonably well, but there were significant performance decreases. The cross-institutional performance was encouraging because it was not significantly different from that of a reoptimized model using the second data set at high sensitivities. This study indicates the need for further work to collect more data and to improve the robustness of the model.
Collapse
Affiliation(s)
- Joseph Y Lo
- Department of Radiology, Duke University Medical Center, DUMC-3302, Bryan Research Bldg., Rm. 161G, Durham, NC 27710, USA
| | | | | | | |
Collapse
|
23
|
Tourassi GD, Markey MK, Lo JY, Floyd CE. A neural network approach to breast cancer diagnosis as a constraint satisfaction problem. Med Phys 2001; 28:804-11. [PMID: 11393476 DOI: 10.1118/1.1367861] [Citation(s) in RCA: 23] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/07/2022] Open
Abstract
A constraint satisfaction neural network (CSNN) approach is proposed for breast cancer diagnosis using mammographic and patient history findings. Initially, the diagnostic decision to biopsy was formulated as a constraint satisfaction problem. Then, an associative memory type neural network was applied to solve the problem. The proposed network has a flexible, nonhierarchical architecture that allows it to operate not only as a predictive tool but also as an analysis tool for knowledge discovery of association rules. The CSNN was developed and evaluated using a database of 500 nonpalpable breast lesions with definitive histopathological diagnosis. The CSNN diagnostic performance was evaluated using receiver operating characteristic analysis (ROC). The results of the study showed that the CSNN ROC area index was 0.84+/-0.02. The CSNN predictive performance is competitive with that achieved by experienced radiologists and backpropagation artificial neural networks (BP-ANNs) presented before. Furthermore, the study illustrates how CSNN can be used as a knowledge discovery tool overcoming some of the well-known limitations of BP-ANNs.
Collapse
Affiliation(s)
- G D Tourassi
- Department of Radiology, Duke University Medical Center, Durham, North Carolina 27710, USA
| | | | | | | |
Collapse
|
24
|
Floyd CE, Lo JY, Tourassi GD. Case-based reasoning computer algorithm that uses mammographic findings for breast biopsy decisions. AJR Am J Roentgenol 2000; 175:1347-52. [PMID: 11044039 DOI: 10.2214/ajr.175.5.1751347] [Citation(s) in RCA: 39] [Impact Index Per Article: 1.6] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/18/2022]
Abstract
OBJECTIVE We present case-based reasoning computer software developed from mammographic findings to provide support for the clinical decision to perform biopsy of the breast. SUBJECTS AND METHODS The case-based reasoning system is designed to support the decision to perform biopsy in those patients who have suspicious findings on diagnostic mammography. Currently, between 66% and 90% of biopsies are performed on benign lesions. Our system is designed to help decrease the number of benign biopsies without missing malignancies. Clinicians interpret the mammograms using a standard reporting lexicon. The case-based reasoning system compares these findings with a database of cases with known outcomes (from biopsy) and returns the fraction of similar cases that were malignant. This malignancy fraction is an intuitive response that the clinician can then consider when making the decision regarding biopsy. RESULTS The system was evaluated using a round-robin sampling scheme and performed with an area under the receiver operating characteristic curve of 0.83, comparable with the performance of a neural network model. If only the cases returning a malignancy fraction of greater than a threshold of 0.10 are sent to biopsy, no malignancies would be missed, and the number of benign biopsies would be decreased by 25%. At a threshold of 0.21, 98%, of the malignancies would be biopsied, and the number of benign biopsies would be decreased by 41%. CONCLUSION This preliminary investigation indicates that the case-based reasoning approach to computer-aided diagnosis has the potential to improve the accuracy of breast cancer diagnosis on mammography.
Collapse
Affiliation(s)
- C E Floyd
- Department of Radiology, Duke University Medical Center, Box 2623, Durham, NC 27710, USA
| | | | | |
Collapse
|
25
|
Leichter I, Lederman R, Buchbinder S, Bamberger P, Novak B, Fields S. Optimizing parameters for computer-aided diagnosis of microcalcifications at mammography. Acad Radiol 2000; 7:406-12. [PMID: 10845399 DOI: 10.1016/s1076-6332(00)80380-3] [Citation(s) in RCA: 17] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/25/2022]
Abstract
RATIONALE AND OBJECTIVES The purpose of this study was to optimize selection of the mammographic features most useful in discriminating benign from malignant clustered microcalcifications. MATERIALS AND METHODS The computer-aided diagnosis (CAD) system automatically extracted from digitized mammograms 13 quantitative features characterizing microcalcification clusters. Archival cases (n = 134; patient age range, 31-77 years; mean age, 56.8 years) with known histopathologic results (79 malignant, 55 benign) were selected. Three radiologists at three facilities independently analyzed the microcalcifications by using the CAD system. Stepwise discriminant analysis selected the features best discriminating benign from malignant microcalcifications. A classification scheme was constructed on the basis of these optimized features, and its performance was evaluated by using receiver operating characteristic (ROC) analysis. RESULTS Six of the 13 variables extracted by the CAD system were selected by stepwise determinant analysis for generating the classification scheme, which yielded an ROC curve with an area (Az) of 0.98, specificity of 83.64%, positive predictive value of 89.53%, and accuracy of 91.79% for 98% sensitivity. When patient age was an additional variable, the scheme's performance improved, but this was not statistically significant (Az = 0.98). The ROC curve of the classifier (without age as an additional variable) yielded a high Az of 0.96 for patients younger than 50 years and an even higher (P < .02) Az of 0.99 for those 50 years or older. CONCLUSION Stepwise discriminant analysis optimized performance of a classification scheme for microcalcifications by selecting six optimized features. Scheme performance was significantly (P < .02) higher for women 50 years or older, but the addition of patient age as a variable did not produce a statistically significant increase in performance.
Collapse
Affiliation(s)
- I Leichter
- Department of Electro-Optics, Jerusalem College of Technology, Israel
| | | | | | | | | | | |
Collapse
|
26
|
Affiliation(s)
- P J Drew
- University of Hull Academic Surgical Unit, Castle Hill Hospital, United Kingdom
| | | |
Collapse
|