1
|
Pedemonte S, Tsue T, Mombourquette B, Truong Vu YN, Matthews T, Morales Hoil R, Shah M, Ghare N, Zingman-Daniels N, Holley S, Appleton CM, Su J, Wahl RL. A Semiautonomous Deep Learning System to Reduce False Positives in Screening Mammography. Radiol Artif Intell 2024; 6:e230033. [PMID: 38597785 PMCID: PMC11140506 DOI: 10.1148/ryai.230033] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/02/2023] [Revised: 02/16/2024] [Accepted: 03/19/2024] [Indexed: 04/11/2024]
Abstract
Purpose To evaluate the ability of a semiautonomous artificial intelligence (AI) model to identify screening mammograms not suspicious for breast cancer and reduce the number of false-positive examinations. Materials and Methods The deep learning algorithm was trained using 123 248 two-dimensional digital mammograms (6161 cancers) and a retrospective study was performed on three nonoverlapping datasets of 14 831 screening mammography examinations (1026 cancers) from two U.S. institutions and one U.K. institution (2008-2017). The stand-alone performance of humans and AI was compared. Human plus AI performance was simulated to examine reductions in the cancer detection rate, number of examinations, false-positive callbacks, and benign biopsies. Metrics were adjusted to mimic the natural distribution of a screening population, and bootstrapped CIs and P values were calculated. Results Retrospective evaluation on all datasets showed minimal changes to the cancer detection rate with use of the AI device (noninferiority margin of 0.25 cancers per 1000 examinations: U.S. dataset 1, P = .02; U.S. dataset 2, P < .001; U.K. dataset, P < .001). On U.S. dataset 1 (11 592 mammograms; 101 cancers; 3810 female patients; mean age, 57.3 years ± 10.0 [SD]), the device reduced screening examinations requiring radiologist interpretation by 41.6% (95% CI: 40.6%, 42.4%; P < .001), diagnostic examinations callbacks by 31.1% (95% CI: 28.7%, 33.4%; P < .001), and benign needle biopsies by 7.4% (95% CI: 4.1%, 12.4%; P < .001). U.S. dataset 2 (1362 mammograms; 330 cancers; 1293 female patients; mean age, 55.4 years ± 10.5) was reduced by 19.5% (95% CI: 16.9%, 22.1%; P < .001), 11.9% (95% CI: 8.6%, 15.7%; P < .001), and 6.5% (95% CI: 0.0%, 19.0%; P = .08), respectively. The U.K. dataset (1877 mammograms; 595 cancers; 1491 female patients; mean age, 63.5 years ± 7.1) was reduced by 36.8% (95% CI: 34.4%, 39.7%; P < .001), 17.1% (95% CI: 5.9%, 30.1%: P < .001), and 5.9% (95% CI: 2.9%, 11.5%; P < .001), respectively. Conclusion This work demonstrates the potential of a semiautonomous breast cancer screening system to reduce false positives, unnecessary procedures, patient anxiety, and medical expenses. Keywords: Artificial Intelligence, Semiautonomous Deep Learning, Breast Cancer, Screening Mammography Supplemental material is available for this article. Published under a CC BY 4.0 license.
Collapse
Affiliation(s)
- Stefano Pedemonte
- From Whiterabbit.ai, 3930 Freedom Cir, Santa Clara, CA 95054 (S.P.,
T.T., B.M., Y.N.T.V., T.M., R.M.H., M.S., N.G., N.Z.D., J.S.); Onsite
Women's Health, Westfield, Mass (S.H.); SSM Health, St Louis, Mo
(C.M.A.); and Mallinckrodt Institute of Radiology, Washington University School
of Medicine, St Louis, Mo (R.L.W.)
| | - Trevor Tsue
- From Whiterabbit.ai, 3930 Freedom Cir, Santa Clara, CA 95054 (S.P.,
T.T., B.M., Y.N.T.V., T.M., R.M.H., M.S., N.G., N.Z.D., J.S.); Onsite
Women's Health, Westfield, Mass (S.H.); SSM Health, St Louis, Mo
(C.M.A.); and Mallinckrodt Institute of Radiology, Washington University School
of Medicine, St Louis, Mo (R.L.W.)
| | - Brent Mombourquette
- From Whiterabbit.ai, 3930 Freedom Cir, Santa Clara, CA 95054 (S.P.,
T.T., B.M., Y.N.T.V., T.M., R.M.H., M.S., N.G., N.Z.D., J.S.); Onsite
Women's Health, Westfield, Mass (S.H.); SSM Health, St Louis, Mo
(C.M.A.); and Mallinckrodt Institute of Radiology, Washington University School
of Medicine, St Louis, Mo (R.L.W.)
| | - Yen Nhi Truong Vu
- From Whiterabbit.ai, 3930 Freedom Cir, Santa Clara, CA 95054 (S.P.,
T.T., B.M., Y.N.T.V., T.M., R.M.H., M.S., N.G., N.Z.D., J.S.); Onsite
Women's Health, Westfield, Mass (S.H.); SSM Health, St Louis, Mo
(C.M.A.); and Mallinckrodt Institute of Radiology, Washington University School
of Medicine, St Louis, Mo (R.L.W.)
| | - Thomas Matthews
- From Whiterabbit.ai, 3930 Freedom Cir, Santa Clara, CA 95054 (S.P.,
T.T., B.M., Y.N.T.V., T.M., R.M.H., M.S., N.G., N.Z.D., J.S.); Onsite
Women's Health, Westfield, Mass (S.H.); SSM Health, St Louis, Mo
(C.M.A.); and Mallinckrodt Institute of Radiology, Washington University School
of Medicine, St Louis, Mo (R.L.W.)
| | - Rodrigo Morales Hoil
- From Whiterabbit.ai, 3930 Freedom Cir, Santa Clara, CA 95054 (S.P.,
T.T., B.M., Y.N.T.V., T.M., R.M.H., M.S., N.G., N.Z.D., J.S.); Onsite
Women's Health, Westfield, Mass (S.H.); SSM Health, St Louis, Mo
(C.M.A.); and Mallinckrodt Institute of Radiology, Washington University School
of Medicine, St Louis, Mo (R.L.W.)
| | - Meet Shah
- From Whiterabbit.ai, 3930 Freedom Cir, Santa Clara, CA 95054 (S.P.,
T.T., B.M., Y.N.T.V., T.M., R.M.H., M.S., N.G., N.Z.D., J.S.); Onsite
Women's Health, Westfield, Mass (S.H.); SSM Health, St Louis, Mo
(C.M.A.); and Mallinckrodt Institute of Radiology, Washington University School
of Medicine, St Louis, Mo (R.L.W.)
| | - Nikita Ghare
- From Whiterabbit.ai, 3930 Freedom Cir, Santa Clara, CA 95054 (S.P.,
T.T., B.M., Y.N.T.V., T.M., R.M.H., M.S., N.G., N.Z.D., J.S.); Onsite
Women's Health, Westfield, Mass (S.H.); SSM Health, St Louis, Mo
(C.M.A.); and Mallinckrodt Institute of Radiology, Washington University School
of Medicine, St Louis, Mo (R.L.W.)
| | - Naomi Zingman-Daniels
- From Whiterabbit.ai, 3930 Freedom Cir, Santa Clara, CA 95054 (S.P.,
T.T., B.M., Y.N.T.V., T.M., R.M.H., M.S., N.G., N.Z.D., J.S.); Onsite
Women's Health, Westfield, Mass (S.H.); SSM Health, St Louis, Mo
(C.M.A.); and Mallinckrodt Institute of Radiology, Washington University School
of Medicine, St Louis, Mo (R.L.W.)
| | - Susan Holley
- From Whiterabbit.ai, 3930 Freedom Cir, Santa Clara, CA 95054 (S.P.,
T.T., B.M., Y.N.T.V., T.M., R.M.H., M.S., N.G., N.Z.D., J.S.); Onsite
Women's Health, Westfield, Mass (S.H.); SSM Health, St Louis, Mo
(C.M.A.); and Mallinckrodt Institute of Radiology, Washington University School
of Medicine, St Louis, Mo (R.L.W.)
| | - Catherine M. Appleton
- From Whiterabbit.ai, 3930 Freedom Cir, Santa Clara, CA 95054 (S.P.,
T.T., B.M., Y.N.T.V., T.M., R.M.H., M.S., N.G., N.Z.D., J.S.); Onsite
Women's Health, Westfield, Mass (S.H.); SSM Health, St Louis, Mo
(C.M.A.); and Mallinckrodt Institute of Radiology, Washington University School
of Medicine, St Louis, Mo (R.L.W.)
| | - Jason Su
- From Whiterabbit.ai, 3930 Freedom Cir, Santa Clara, CA 95054 (S.P.,
T.T., B.M., Y.N.T.V., T.M., R.M.H., M.S., N.G., N.Z.D., J.S.); Onsite
Women's Health, Westfield, Mass (S.H.); SSM Health, St Louis, Mo
(C.M.A.); and Mallinckrodt Institute of Radiology, Washington University School
of Medicine, St Louis, Mo (R.L.W.)
| | - Richard L. Wahl
- From Whiterabbit.ai, 3930 Freedom Cir, Santa Clara, CA 95054 (S.P.,
T.T., B.M., Y.N.T.V., T.M., R.M.H., M.S., N.G., N.Z.D., J.S.); Onsite
Women's Health, Westfield, Mass (S.H.); SSM Health, St Louis, Mo
(C.M.A.); and Mallinckrodt Institute of Radiology, Washington University School
of Medicine, St Louis, Mo (R.L.W.)
| |
Collapse
|
2
|
Zizaan A, Idri A. Machine learning based Breast Cancer screening: trends, challenges, and opportunities. COMPUTER METHODS IN BIOMECHANICS AND BIOMEDICAL ENGINEERING: IMAGING & VISUALIZATION 2023. [DOI: 10.1080/21681163.2023.2172615] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 02/10/2023]
Affiliation(s)
- Asma Zizaan
- Mohammed VI Polytechnic University, Benguerir, Morocco
| | - Ali Idri
- Mohammed VI Polytechnic University, Benguerir, Morocco
- Software Project Management Research Team, ENSIAS, Mohammed V University, Rabat, Morocco
| |
Collapse
|
3
|
The Systematic Review of Artificial Intelligence Applications in Breast Cancer Diagnosis. Diagnostics (Basel) 2022; 13:diagnostics13010045. [PMID: 36611337 PMCID: PMC9818874 DOI: 10.3390/diagnostics13010045] [Citation(s) in RCA: 5] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/07/2022] [Revised: 12/16/2022] [Accepted: 12/17/2022] [Indexed: 12/28/2022] Open
Abstract
Several studies have demonstrated the value of artificial intelligence (AI) applications in breast cancer diagnosis. The systematic review of AI applications in breast cancer diagnosis includes several studies that compare breast cancer diagnosis and AI. However, they lack systematization, and each study appears to be conducted uniquely. The purpose and contributions of this study are to offer elaborative knowledge on the applications of AI in the diagnosis of breast cancer through citation analysis in order to categorize the main area of specialization that attracts the attention of the academic community, as well as thematic issue analysis to identify the species being researched in each category. In this study, a total number of 17,900 studies addressing breast cancer and AI published between 2012 and 2022 were obtained from these databases: IEEE, Embase: Excerpta Medica Database Guide-Ovid, PubMed, Springer, Web of Science, and Google Scholar. We applied inclusion and exclusion criteria to the search; 36 studies were identified. The vast majority of AI applications used classification models for the prediction of breast cancer. Howbeit, accuracy (99%) has the highest number of performance metrics, followed by specificity (98%) and area under the curve (0.95). Additionally, the Convolutional Neural Network (CNN) was the best model of choice in several studies. This study shows that the quantity and caliber of studies that use AI applications in breast cancer diagnosis will continue to rise annually. As a result, AI-based applications are viewed as a supplement to doctors' clinical reasoning, with the ultimate goal of providing quality healthcare that is both affordable and accessible to everyone worldwide.
Collapse
|
4
|
Díaz O, Rodríguez-Ruiz A, Gubern-Mérida A, Martí R, Chevalier M. Are artificial intelligence systems useful in breast cancer screening programmes? RADIOLOGIA 2021. [DOI: 10.1016/j.rxeng.2020.11.005] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/24/2022]
|
5
|
Díaz O, Rodríguez-Ruiz A, Gubern-Mérida A, Martí R, Chevalier M. Are artificial intelligence systems useful in breast cancer screening programs? RADIOLOGIA 2021; 63:236-244. [PMID: 33461750 DOI: 10.1016/j.rx.2020.11.006] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/16/2020] [Revised: 11/03/2020] [Accepted: 11/16/2020] [Indexed: 12/24/2022]
Abstract
Population-based breast cancer screening programs are efficacious in reducing the mortality due to breast cancer. These programs use mammography to screen the women who are invited to participate. Digital mammography makes it possible to develop computer-assisted diagnosis (CAD) systems that promise to reduce the workload of radiologists participating in screening programs. However, various studies have shown that CAD results in a high rate of false positive diagnoses. Systems based on artificial intelligence are being more widely implemented, and studies have shown that these systems have better diagnostic performance than traditional CAD systems. This article explains the fundamentals of artificial intelligence systems and an overview of possible applications of these systems within the framework of breast cancer screening programs.
Collapse
Affiliation(s)
- O Díaz
- Departamento de Matemáticas e Informática, Universidad de Barcelona, Barcelona, España
| | | | | | - R Martí
- Instituto de Visión Artificial y Robótica (VICOROB), Universitat de Girona, Girona, España
| | - M Chevalier
- Física Médica, Departamento de Radiología, Rehabilitación y Fisioterapia, Universidad Complutense de Madrid, Madrid, España; Instituto de Investigación Sanitaria, Hospital Clínico San Carlos, Madrid, España.
| |
Collapse
|
6
|
Love SM, Berg WA, Podilchuk C, López Aldrete AL, Gaxiola Mascareño AP, Pathicherikollamparambil K, Sankarasubramanian A, Eshraghi L, Mammone R. Palpable Breast Lump Triage by Minimally Trained Operators in Mexico Using Computer-Assisted Diagnosis and Low-Cost Ultrasound. J Glob Oncol 2019; 4:1-9. [PMID: 30156946 PMCID: PMC6223536 DOI: 10.1200/jgo.17.00222] [Citation(s) in RCA: 10] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/24/2022] Open
Abstract
Purpose In low- to middle-income countries (LMICs), most breast cancers present as palpable lumps; however, most palpable lumps are benign. We have developed artificial intelligence–based computer-assisted diagnosis (CADx) for an existing low-cost portable ultrasound system to triage which lumps need further evaluation and which are clearly benign. This pilot study was conducted to demonstrate that this approach can be successfully used by minimally trained health care workers in an LMIC country. Patients and Methods We recruited and trained three nonradiologist health care workers to participate in an institutional review board–approved, Health Insurance Portability and Accountability Act–compliant pilot study in Jalisco, Mexico, to determine whether they could use portable ultrasound (GE Vscan Dual Probe) to acquire images of palpable breast lumps of adequate quality for accurate computer analysis. Images from 32 women with 32 breast masses were then analyzed with a triage-CADx system, generating an output of benign or suspicious (biopsy recommended). Triage-CADx outputs were compared with radiologist readings. Results The nonradiologists were able to acquire adequate images. Triage by the CADx software was as accurate as assessment by specialist radiologists, with two (100%) of two cancers considered suspicious and 30 (100%) of 30 benign lesions classified as benign. Conclusion A portable ultrasound system with CADx software can be successfully used by first-level health care workers to triage palpable breast lumps. These results open up the possibility of implementing practical, cost-effective triage of palpable breast lumps, ensuring that scarce resources can be dedicated to suspicious lesions requiring further workup.
Collapse
Affiliation(s)
- Susan M Love
- Susan M. Love and Leah Eshraghi, Dr Susan Love Research Foundation, Encino, CA; Wendie A. Berg, Magee-Womens Hospital, University of Pittsburgh School of Medicine, Pittsburgh, PA; Christine Podilchuk, Krishnamohan Pathicherikollamparambil, Ananth Sankarasubramanian, and Richard Mammone, AI Strategy, Warren, NJ; Richard Mammone, Rutgers University, New Brunswick, NJ; and Ana Lilia López Aldrete and Aarón Patricio Gaxiola Mascareño, Instituto de Seguridad y Servicios Sociales de los Trabajadores del Estado Hospital Regional Valentin Gomez Farias, Jalisco, Mexico
| | - Wendie A Berg
- Susan M. Love and Leah Eshraghi, Dr Susan Love Research Foundation, Encino, CA; Wendie A. Berg, Magee-Womens Hospital, University of Pittsburgh School of Medicine, Pittsburgh, PA; Christine Podilchuk, Krishnamohan Pathicherikollamparambil, Ananth Sankarasubramanian, and Richard Mammone, AI Strategy, Warren, NJ; Richard Mammone, Rutgers University, New Brunswick, NJ; and Ana Lilia López Aldrete and Aarón Patricio Gaxiola Mascareño, Instituto de Seguridad y Servicios Sociales de los Trabajadores del Estado Hospital Regional Valentin Gomez Farias, Jalisco, Mexico
| | - Christine Podilchuk
- Susan M. Love and Leah Eshraghi, Dr Susan Love Research Foundation, Encino, CA; Wendie A. Berg, Magee-Womens Hospital, University of Pittsburgh School of Medicine, Pittsburgh, PA; Christine Podilchuk, Krishnamohan Pathicherikollamparambil, Ananth Sankarasubramanian, and Richard Mammone, AI Strategy, Warren, NJ; Richard Mammone, Rutgers University, New Brunswick, NJ; and Ana Lilia López Aldrete and Aarón Patricio Gaxiola Mascareño, Instituto de Seguridad y Servicios Sociales de los Trabajadores del Estado Hospital Regional Valentin Gomez Farias, Jalisco, Mexico
| | - Ana Lilia López Aldrete
- Susan M. Love and Leah Eshraghi, Dr Susan Love Research Foundation, Encino, CA; Wendie A. Berg, Magee-Womens Hospital, University of Pittsburgh School of Medicine, Pittsburgh, PA; Christine Podilchuk, Krishnamohan Pathicherikollamparambil, Ananth Sankarasubramanian, and Richard Mammone, AI Strategy, Warren, NJ; Richard Mammone, Rutgers University, New Brunswick, NJ; and Ana Lilia López Aldrete and Aarón Patricio Gaxiola Mascareño, Instituto de Seguridad y Servicios Sociales de los Trabajadores del Estado Hospital Regional Valentin Gomez Farias, Jalisco, Mexico
| | - Aarón Patricio Gaxiola Mascareño
- Susan M. Love and Leah Eshraghi, Dr Susan Love Research Foundation, Encino, CA; Wendie A. Berg, Magee-Womens Hospital, University of Pittsburgh School of Medicine, Pittsburgh, PA; Christine Podilchuk, Krishnamohan Pathicherikollamparambil, Ananth Sankarasubramanian, and Richard Mammone, AI Strategy, Warren, NJ; Richard Mammone, Rutgers University, New Brunswick, NJ; and Ana Lilia López Aldrete and Aarón Patricio Gaxiola Mascareño, Instituto de Seguridad y Servicios Sociales de los Trabajadores del Estado Hospital Regional Valentin Gomez Farias, Jalisco, Mexico
| | - Krishnamohan Pathicherikollamparambil
- Susan M. Love and Leah Eshraghi, Dr Susan Love Research Foundation, Encino, CA; Wendie A. Berg, Magee-Womens Hospital, University of Pittsburgh School of Medicine, Pittsburgh, PA; Christine Podilchuk, Krishnamohan Pathicherikollamparambil, Ananth Sankarasubramanian, and Richard Mammone, AI Strategy, Warren, NJ; Richard Mammone, Rutgers University, New Brunswick, NJ; and Ana Lilia López Aldrete and Aarón Patricio Gaxiola Mascareño, Instituto de Seguridad y Servicios Sociales de los Trabajadores del Estado Hospital Regional Valentin Gomez Farias, Jalisco, Mexico
| | - Ananth Sankarasubramanian
- Susan M. Love and Leah Eshraghi, Dr Susan Love Research Foundation, Encino, CA; Wendie A. Berg, Magee-Womens Hospital, University of Pittsburgh School of Medicine, Pittsburgh, PA; Christine Podilchuk, Krishnamohan Pathicherikollamparambil, Ananth Sankarasubramanian, and Richard Mammone, AI Strategy, Warren, NJ; Richard Mammone, Rutgers University, New Brunswick, NJ; and Ana Lilia López Aldrete and Aarón Patricio Gaxiola Mascareño, Instituto de Seguridad y Servicios Sociales de los Trabajadores del Estado Hospital Regional Valentin Gomez Farias, Jalisco, Mexico
| | - Leah Eshraghi
- Susan M. Love and Leah Eshraghi, Dr Susan Love Research Foundation, Encino, CA; Wendie A. Berg, Magee-Womens Hospital, University of Pittsburgh School of Medicine, Pittsburgh, PA; Christine Podilchuk, Krishnamohan Pathicherikollamparambil, Ananth Sankarasubramanian, and Richard Mammone, AI Strategy, Warren, NJ; Richard Mammone, Rutgers University, New Brunswick, NJ; and Ana Lilia López Aldrete and Aarón Patricio Gaxiola Mascareño, Instituto de Seguridad y Servicios Sociales de los Trabajadores del Estado Hospital Regional Valentin Gomez Farias, Jalisco, Mexico
| | - Richard Mammone
- Susan M. Love and Leah Eshraghi, Dr Susan Love Research Foundation, Encino, CA; Wendie A. Berg, Magee-Womens Hospital, University of Pittsburgh School of Medicine, Pittsburgh, PA; Christine Podilchuk, Krishnamohan Pathicherikollamparambil, Ananth Sankarasubramanian, and Richard Mammone, AI Strategy, Warren, NJ; Richard Mammone, Rutgers University, New Brunswick, NJ; and Ana Lilia López Aldrete and Aarón Patricio Gaxiola Mascareño, Instituto de Seguridad y Servicios Sociales de los Trabajadores del Estado Hospital Regional Valentin Gomez Farias, Jalisco, Mexico
| |
Collapse
|
7
|
Geras KJ, Mann RM, Moy L. Artificial Intelligence for Mammography and Digital Breast Tomosynthesis: Current Concepts and Future Perspectives. Radiology 2019; 293:246-259. [PMID: 31549948 DOI: 10.1148/radiol.2019182627] [Citation(s) in RCA: 146] [Impact Index Per Article: 29.2] [Reference Citation Analysis] [Abstract] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/12/2022]
Abstract
Although computer-aided diagnosis (CAD) is widely used in mammography, conventional CAD programs that use prompts to indicate potential cancers on the mammograms have not led to an improvement in diagnostic accuracy. Because of the advances in machine learning, especially with use of deep (multilayered) convolutional neural networks, artificial intelligence has undergone a transformation that has improved the quality of the predictions of the models. Recently, such deep learning algorithms have been applied to mammography and digital breast tomosynthesis (DBT). In this review, the authors explain how deep learning works in the context of mammography and DBT and define the important technical challenges. Subsequently, they discuss the current status and future perspectives of artificial intelligence-based clinical applications for mammography, DBT, and radiomics. Available algorithms are advanced and approach the performance of radiologists-especially for cancer detection and risk prediction at mammography. However, clinical validation is largely lacking, and it is not clear how the power of deep learning should be used to optimize practice. Further development of deep learning models is necessary for DBT, and this requires collection of larger databases. It is expected that deep learning will eventually have an important role in DBT, including the generation of synthetic images.
Collapse
Affiliation(s)
- Krzysztof J Geras
- From the Center for Biomedical Imaging (K.J.G., L.M.), Center for Data Science (K.J.G.), Center for Advanced Imaging Innovation and Research (L.M.), and Laura and Isaac Perlmutter Cancer Center (L.M.), New York University School of Medicine, 160 E 34th St, 3rd Floor, New York, NY 10016; Department of Radiology and Nuclear Medicine, Radboud University Medical Centre, Nijmegen, the Netherlands (R.M.M.); and Department of Radiology, the Netherlands Cancer Institute-Antoni van Leeuwenhoek Hospital, Amsterdam, the Netherlands (R.M.M.)
| | - Ritse M Mann
- From the Center for Biomedical Imaging (K.J.G., L.M.), Center for Data Science (K.J.G.), Center for Advanced Imaging Innovation and Research (L.M.), and Laura and Isaac Perlmutter Cancer Center (L.M.), New York University School of Medicine, 160 E 34th St, 3rd Floor, New York, NY 10016; Department of Radiology and Nuclear Medicine, Radboud University Medical Centre, Nijmegen, the Netherlands (R.M.M.); and Department of Radiology, the Netherlands Cancer Institute-Antoni van Leeuwenhoek Hospital, Amsterdam, the Netherlands (R.M.M.)
| | - Linda Moy
- From the Center for Biomedical Imaging (K.J.G., L.M.), Center for Data Science (K.J.G.), Center for Advanced Imaging Innovation and Research (L.M.), and Laura and Isaac Perlmutter Cancer Center (L.M.), New York University School of Medicine, 160 E 34th St, 3rd Floor, New York, NY 10016; Department of Radiology and Nuclear Medicine, Radboud University Medical Centre, Nijmegen, the Netherlands (R.M.M.); and Department of Radiology, the Netherlands Cancer Institute-Antoni van Leeuwenhoek Hospital, Amsterdam, the Netherlands (R.M.M.)
| |
Collapse
|
8
|
Ma G, Dou Y, Dang S, Yu N, Guo Y, Yang C, Lu S, Han D, Jin C. Influence of Monoenergetic Images at Different Energy Levels in Dual-Energy Spectral CT on the Accuracy of Computer-Aided Detection for Pulmonary Embolism. Acad Radiol 2019; 26:967-973. [PMID: 30803897 DOI: 10.1016/j.acra.2018.09.007] [Citation(s) in RCA: 7] [Impact Index Per Article: 1.4] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/06/2018] [Revised: 09/07/2018] [Accepted: 09/09/2018] [Indexed: 11/16/2022]
Abstract
PURPOSE To investigate the influence of monoenergetic images of different energy levels in spectral computed tomography (CT) on the accuracy of computer aided detection (CAD) for pulmonary embolism (PE). MATERIALS AND METHODS CT images of 20 PE patients who underwent spectral CT pulmonary angiography were retrospectively analyzed. Nine sets of monochromatic images from 40 to 80 keV at 5 keV interval were reconstructed and then independently analyzed for detecting PE using a commercially available CAD software. Two experienced radiologists reviewed all images and recorded the number of emboli manually, which was used as the reference standard. The CAD findings for the number of PE at different energies were compared with the reference standard to determine the number of true positives and false positives with CAD and to calculate the sensitivity and false positive rate at different energies. RESULT There were 120 true emboli. The total numbers of CAD-detected PE at 40-80 keV were 48, 67, 63, 87, 106, 115, 138, 157, and 226. Images at low energies had low sensitivities and low false positive rates; images at high energies had high sensitivities and high false positive rates. At 60 keV and 65 keV, CAD achieved sensitivity at 81.67% and 84.17%, respectively and false positive rate at 7.55% and 12.17%, respectively to provide the optimum combination of high sensitivity and low false positive rate. CONCLUSION Monochromatic images of different energies in dual-energy spectral CT affect the accuracy of CAD for PE. The combination of CAD with images at 60-65 keV provides the optimum combination of high sensitivity and low false positive rate in detecting PE.
Collapse
Affiliation(s)
- Guangming Ma
- Department of Radiology, The First Affiliated Hospital of Xi'an Jiaotong University, Yanta Western Road, Xi'an, Shaanxi 710061, China; Department of Radiology, Affiliated Hospital of Shaanxi University of Chinese Medicine, Xianyang, Shaanxi, China
| | - Yuequn Dou
- Department of Radiology, Affiliated Hospital of Shaanxi University of Chinese Medicine, Xianyang, Shaanxi, China
| | - Shan Dang
- Department of Radiology, Affiliated Hospital of Shaanxi University of Chinese Medicine, Xianyang, Shaanxi, China
| | - Nan Yu
- Department of Radiology, Affiliated Hospital of Shaanxi University of Chinese Medicine, Xianyang, Shaanxi, China
| | - Yanbing Guo
- Department of Radiology, Affiliated Hospital of Shaanxi University of Chinese Medicine, Xianyang, Shaanxi, China
| | - Chuangbo Yang
- Department of Radiology, Affiliated Hospital of Shaanxi University of Chinese Medicine, Xianyang, Shaanxi, China
| | - Shuanhong Lu
- Department of Radiology, Affiliated Hospital of Shaanxi University of Chinese Medicine, Xianyang, Shaanxi, China
| | - Dong Han
- Department of Radiology, Affiliated Hospital of Shaanxi University of Chinese Medicine, Xianyang, Shaanxi, China
| | - Chenwang Jin
- Department of Radiology, The First Affiliated Hospital of Xi'an Jiaotong University, Yanta Western Road, Xi'an, Shaanxi 710061, China; Russell H. Morgan Department of Radiology and Radiological Science, The Johns Hopkins University School of Medicine, Baltimore, Maryland.
| |
Collapse
|
9
|
Katzen J, Dodelzon K. A review of computer aided detection in mammography. Clin Imaging 2018; 52:305-309. [PMID: 30216858 DOI: 10.1016/j.clinimag.2018.08.014] [Citation(s) in RCA: 34] [Impact Index Per Article: 5.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/08/2018] [Revised: 08/13/2018] [Accepted: 08/16/2018] [Indexed: 01/23/2023]
Abstract
Breast screening with mammography is widely recognized as the most effective method of detecting early breast cancer and has consistently demonstrated a 20-40% decrease in mortality among screened women. Despite this, the sensitivity of mammography ranges between 70 and 90%. Computer aided detection (CAD) is an artificial intelligence (AI) technique that utilizes pattern recognition to highlight suspicious features on imaging and marks them for the radiologist to review and interpret. It aims to decrease oversights made by interpreting radiologists. Here we review the efficacy of CAD and potential future directions.
Collapse
Affiliation(s)
- Janine Katzen
- Department of Radiology, Weill Cornell Medicine, 425 E 61st Street, New York, NY 10065, United States of America.
| | - Katerina Dodelzon
- Department of Radiology, Weill Cornell Medicine, 425 E 61st Street, New York, NY 10065, United States of America
| |
Collapse
|
10
|
Mordang JJ, Gubern-Mérida A, Bria A, Tortorella F, den Heeten G, Karssemeijer N. Improving computer-aided detection assistance in breast cancer screening by removal of obviously false-positive findings. Med Phys 2017; 44:1390-1401. [DOI: 10.1002/mp.12152] [Citation(s) in RCA: 10] [Impact Index Per Article: 1.4] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/17/2016] [Revised: 01/11/2017] [Accepted: 02/01/2017] [Indexed: 11/06/2022] Open
Affiliation(s)
- Jan-Jurre Mordang
- Diagnostic Image Analysis Group; Department of Radiology and Nuclear Medicine; Radboud University Medical Center; Nijmegen The Netherlands
| | - Albert Gubern-Mérida
- Diagnostic Image Analysis Group; Department of Radiology and Nuclear Medicine; Radboud University Medical Center; Nijmegen The Netherlands
| | - Alessandro Bria
- Department of Electrical and Information Engineering; University of Cassino and Southern Lazio; Cassino Italy
| | - Francesco Tortorella
- Department of Electrical and Information Engineering; University of Cassino and Southern Lazio; Cassino Italy
| | - Gerard den Heeten
- Department of Radiology; Amsterdam Medical Center; Amsterdam The Netherlands
| | - Nico Karssemeijer
- Diagnostic Image Analysis Group; Department of Radiology and Nuclear Medicine; Radboud University Medical Center; Nijmegen The Netherlands
| |
Collapse
|
11
|
Mordang JJ, Gubern-Mérida A, den Heeten G, Karssemeijer N. Reducing false positives of microcalcification detection systems by removal of breast arterial calcifications. Med Phys 2016; 43:1676. [PMID: 27036566 DOI: 10.1118/1.4943376] [Citation(s) in RCA: 9] [Impact Index Per Article: 1.1] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/07/2022] Open
Abstract
PURPOSE In the past decades, computer-aided detection (CADe) systems have been developed to aid screening radiologists in the detection of malignant microcalcifications. These systems are useful to avoid perceptual oversights and can increase the radiologists' detection rate. However, due to the high number of false positives marked by these CADe systems, they are not yet suitable as an independent reader. Breast arterial calcifications (BACs) are one of the most frequent false positives marked by CADe systems. In this study, a method is proposed for the elimination of BACs as positive findings. Removal of these false positives will increase the performance of the CADe system in finding malignant microcalcifications. METHODS A multistage method is proposed for the removal of BAC findings. The first stage consists of a microcalcification candidate selection, segmentation and grouping of the microcalcifications, and classification to remove obvious false positives. In the second stage, a case-based selection is applied where cases are selected which contain BACs. In the final stage, BACs are removed from the selected cases. The BACs removal stage consists of a GentleBoost classifier trained on microcalcification features describing their shape, topology, and texture. Additionally, novel features are introduced to discriminate BACs from other positive findings. RESULTS The CADe system was evaluated with and without BACs removal. Here, both systems were applied on a validation set containing 1088 cases of which 95 cases contained malignant microcalcifications. After bootstrapping, free-response receiver operating characteristics and receiver operating characteristics analyses were carried out. Performance between the two systems was compared at 0.98 and 0.95 specificity. At a specificity of 0.98, the sensitivity increased from 37% to 52% and the sensitivity increased from 62% up to 76% at a specificity of 0.95. Partial areas under the curve in the specificity range of 0.8-1.0 were significantly different between the system without BACs removal and the system with BACs removal, 0.129 ± 0.009 versus 0.144 ± 0.008 (p<0.05), respectively. Additionally, the sensitivity at one false positive per 50 cases and one false positive per 25 cases increased as well, 37% versus 51% (p<0.05) and 58% versus 67% (p<0.05) sensitivity, respectively. Additionally, the CADe system with BACs removal reduces the number of false positives per case by 29% on average. The same sensitivity at one false positive per 50 cases in the CADe system without BACs removal can be achieved at one false positive per 80 cases in the CADe system with BACs removal. CONCLUSIONS By using dedicated algorithms to detect and remove breast arterial calcifications, the performance of CADe systems can be improved, in particular, at false positive rates representative for operating points used in screening.
Collapse
Affiliation(s)
- Jan-Jurre Mordang
- Diagnostic Image Analysis Group, Department of Radiology and Nuclear Medicine, Radboud University Medical Center, Nijmegen 6525 GA, The Netherlands
| | - Albert Gubern-Mérida
- Diagnostic Image Analysis Group, Department of Radiology and Nuclear Medicine, Radboud University Medical Center, Nijmegen 6525 GA, The Netherlands
| | - Gerard den Heeten
- The National Training Centre for Breast Cancer Screening, Nijmegen 6503 GJ, The Netherlands and Department of Radiology, Amsterdam Medical Center, Amsterdam 1100 DD, The Netherlands
| | - Nico Karssemeijer
- Diagnostic Image Analysis Group, Department of Radiology and Nuclear Medicine, Radboud University Medical Center, Nijmegen 6525 GA, The Netherlands
| |
Collapse
|
12
|
Tanaka R, Takamori M, Uchiyama Y, Nishikawa RM, Shiraishi J. Using breast radiographers' reports as a second opinion for radiologists' readings of microcalcifications in digital mammography. Br J Radiol 2014; 88:20140565. [PMID: 25536443 DOI: 10.1259/bjr.20140565] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.2] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/05/2022] Open
Abstract
OBJECTIVE The aim of this study was to investigate a practical method for incorporating radiographers' reports with radiologists' readings of digital mammograms. METHODS This simulation study was conducted using data from a free-response receiver operating characteristic observer study obtained with 75 cases (25 malignant, 25 benign and 25 normal cases) of digital mammograms. Each of the rating scores obtained by six breast radiographers was utilized as a second opinion for four radiologists' readings with the radiographers' reports. A logical "OR" operation with various criteria settings was simulated for deciding an appropriate method to select a radiographer's report in all combinations of radiologists and radiographers. The average figure of merit (FOM) of the radiologists' performances was statistically analysed using a jackknife procedure (JAFROC) to verify the clinical utility of using radiographers' reports. RESULTS Potential improvement of the average FOM of the radiologists' performances for identifying malignant microcalcifications could be expected when using radiographers' reports as a second opinion. When the threshold value of 2.6 in Breast Imaging-Reporting and Data System (BI-RADS®) assessment was applied to adopt/reject a radiographer's report, FOMs of radiologists' performances were further improved. CONCLUSION When using breast radiographers' reports as a second opinion, radiologists' performances potentially improved when reading digital mammograms. It could be anticipated that radiologists' performances were improved further by setting a threshold value on the BI-RADS assessment provided by the radiographers. ADVANCES IN KNOWLEDGE For the effective use of a radiographer's report as a second opinion, radiographers' rating scores and its criteria setting for adoption/rejection would be necessary.
Collapse
Affiliation(s)
- R Tanaka
- 1 Department of Radiological Technology, School of Health Sciences, College of Medical, Pharmaceutical and Health Sciences, Kanazawa University, Kanazawa, Japan
| | | | | | | | | |
Collapse
|
13
|
Improved Accuracy of Pulmonary Embolism Computer-Aided Detection Using Iterative Reconstruction Compared With Filtered Back Projection. AJR Am J Roentgenol 2014; 203:763-71. [DOI: 10.2214/ajr.13.11838] [Citation(s) in RCA: 20] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/18/2022]
|
14
|
Cole EB, Zhang Z, Marques HS, Edward Hendrick R, Yaffe MJ, Pisano ED. Impact of computer-aided detection systems on radiologist accuracy with digital mammography. AJR Am J Roentgenol 2014; 203:909-16. [PMID: 25247960 PMCID: PMC4286296 DOI: 10.2214/ajr.12.10187] [Citation(s) in RCA: 44] [Impact Index Per Article: 4.4] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/18/2022]
Abstract
OBJECTIVE The purpose of this study was to assess the impact of computer-aided detection (CAD) systems on the performance of radiologists with digital mammograms acquired during the Digital Mammographic Imaging Screening Trial (DMIST). MATERIALS AND METHODS Only those DMIST cases with proven cancer status by biopsy or 1-year follow-up that had available digital images were included in this multireader, multicase ROC study. Two commercially available CAD systems for digital mammography were used: iCAD SecondLook, version 1.4; and R2 ImageChecker Cenova, version 1.0. Fourteen radiologists interpreted, without and with CAD, a set of 300 cases (150 cancer, 150 benign or normal) on the iCAD SecondLook system, and 15 radiologists interpreted a different set of 300 cases (150 cancer, 150 benign or normal) on the R2 ImageChecker Cenova system. RESULTS The average AUC was 0.71 (95% CI, 0.66-0.76) without and 0.72 (95% CI, 0.67-0.77) with the iCAD system (p = 0.07). Similarly, the average AUC was 0.71 (95% CI, 0.66-0.76) without and 0.72 (95% CI 0.67-0.77) with the R2 system (p = 0.08). Sensitivity and specificity differences without and with CAD for both systems also were not significant. CONCLUSION Radiologists in our studies rarely changed their diagnostic decisions after the addition of CAD. The application of CAD had no statistically significant effect on radiologist AUC, sensitivity, or specificity performance with digital mammograms from DMIST.
Collapse
Affiliation(s)
- Elodia B Cole
- 1 Department of Radiology and Radiological Science, Medical University of South Carolina, 96 Jonathan Lucas St, Ste 210, MSC 323, Charleston, SC 29425
| | | | | | | | | | | |
Collapse
|
15
|
Nishikawa RM, Gur D. CADe for early detection of breast cancer-current status and why we need to continue to explore new approaches. Acad Radiol 2014; 21:1320-1. [PMID: 25086951 DOI: 10.1016/j.acra.2014.05.018] [Citation(s) in RCA: 24] [Impact Index Per Article: 2.4] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/24/2014] [Revised: 04/24/2014] [Accepted: 05/14/2014] [Indexed: 02/06/2023]
Abstract
The authors describe where we are in terms of using computer-aided detection (CADe) systems during clinical mammographic interpretations, what are the issues that we face, and why they believe that, despite disappointment in terms of verified added value when it comes to detection of soft tissue abnormalities, we need to continue to explore new approaches to improving CADe-alone performance levels and, more important perhaps, new approaches to optimal communication of CADe-generated information.
Collapse
|
16
|
Buist DSM, Anderson ML, Smith RA, Carney PA, Miglioretti DL, Monsees BS, Sickles EA, Taplin SH, Geller BM, Yankaskas BC, Onega TL. Effect of radiologists' diagnostic work-up volume on interpretive performance. Radiology 2014; 273:351-64. [PMID: 24960110 DOI: 10.1148/radiol.14132806] [Citation(s) in RCA: 21] [Impact Index Per Article: 2.1] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/11/2022]
Abstract
PURPOSE To examine radiologists' screening performance in relation to the number of diagnostic work-ups performed after abnormal findings are discovered at screening mammography by the same radiologist or by different radiologists. MATERIALS AND METHODS In an institutional review board-approved HIPAA-compliant study, the authors linked 651 671 screening mammograms interpreted from 2002 to 2006 by 96 radiologists in the Breast Cancer Surveillance Consortium to cancer registries (standard of reference) to evaluate the performance of screening mammography (sensitivity, false-positive rate [ FPR false-positive rate ], and cancer detection rate [ CDR cancer detection rate ]). Logistic regression was used to assess the association between the volume of recalled screening mammograms ("own" mammograms, where the radiologist who interpreted the diagnostic image was the same radiologist who had interpreted the screening image, and "any" mammograms, where the radiologist who interpreted the diagnostic image may or may not have been the radiologist who interpreted the screening image) and screening performance and whether the association between total annual volume and performance differed according to the volume of diagnostic work-up. RESULTS Annually, 38% of radiologists performed the diagnostic work-up for 25 or fewer of their own recalled screening mammograms, 24% performed the work-up for 0-50, and 39% performed the work-up for more than 50. For the work-up of recalled screening mammograms from any radiologist, 24% of radiologists performed the work-up for 0-50 mammograms, 32% performed the work-up for 51-125, and 44% performed the work-up for more than 125. With increasing numbers of radiologist work-ups for their own recalled mammograms, the sensitivity (P = .039), FPR false-positive rate (P = .004), and CDR cancer detection rate (P < .001) of screening mammography increased, yielding a stepped increase in women recalled per cancer detected from 17.4 for 25 or fewer mammograms to 24.6 for more than 50 mammograms. Increases in work-ups for any radiologist yielded significant increases in FPR false-positive rate (P = .011) and CDR cancer detection rate (P = .001) and a nonsignificant increase in sensitivity (P = .15). Radiologists with a lower annual volume of any work-ups had consistently lower FPR false-positive rate , sensitivity, and CDR cancer detection rate at all annual interpretive volumes. CONCLUSION These findings support the hypothesis that radiologists may improve their screening performance by performing the diagnostic work-up for their own recalled screening mammograms and directly receiving feedback afforded by means of the outcomes associated with their initial decision to recall. Arranging for radiologists to work up a minimum number of their own recalled cases could improve screening performance but would need systems to facilitate this workflow.
Collapse
Affiliation(s)
- Diana S M Buist
- From the Group Health Research Institute, Group Health Cooperative, 1730 Minor Ave, Suite 1600, Seattle, WA 98101 (D.S.M.B., M.L.A., D.L.M.); Cancer Control Science Department, American Cancer Society, Atlanta, Ga (R.A.S.); Departments of Family Medicine and Public Health and Preventive Medicine, Oregon Health & Science University, Portland, Ore (P.A.C.); Department of Biostatistics, University of Washington School of Public Health, Seattle, Wash (D.L.M.); Mallinckrodt Institute of Radiology, Washington University School of Medicine, St Louis, Mo (B.S.M.); Department of Radiology, University of California, San Francisco, Calif (E.A.S.); Division of Cancer Control and Population Science, Behavioral Research Program, National Cancer Institute, Rockville, Md (S.H.T.); Department of Family Medicine, University of Vermont, College of Medicine, Burlington, Vt (B.M.G.); Department of Radiology, University of North Carolina, Chapel Hill, NC (B.C.Y.); and Department of Community and Family Medicine, Dartmouth Institute for Health Policy and Clinical Practice, Geisel School of Medicine at Dartmouth, Norris Cotton Cancer Center, Lebanon, NH (T.L.O.)
| | | | | | | | | | | | | | | | | | | | | |
Collapse
|
17
|
Cascio D, Fauci F, Iacomi M, Raso G, Magro R, Castrogiovanni D, Filosto G, Ienzi R, Vasile MS. Computer-aided diagnosis in digital mammography: comparison of two commercial systems. ACTA ACUST UNITED AC 2014. [DOI: 10.2217/iim.13.68] [Citation(s) in RCA: 12] [Impact Index Per Article: 1.2] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/21/2022]
|
18
|
Bria A, Karssemeijer N, Tortorella F. Learning from unbalanced data: A cascade-based approach for detecting clustered microcalcifications. Med Image Anal 2014; 18:241-52. [DOI: 10.1016/j.media.2013.10.014] [Citation(s) in RCA: 57] [Impact Index Per Article: 5.7] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/26/2013] [Revised: 10/18/2013] [Accepted: 10/31/2013] [Indexed: 11/29/2022]
|