8
|
Yoon JH, Strand F, Baltzer PAT, Conant EF, Gilbert FJ, Lehman CD, Morris EA, Mullen LA, Nishikawa RM, Sharma N, Vejborg I, Moy L, Mann RM. Standalone AI for Breast Cancer Detection at Screening Digital Mammography and Digital Breast Tomosynthesis: A Systematic Review and Meta-Analysis. Radiology 2023; 307:e222639. [PMID: 37219445 PMCID: PMC10315526 DOI: 10.1148/radiol.222639] [Citation(s) in RCA: 27] [Impact Index Per Article: 27.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/14/2022] [Revised: 03/23/2023] [Accepted: 04/03/2023] [Indexed: 05/24/2023]
Abstract
Background There is considerable interest in the potential use of artificial intelligence (AI) systems in mammographic screening. However, it is essential to critically evaluate the performance of AI before it can become a modality used for independent mammographic interpretation. Purpose To evaluate the reported standalone performances of AI for interpretation of digital mammography and digital breast tomosynthesis (DBT). Materials and Methods A systematic search was conducted in PubMed, Google Scholar, Embase (Ovid), and Web of Science databases for studies published from January 2017 to June 2022. Sensitivity, specificity, and area under the receiver operating characteristic curve (AUC) values were reviewed. Study quality was assessed using the Quality Assessment of Diagnostic Accuracy Studies 2 and Comparative (QUADAS-2 and QUADAS-C, respectively). A random effects meta-analysis and meta-regression analysis were performed for overall studies and for different study types (reader studies vs historic cohort studies) and imaging techniques (digital mammography vs DBT). Results In total, 16 studies that include 1 108 328 examinations in 497 091 women were analyzed (six reader studies, seven historic cohort studies on digital mammography, and four studies on DBT). Pooled AUCs were significantly higher for standalone AI than radiologists in the six reader studies on digital mammography (0.87 vs 0.81, P = .002), but not for historic cohort studies (0.89 vs 0.96, P = .152). Four studies on DBT showed significantly higher AUCs in AI compared with radiologists (0.90 vs 0.79, P < .001). Higher sensitivity and lower specificity were seen for standalone AI compared with radiologists. Conclusion Standalone AI for screening digital mammography performed as well as or better than radiologists. Compared with digital mammography, there is an insufficient number of studies to assess the performance of AI systems in the interpretation of DBT screening examinations. © RSNA, 2023 Supplemental material is available for this article. See also the editorial by Scaranelo in this issue.
Collapse
Affiliation(s)
- Jung Hyun Yoon
- From the Department of Radiology, Severance Hospital, Research
Institute of Radiological Science, Yonsei University, College of Medicine, 50
Yonsei-ro, Seodaemun-gu, 03722 Seoul, Korea (J.H.Y.); Department of Oncology and
Pathology, Karolinska Institute, Stockholm, Sweden (F.S.); Department of
Radiology, Unit of Breast Imaging, Karolinska University Hospital, Stockholm,
Sweden (F.S.); Department of Biomedical Imaging and Image-guided Therapy,
Medical University of Vienna, Vienna, Austria (P.A.T.B.); Department of
Radiology, University of Pennsylvania, Philadelphia, Pa (E.F.C.); Department of
Radiology, University of Cambridge, Cambridge, UK (F.J.G.); Department of
Radiology, Harvard Medical School, Massachusetts General Hospital, Boston, Mass
(C.D.L.); Department of Radiology, University of California Davis, Davis, Calif
(E.A.M.); Department of Radiology, Breast Imaging Division, Johns Hopkins
Medicine, Baltimore, Md (L.A.M.); Department of Radiology, University of
Pittsburgh, UPMC Magee-Womens Hospital, Pittsburgh, Pa (R.M.N.); Department of
Radiology, St James Hospital, Leeds, UK (N.S.); Department of Breast
Examinations, Copenhagen University Hospital Herlev-Gentofte, Copenhagen,
Denmark (I.V.); Department of Radiology, Laura and Isaac Perlmutter Cancer
Center, Center for Biomedical Imaging, Center for Advanced Imaging Innovation
and Research, New York University Grossman School of Medicine, New York, NY
(L.M.); Department of Medical Imaging, Radboud University Medical Center,
Nijmegen, the Netherlands (R.M.M.); and Department of Radiology, Netherlands
Cancer Institute, Amsterdam, the Netherlands (R.M.M.)
| | - Fredrik Strand
- From the Department of Radiology, Severance Hospital, Research
Institute of Radiological Science, Yonsei University, College of Medicine, 50
Yonsei-ro, Seodaemun-gu, 03722 Seoul, Korea (J.H.Y.); Department of Oncology and
Pathology, Karolinska Institute, Stockholm, Sweden (F.S.); Department of
Radiology, Unit of Breast Imaging, Karolinska University Hospital, Stockholm,
Sweden (F.S.); Department of Biomedical Imaging and Image-guided Therapy,
Medical University of Vienna, Vienna, Austria (P.A.T.B.); Department of
Radiology, University of Pennsylvania, Philadelphia, Pa (E.F.C.); Department of
Radiology, University of Cambridge, Cambridge, UK (F.J.G.); Department of
Radiology, Harvard Medical School, Massachusetts General Hospital, Boston, Mass
(C.D.L.); Department of Radiology, University of California Davis, Davis, Calif
(E.A.M.); Department of Radiology, Breast Imaging Division, Johns Hopkins
Medicine, Baltimore, Md (L.A.M.); Department of Radiology, University of
Pittsburgh, UPMC Magee-Womens Hospital, Pittsburgh, Pa (R.M.N.); Department of
Radiology, St James Hospital, Leeds, UK (N.S.); Department of Breast
Examinations, Copenhagen University Hospital Herlev-Gentofte, Copenhagen,
Denmark (I.V.); Department of Radiology, Laura and Isaac Perlmutter Cancer
Center, Center for Biomedical Imaging, Center for Advanced Imaging Innovation
and Research, New York University Grossman School of Medicine, New York, NY
(L.M.); Department of Medical Imaging, Radboud University Medical Center,
Nijmegen, the Netherlands (R.M.M.); and Department of Radiology, Netherlands
Cancer Institute, Amsterdam, the Netherlands (R.M.M.)
| | - Pascal A. T. Baltzer
- From the Department of Radiology, Severance Hospital, Research
Institute of Radiological Science, Yonsei University, College of Medicine, 50
Yonsei-ro, Seodaemun-gu, 03722 Seoul, Korea (J.H.Y.); Department of Oncology and
Pathology, Karolinska Institute, Stockholm, Sweden (F.S.); Department of
Radiology, Unit of Breast Imaging, Karolinska University Hospital, Stockholm,
Sweden (F.S.); Department of Biomedical Imaging and Image-guided Therapy,
Medical University of Vienna, Vienna, Austria (P.A.T.B.); Department of
Radiology, University of Pennsylvania, Philadelphia, Pa (E.F.C.); Department of
Radiology, University of Cambridge, Cambridge, UK (F.J.G.); Department of
Radiology, Harvard Medical School, Massachusetts General Hospital, Boston, Mass
(C.D.L.); Department of Radiology, University of California Davis, Davis, Calif
(E.A.M.); Department of Radiology, Breast Imaging Division, Johns Hopkins
Medicine, Baltimore, Md (L.A.M.); Department of Radiology, University of
Pittsburgh, UPMC Magee-Womens Hospital, Pittsburgh, Pa (R.M.N.); Department of
Radiology, St James Hospital, Leeds, UK (N.S.); Department of Breast
Examinations, Copenhagen University Hospital Herlev-Gentofte, Copenhagen,
Denmark (I.V.); Department of Radiology, Laura and Isaac Perlmutter Cancer
Center, Center for Biomedical Imaging, Center for Advanced Imaging Innovation
and Research, New York University Grossman School of Medicine, New York, NY
(L.M.); Department of Medical Imaging, Radboud University Medical Center,
Nijmegen, the Netherlands (R.M.M.); and Department of Radiology, Netherlands
Cancer Institute, Amsterdam, the Netherlands (R.M.M.)
| | - Emily F. Conant
- From the Department of Radiology, Severance Hospital, Research
Institute of Radiological Science, Yonsei University, College of Medicine, 50
Yonsei-ro, Seodaemun-gu, 03722 Seoul, Korea (J.H.Y.); Department of Oncology and
Pathology, Karolinska Institute, Stockholm, Sweden (F.S.); Department of
Radiology, Unit of Breast Imaging, Karolinska University Hospital, Stockholm,
Sweden (F.S.); Department of Biomedical Imaging and Image-guided Therapy,
Medical University of Vienna, Vienna, Austria (P.A.T.B.); Department of
Radiology, University of Pennsylvania, Philadelphia, Pa (E.F.C.); Department of
Radiology, University of Cambridge, Cambridge, UK (F.J.G.); Department of
Radiology, Harvard Medical School, Massachusetts General Hospital, Boston, Mass
(C.D.L.); Department of Radiology, University of California Davis, Davis, Calif
(E.A.M.); Department of Radiology, Breast Imaging Division, Johns Hopkins
Medicine, Baltimore, Md (L.A.M.); Department of Radiology, University of
Pittsburgh, UPMC Magee-Womens Hospital, Pittsburgh, Pa (R.M.N.); Department of
Radiology, St James Hospital, Leeds, UK (N.S.); Department of Breast
Examinations, Copenhagen University Hospital Herlev-Gentofte, Copenhagen,
Denmark (I.V.); Department of Radiology, Laura and Isaac Perlmutter Cancer
Center, Center for Biomedical Imaging, Center for Advanced Imaging Innovation
and Research, New York University Grossman School of Medicine, New York, NY
(L.M.); Department of Medical Imaging, Radboud University Medical Center,
Nijmegen, the Netherlands (R.M.M.); and Department of Radiology, Netherlands
Cancer Institute, Amsterdam, the Netherlands (R.M.M.)
| | - Fiona J. Gilbert
- From the Department of Radiology, Severance Hospital, Research
Institute of Radiological Science, Yonsei University, College of Medicine, 50
Yonsei-ro, Seodaemun-gu, 03722 Seoul, Korea (J.H.Y.); Department of Oncology and
Pathology, Karolinska Institute, Stockholm, Sweden (F.S.); Department of
Radiology, Unit of Breast Imaging, Karolinska University Hospital, Stockholm,
Sweden (F.S.); Department of Biomedical Imaging and Image-guided Therapy,
Medical University of Vienna, Vienna, Austria (P.A.T.B.); Department of
Radiology, University of Pennsylvania, Philadelphia, Pa (E.F.C.); Department of
Radiology, University of Cambridge, Cambridge, UK (F.J.G.); Department of
Radiology, Harvard Medical School, Massachusetts General Hospital, Boston, Mass
(C.D.L.); Department of Radiology, University of California Davis, Davis, Calif
(E.A.M.); Department of Radiology, Breast Imaging Division, Johns Hopkins
Medicine, Baltimore, Md (L.A.M.); Department of Radiology, University of
Pittsburgh, UPMC Magee-Womens Hospital, Pittsburgh, Pa (R.M.N.); Department of
Radiology, St James Hospital, Leeds, UK (N.S.); Department of Breast
Examinations, Copenhagen University Hospital Herlev-Gentofte, Copenhagen,
Denmark (I.V.); Department of Radiology, Laura and Isaac Perlmutter Cancer
Center, Center for Biomedical Imaging, Center for Advanced Imaging Innovation
and Research, New York University Grossman School of Medicine, New York, NY
(L.M.); Department of Medical Imaging, Radboud University Medical Center,
Nijmegen, the Netherlands (R.M.M.); and Department of Radiology, Netherlands
Cancer Institute, Amsterdam, the Netherlands (R.M.M.)
| | - Constance D. Lehman
- From the Department of Radiology, Severance Hospital, Research
Institute of Radiological Science, Yonsei University, College of Medicine, 50
Yonsei-ro, Seodaemun-gu, 03722 Seoul, Korea (J.H.Y.); Department of Oncology and
Pathology, Karolinska Institute, Stockholm, Sweden (F.S.); Department of
Radiology, Unit of Breast Imaging, Karolinska University Hospital, Stockholm,
Sweden (F.S.); Department of Biomedical Imaging and Image-guided Therapy,
Medical University of Vienna, Vienna, Austria (P.A.T.B.); Department of
Radiology, University of Pennsylvania, Philadelphia, Pa (E.F.C.); Department of
Radiology, University of Cambridge, Cambridge, UK (F.J.G.); Department of
Radiology, Harvard Medical School, Massachusetts General Hospital, Boston, Mass
(C.D.L.); Department of Radiology, University of California Davis, Davis, Calif
(E.A.M.); Department of Radiology, Breast Imaging Division, Johns Hopkins
Medicine, Baltimore, Md (L.A.M.); Department of Radiology, University of
Pittsburgh, UPMC Magee-Womens Hospital, Pittsburgh, Pa (R.M.N.); Department of
Radiology, St James Hospital, Leeds, UK (N.S.); Department of Breast
Examinations, Copenhagen University Hospital Herlev-Gentofte, Copenhagen,
Denmark (I.V.); Department of Radiology, Laura and Isaac Perlmutter Cancer
Center, Center for Biomedical Imaging, Center for Advanced Imaging Innovation
and Research, New York University Grossman School of Medicine, New York, NY
(L.M.); Department of Medical Imaging, Radboud University Medical Center,
Nijmegen, the Netherlands (R.M.M.); and Department of Radiology, Netherlands
Cancer Institute, Amsterdam, the Netherlands (R.M.M.)
| | - Elizabeth A. Morris
- From the Department of Radiology, Severance Hospital, Research
Institute of Radiological Science, Yonsei University, College of Medicine, 50
Yonsei-ro, Seodaemun-gu, 03722 Seoul, Korea (J.H.Y.); Department of Oncology and
Pathology, Karolinska Institute, Stockholm, Sweden (F.S.); Department of
Radiology, Unit of Breast Imaging, Karolinska University Hospital, Stockholm,
Sweden (F.S.); Department of Biomedical Imaging and Image-guided Therapy,
Medical University of Vienna, Vienna, Austria (P.A.T.B.); Department of
Radiology, University of Pennsylvania, Philadelphia, Pa (E.F.C.); Department of
Radiology, University of Cambridge, Cambridge, UK (F.J.G.); Department of
Radiology, Harvard Medical School, Massachusetts General Hospital, Boston, Mass
(C.D.L.); Department of Radiology, University of California Davis, Davis, Calif
(E.A.M.); Department of Radiology, Breast Imaging Division, Johns Hopkins
Medicine, Baltimore, Md (L.A.M.); Department of Radiology, University of
Pittsburgh, UPMC Magee-Womens Hospital, Pittsburgh, Pa (R.M.N.); Department of
Radiology, St James Hospital, Leeds, UK (N.S.); Department of Breast
Examinations, Copenhagen University Hospital Herlev-Gentofte, Copenhagen,
Denmark (I.V.); Department of Radiology, Laura and Isaac Perlmutter Cancer
Center, Center for Biomedical Imaging, Center for Advanced Imaging Innovation
and Research, New York University Grossman School of Medicine, New York, NY
(L.M.); Department of Medical Imaging, Radboud University Medical Center,
Nijmegen, the Netherlands (R.M.M.); and Department of Radiology, Netherlands
Cancer Institute, Amsterdam, the Netherlands (R.M.M.)
| | - Lisa A. Mullen
- From the Department of Radiology, Severance Hospital, Research
Institute of Radiological Science, Yonsei University, College of Medicine, 50
Yonsei-ro, Seodaemun-gu, 03722 Seoul, Korea (J.H.Y.); Department of Oncology and
Pathology, Karolinska Institute, Stockholm, Sweden (F.S.); Department of
Radiology, Unit of Breast Imaging, Karolinska University Hospital, Stockholm,
Sweden (F.S.); Department of Biomedical Imaging and Image-guided Therapy,
Medical University of Vienna, Vienna, Austria (P.A.T.B.); Department of
Radiology, University of Pennsylvania, Philadelphia, Pa (E.F.C.); Department of
Radiology, University of Cambridge, Cambridge, UK (F.J.G.); Department of
Radiology, Harvard Medical School, Massachusetts General Hospital, Boston, Mass
(C.D.L.); Department of Radiology, University of California Davis, Davis, Calif
(E.A.M.); Department of Radiology, Breast Imaging Division, Johns Hopkins
Medicine, Baltimore, Md (L.A.M.); Department of Radiology, University of
Pittsburgh, UPMC Magee-Womens Hospital, Pittsburgh, Pa (R.M.N.); Department of
Radiology, St James Hospital, Leeds, UK (N.S.); Department of Breast
Examinations, Copenhagen University Hospital Herlev-Gentofte, Copenhagen,
Denmark (I.V.); Department of Radiology, Laura and Isaac Perlmutter Cancer
Center, Center for Biomedical Imaging, Center for Advanced Imaging Innovation
and Research, New York University Grossman School of Medicine, New York, NY
(L.M.); Department of Medical Imaging, Radboud University Medical Center,
Nijmegen, the Netherlands (R.M.M.); and Department of Radiology, Netherlands
Cancer Institute, Amsterdam, the Netherlands (R.M.M.)
| | - Robert M. Nishikawa
- From the Department of Radiology, Severance Hospital, Research
Institute of Radiological Science, Yonsei University, College of Medicine, 50
Yonsei-ro, Seodaemun-gu, 03722 Seoul, Korea (J.H.Y.); Department of Oncology and
Pathology, Karolinska Institute, Stockholm, Sweden (F.S.); Department of
Radiology, Unit of Breast Imaging, Karolinska University Hospital, Stockholm,
Sweden (F.S.); Department of Biomedical Imaging and Image-guided Therapy,
Medical University of Vienna, Vienna, Austria (P.A.T.B.); Department of
Radiology, University of Pennsylvania, Philadelphia, Pa (E.F.C.); Department of
Radiology, University of Cambridge, Cambridge, UK (F.J.G.); Department of
Radiology, Harvard Medical School, Massachusetts General Hospital, Boston, Mass
(C.D.L.); Department of Radiology, University of California Davis, Davis, Calif
(E.A.M.); Department of Radiology, Breast Imaging Division, Johns Hopkins
Medicine, Baltimore, Md (L.A.M.); Department of Radiology, University of
Pittsburgh, UPMC Magee-Womens Hospital, Pittsburgh, Pa (R.M.N.); Department of
Radiology, St James Hospital, Leeds, UK (N.S.); Department of Breast
Examinations, Copenhagen University Hospital Herlev-Gentofte, Copenhagen,
Denmark (I.V.); Department of Radiology, Laura and Isaac Perlmutter Cancer
Center, Center for Biomedical Imaging, Center for Advanced Imaging Innovation
and Research, New York University Grossman School of Medicine, New York, NY
(L.M.); Department of Medical Imaging, Radboud University Medical Center,
Nijmegen, the Netherlands (R.M.M.); and Department of Radiology, Netherlands
Cancer Institute, Amsterdam, the Netherlands (R.M.M.)
| | - Nisha Sharma
- From the Department of Radiology, Severance Hospital, Research
Institute of Radiological Science, Yonsei University, College of Medicine, 50
Yonsei-ro, Seodaemun-gu, 03722 Seoul, Korea (J.H.Y.); Department of Oncology and
Pathology, Karolinska Institute, Stockholm, Sweden (F.S.); Department of
Radiology, Unit of Breast Imaging, Karolinska University Hospital, Stockholm,
Sweden (F.S.); Department of Biomedical Imaging and Image-guided Therapy,
Medical University of Vienna, Vienna, Austria (P.A.T.B.); Department of
Radiology, University of Pennsylvania, Philadelphia, Pa (E.F.C.); Department of
Radiology, University of Cambridge, Cambridge, UK (F.J.G.); Department of
Radiology, Harvard Medical School, Massachusetts General Hospital, Boston, Mass
(C.D.L.); Department of Radiology, University of California Davis, Davis, Calif
(E.A.M.); Department of Radiology, Breast Imaging Division, Johns Hopkins
Medicine, Baltimore, Md (L.A.M.); Department of Radiology, University of
Pittsburgh, UPMC Magee-Womens Hospital, Pittsburgh, Pa (R.M.N.); Department of
Radiology, St James Hospital, Leeds, UK (N.S.); Department of Breast
Examinations, Copenhagen University Hospital Herlev-Gentofte, Copenhagen,
Denmark (I.V.); Department of Radiology, Laura and Isaac Perlmutter Cancer
Center, Center for Biomedical Imaging, Center for Advanced Imaging Innovation
and Research, New York University Grossman School of Medicine, New York, NY
(L.M.); Department of Medical Imaging, Radboud University Medical Center,
Nijmegen, the Netherlands (R.M.M.); and Department of Radiology, Netherlands
Cancer Institute, Amsterdam, the Netherlands (R.M.M.)
| | - Ilse Vejborg
- From the Department of Radiology, Severance Hospital, Research
Institute of Radiological Science, Yonsei University, College of Medicine, 50
Yonsei-ro, Seodaemun-gu, 03722 Seoul, Korea (J.H.Y.); Department of Oncology and
Pathology, Karolinska Institute, Stockholm, Sweden (F.S.); Department of
Radiology, Unit of Breast Imaging, Karolinska University Hospital, Stockholm,
Sweden (F.S.); Department of Biomedical Imaging and Image-guided Therapy,
Medical University of Vienna, Vienna, Austria (P.A.T.B.); Department of
Radiology, University of Pennsylvania, Philadelphia, Pa (E.F.C.); Department of
Radiology, University of Cambridge, Cambridge, UK (F.J.G.); Department of
Radiology, Harvard Medical School, Massachusetts General Hospital, Boston, Mass
(C.D.L.); Department of Radiology, University of California Davis, Davis, Calif
(E.A.M.); Department of Radiology, Breast Imaging Division, Johns Hopkins
Medicine, Baltimore, Md (L.A.M.); Department of Radiology, University of
Pittsburgh, UPMC Magee-Womens Hospital, Pittsburgh, Pa (R.M.N.); Department of
Radiology, St James Hospital, Leeds, UK (N.S.); Department of Breast
Examinations, Copenhagen University Hospital Herlev-Gentofte, Copenhagen,
Denmark (I.V.); Department of Radiology, Laura and Isaac Perlmutter Cancer
Center, Center for Biomedical Imaging, Center for Advanced Imaging Innovation
and Research, New York University Grossman School of Medicine, New York, NY
(L.M.); Department of Medical Imaging, Radboud University Medical Center,
Nijmegen, the Netherlands (R.M.M.); and Department of Radiology, Netherlands
Cancer Institute, Amsterdam, the Netherlands (R.M.M.)
| | | | | |
Collapse
|