1
|
Sim Y, Chung MJ, Kotter E, Yune S, Kim M, Do S, Han K, Kim H, Yang S, Lee DJ, Choi BW. Deep Convolutional Neural Network-based Software Improves Radiologist Detection of Malignant Lung Nodules on Chest Radiographs. Radiology 2019; 294:199-209. [PMID: 31714194 DOI: 10.1148/radiol.2019182465] [Citation(s) in RCA: 144] [Impact Index Per Article: 28.8] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/25/2022]
Abstract
Background Multicenter studies are required to validate the added benefit of using deep convolutional neural network (DCNN) software for detecting malignant pulmonary nodules on chest radiographs. Purpose To compare the performance of radiologists in detecting malignant pulmonary nodules on chest radiographs when assisted by deep learning-based DCNN software with that of radiologists or DCNN software alone in a multicenter setting. Materials and Methods Investigators at four medical centers retrospectively identified 600 lung cancer-containing chest radiographs and 200 normal chest radiographs. Each radiograph with a lung cancer had at least one malignant nodule confirmed by CT and pathologic examination. Twelve radiologists from the four centers independently analyzed the chest radiographs and marked regions of interest. Commercially available deep learning-based computer-aided detection software separately trained, tested, and validated with 19 330 radiographs was used to find suspicious nodules. The radiologists then reviewed the images with the assistance of DCNN software. The sensitivity and number of false-positive findings per image of DCNN software, radiologists alone, and radiologists with the use of DCNN software were analyzed by using logistic regression and Poisson regression. Results The average sensitivity of radiologists improved (from 65.1% [1375 of 2112; 95% confidence interval {CI}: 62.0%, 68.1%] to 70.3% [1484 of 2112; 95% CI: 67.2%, 73.1%], P < .001) and the number of false-positive findings per radiograph declined (from 0.2 [488 of 2400; 95% CI: 0.18, 0.22] to 0.18 [422 of 2400; 95% CI: 0.16, 0.2], P < .001) when the radiologists re-reviewed radiographs with the DCNN software. For the 12 radiologists in this study, 104 of 2400 radiographs were positively changed (from false-negative to true-positive or from false-positive to true-negative) using the DCNN, while 56 of 2400 radiographs were changed negatively. Conclusion Radiologists had better performance with deep convolutional network software for the detection of malignant pulmonary nodules on chest radiographs than without. © RSNA, 2019 Online supplemental material is available for this article. See also the editorial by Jacobson in this issue.
Collapse
Affiliation(s)
- Yongsik Sim
- From the Department of Radiology, Research Institute of Radiological Science, Center for Clinical Imaging Data Science, Yonsei University College of Medicine, 50-1 Yonsei-ro, Seodaemun-gu, Seoul 03722, Republic of Korea (Y.S., K.H., B.W.C.); Department of Radiology, Samsung Medical Center, Sungkyunkwan University School of Medicine, Seoul, Republic of Korea (M.J.C.); Department of Radiology, University Medical Center Freiburg, Freiburg, Germany (E.K.); Department of Radiology, Massachusetts General Hospital and Harvard Medical School, Boston, Mass (S. Yune, M.K., S.D.); and Samsung Electronics, Suwon, Republic of Korea (H.K., S. Yang, D.J.L.)
| | - Myung Jin Chung
- From the Department of Radiology, Research Institute of Radiological Science, Center for Clinical Imaging Data Science, Yonsei University College of Medicine, 50-1 Yonsei-ro, Seodaemun-gu, Seoul 03722, Republic of Korea (Y.S., K.H., B.W.C.); Department of Radiology, Samsung Medical Center, Sungkyunkwan University School of Medicine, Seoul, Republic of Korea (M.J.C.); Department of Radiology, University Medical Center Freiburg, Freiburg, Germany (E.K.); Department of Radiology, Massachusetts General Hospital and Harvard Medical School, Boston, Mass (S. Yune, M.K., S.D.); and Samsung Electronics, Suwon, Republic of Korea (H.K., S. Yang, D.J.L.)
| | - Elmar Kotter
- From the Department of Radiology, Research Institute of Radiological Science, Center for Clinical Imaging Data Science, Yonsei University College of Medicine, 50-1 Yonsei-ro, Seodaemun-gu, Seoul 03722, Republic of Korea (Y.S., K.H., B.W.C.); Department of Radiology, Samsung Medical Center, Sungkyunkwan University School of Medicine, Seoul, Republic of Korea (M.J.C.); Department of Radiology, University Medical Center Freiburg, Freiburg, Germany (E.K.); Department of Radiology, Massachusetts General Hospital and Harvard Medical School, Boston, Mass (S. Yune, M.K., S.D.); and Samsung Electronics, Suwon, Republic of Korea (H.K., S. Yang, D.J.L.)
| | - Sehyo Yune
- From the Department of Radiology, Research Institute of Radiological Science, Center for Clinical Imaging Data Science, Yonsei University College of Medicine, 50-1 Yonsei-ro, Seodaemun-gu, Seoul 03722, Republic of Korea (Y.S., K.H., B.W.C.); Department of Radiology, Samsung Medical Center, Sungkyunkwan University School of Medicine, Seoul, Republic of Korea (M.J.C.); Department of Radiology, University Medical Center Freiburg, Freiburg, Germany (E.K.); Department of Radiology, Massachusetts General Hospital and Harvard Medical School, Boston, Mass (S. Yune, M.K., S.D.); and Samsung Electronics, Suwon, Republic of Korea (H.K., S. Yang, D.J.L.)
| | - Myeongchan Kim
- From the Department of Radiology, Research Institute of Radiological Science, Center for Clinical Imaging Data Science, Yonsei University College of Medicine, 50-1 Yonsei-ro, Seodaemun-gu, Seoul 03722, Republic of Korea (Y.S., K.H., B.W.C.); Department of Radiology, Samsung Medical Center, Sungkyunkwan University School of Medicine, Seoul, Republic of Korea (M.J.C.); Department of Radiology, University Medical Center Freiburg, Freiburg, Germany (E.K.); Department of Radiology, Massachusetts General Hospital and Harvard Medical School, Boston, Mass (S. Yune, M.K., S.D.); and Samsung Electronics, Suwon, Republic of Korea (H.K., S. Yang, D.J.L.)
| | - Synho Do
- From the Department of Radiology, Research Institute of Radiological Science, Center for Clinical Imaging Data Science, Yonsei University College of Medicine, 50-1 Yonsei-ro, Seodaemun-gu, Seoul 03722, Republic of Korea (Y.S., K.H., B.W.C.); Department of Radiology, Samsung Medical Center, Sungkyunkwan University School of Medicine, Seoul, Republic of Korea (M.J.C.); Department of Radiology, University Medical Center Freiburg, Freiburg, Germany (E.K.); Department of Radiology, Massachusetts General Hospital and Harvard Medical School, Boston, Mass (S. Yune, M.K., S.D.); and Samsung Electronics, Suwon, Republic of Korea (H.K., S. Yang, D.J.L.)
| | - Kyunghwa Han
- From the Department of Radiology, Research Institute of Radiological Science, Center for Clinical Imaging Data Science, Yonsei University College of Medicine, 50-1 Yonsei-ro, Seodaemun-gu, Seoul 03722, Republic of Korea (Y.S., K.H., B.W.C.); Department of Radiology, Samsung Medical Center, Sungkyunkwan University School of Medicine, Seoul, Republic of Korea (M.J.C.); Department of Radiology, University Medical Center Freiburg, Freiburg, Germany (E.K.); Department of Radiology, Massachusetts General Hospital and Harvard Medical School, Boston, Mass (S. Yune, M.K., S.D.); and Samsung Electronics, Suwon, Republic of Korea (H.K., S. Yang, D.J.L.)
| | - Hanmyoung Kim
- From the Department of Radiology, Research Institute of Radiological Science, Center for Clinical Imaging Data Science, Yonsei University College of Medicine, 50-1 Yonsei-ro, Seodaemun-gu, Seoul 03722, Republic of Korea (Y.S., K.H., B.W.C.); Department of Radiology, Samsung Medical Center, Sungkyunkwan University School of Medicine, Seoul, Republic of Korea (M.J.C.); Department of Radiology, University Medical Center Freiburg, Freiburg, Germany (E.K.); Department of Radiology, Massachusetts General Hospital and Harvard Medical School, Boston, Mass (S. Yune, M.K., S.D.); and Samsung Electronics, Suwon, Republic of Korea (H.K., S. Yang, D.J.L.)
| | - Seungwook Yang
- From the Department of Radiology, Research Institute of Radiological Science, Center for Clinical Imaging Data Science, Yonsei University College of Medicine, 50-1 Yonsei-ro, Seodaemun-gu, Seoul 03722, Republic of Korea (Y.S., K.H., B.W.C.); Department of Radiology, Samsung Medical Center, Sungkyunkwan University School of Medicine, Seoul, Republic of Korea (M.J.C.); Department of Radiology, University Medical Center Freiburg, Freiburg, Germany (E.K.); Department of Radiology, Massachusetts General Hospital and Harvard Medical School, Boston, Mass (S. Yune, M.K., S.D.); and Samsung Electronics, Suwon, Republic of Korea (H.K., S. Yang, D.J.L.)
| | - Dong-Jae Lee
- From the Department of Radiology, Research Institute of Radiological Science, Center for Clinical Imaging Data Science, Yonsei University College of Medicine, 50-1 Yonsei-ro, Seodaemun-gu, Seoul 03722, Republic of Korea (Y.S., K.H., B.W.C.); Department of Radiology, Samsung Medical Center, Sungkyunkwan University School of Medicine, Seoul, Republic of Korea (M.J.C.); Department of Radiology, University Medical Center Freiburg, Freiburg, Germany (E.K.); Department of Radiology, Massachusetts General Hospital and Harvard Medical School, Boston, Mass (S. Yune, M.K., S.D.); and Samsung Electronics, Suwon, Republic of Korea (H.K., S. Yang, D.J.L.)
| | - Byoung Wook Choi
- From the Department of Radiology, Research Institute of Radiological Science, Center for Clinical Imaging Data Science, Yonsei University College of Medicine, 50-1 Yonsei-ro, Seodaemun-gu, Seoul 03722, Republic of Korea (Y.S., K.H., B.W.C.); Department of Radiology, Samsung Medical Center, Sungkyunkwan University School of Medicine, Seoul, Republic of Korea (M.J.C.); Department of Radiology, University Medical Center Freiburg, Freiburg, Germany (E.K.); Department of Radiology, Massachusetts General Hospital and Harvard Medical School, Boston, Mass (S. Yune, M.K., S.D.); and Samsung Electronics, Suwon, Republic of Korea (H.K., S. Yang, D.J.L.)
| |
Collapse
|