1
|
Masci GM, Chassagnon G, Alifano M, Tlemsani C, Boudou-Rouquette P, La Torre G, Calinghen A, Canniff E, Fournel L, Revel MP. Performance of AI for preoperative CT assessment of lung metastases: Retrospective analysis of 167 patients. Eur J Radiol 2024; 179:111667. [PMID: 39121746 DOI: 10.1016/j.ejrad.2024.111667] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/12/2024] [Revised: 07/30/2024] [Accepted: 08/02/2024] [Indexed: 08/12/2024]
Abstract
OBJECTIVES To evaluate the performance of artificial intelligence (AI) in the preoperative detection of lung metastases on CT. MATERIALS AND METHODS Patients who underwent lung metastasectomy in our institution between 2016 and 2020 were enrolled, their preoperative CT reports having been performed before an AI solution (Veye Lung Nodules, version 3.9.2, Aidence) became available as a second reader in our department. All CT scans were retrospectively processed by AI. The sensitivities of unassisted radiologists (original CT radiology reports), AI reports alone and both combined were compared. Ground truth was established by a consensus reading of two radiologists, who analyzed whether the nodules mentioned in the pathology report were retrospectively visible on CT. Multivariate analysis was performed to identify nodule characteristics associated with detectability. RESULTS A total of 167 patients (men: 62.9 %; median age, 59 years [47-68]) with 475 resected nodules were included. AI detected an average of 4 nodules (0-17) per CT, of which 97 % were true nodules. The combination of radiologist plus AI (92.4 %) had significantly higher sensitivity than unassisted radiologists (80.4 %) (p < 0.001). In 27/57 (47.4 %) patients who had multiple preoperative CT scans, AI detected lung nodules earlier than the radiologist. Vascular contact was associated with non-detection by radiologists (OR:0.32[0.19, 0.54], p < 0.001), whilst the presence of cavitation (OR:0.26[0.13, 0.54], p < 0.001) or pleural contact (OR:0.10[0.04, 0.22], p < 0.001) was associated with non-detection by AI. CONCLUSION AI significantly increases the sensitivity of preoperative detection of lung metastases and enables earlier detection, with a significant potential benefit for patient management.
Collapse
Affiliation(s)
- Giorgio Maria Masci
- Radiology Department, Hôpital Cochin, AP-HP, 27 rue du Faubourg Saint-Jacques, 75014 Paris, France; Department of Radiological, Oncological and Pathological Sciences, Policlinico Umberto I, Università degli Studi di Roma La Sapienza, Viale del Policlinico 155, 00161 Rome, Italy
| | - Guillaume Chassagnon
- Radiology Department, Hôpital Cochin, AP-HP, 27 rue du Faubourg Saint-Jacques, 75014 Paris, France; Université de Paris Cité, 85 boulevard Saint-Germain, 75006 Paris, France
| | - Marco Alifano
- Université de Paris Cité, 85 boulevard Saint-Germain, 75006 Paris, France; Department of Thoracic Surgery, Hôpital Cochin, AP-HP, 27 rue du Faubourg Saint-Jacques, 75014 Paris, France
| | - Camille Tlemsani
- Université de Paris Cité, 85 boulevard Saint-Germain, 75006 Paris, France; Department of Medical Oncology, Hôpital Cochin, AP-HP, 27 rue du Faubourg Saint-Jacques, 75014 Paris, France
| | - Pascaline Boudou-Rouquette
- Department of Medical Oncology, Hôpital Cochin, AP-HP, 27 rue du Faubourg Saint-Jacques, 75014 Paris, France
| | - Giuseppe La Torre
- Department of Public Health and Infectious Diseases, Università degli Studi di Roma La Sapienza, Piazzale Aldo Moro 5, 00185 Rome, Italy
| | - Arvin Calinghen
- Radiology Department, Hôpital Cochin, AP-HP, 27 rue du Faubourg Saint-Jacques, 75014 Paris, France
| | - Emma Canniff
- Radiology Department, Hôpital Cochin, AP-HP, 27 rue du Faubourg Saint-Jacques, 75014 Paris, France
| | - Ludovic Fournel
- Université de Paris Cité, 85 boulevard Saint-Germain, 75006 Paris, France; Department of Thoracic Surgery, Hôpital Cochin, AP-HP, 27 rue du Faubourg Saint-Jacques, 75014 Paris, France
| | - Marie-Pierre Revel
- Radiology Department, Hôpital Cochin, AP-HP, 27 rue du Faubourg Saint-Jacques, 75014 Paris, France; Université de Paris Cité, 85 boulevard Saint-Germain, 75006 Paris, France.
| |
Collapse
|
2
|
Chen JL, Stumpe MC, Cohen E. Evolving From Discrete Molecular Data Integrations to Actionable Molecular Insights Within the Electronic Health Record. JCO Clin Cancer Inform 2024; 8:e2400011. [PMID: 38603638 DOI: 10.1200/cci.24.00011] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/15/2024] [Revised: 02/09/2024] [Accepted: 02/13/2024] [Indexed: 04/13/2024] Open
|
3
|
Takeshita Y, Onozawa S, Katase S, Shirakawa Y, Yamashita K, Shudo J, Nakanishi A, Akahori S, Yokoyama K. Evaluation of an artificial intelligence U-net algorithm for pulmonary nodule tracking on chest computed tomography images. J Int Med Res 2024; 52:3000605241230033. [PMID: 38321885 PMCID: PMC10851735 DOI: 10.1177/03000605241230033] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/01/2023] [Accepted: 01/13/2024] [Indexed: 02/08/2024] Open
Abstract
OBJECTIVES To apply image registration in the follow up of lung nodules and verify the feasibility of automatic tracking of lung nodules using an artificial intelligence (AI) method. METHODS For this retrospective, observational study, patients with pulmonary nodules 5-30 mm in diameter on computed tomography (CT) and who had at least six months follow-up were identified. Two radiologists defined a 'correct' cuboid circumscribing each nodule which was used to judge the success/failure of nodule tracking. An AI algorithm was applied in which a U-net type neural network model was trained to predict the deformation vector field between two examinations. When the estimated position was within a defined cuboid, the AI algorithm was judged a success. RESULTS In total, 49 lung nodules in 40 patients, with a total of 368 follow-up CT examinations were examined. The success rate for each time evaluation was 94% (345/368) and for 'nodule-by-nodule evaluation' was 78% (38/49). Reasons for a decrease in success rate were related to small nodules and those that decreased in size. CONCLUSION Automatic tracking of lung nodules is highly feasible.
Collapse
Affiliation(s)
- Yuhei Takeshita
- Department of Radiology, Kyorin University School of Medicine, 6-20-2 Shimorenjaku, Mitaka-shi, Tokyo, Japan 181-8611 Japan
| | - Shiro Onozawa
- Department of Radiology, Kyorin University School of Medicine, 6-20-2 Shimorenjaku, Mitaka-shi, Tokyo, Japan 181-8611 Japan
| | - Shichiro Katase
- Department of Radiology, Kyorin University School of Medicine, 6-20-2 Shimorenjaku, Mitaka-shi, Tokyo, Japan 181-8611 Japan
| | - Yuya Shirakawa
- Department of Radiology, Kyorin University School of Medicine, 6-20-2 Shimorenjaku, Mitaka-shi, Tokyo, Japan 181-8611 Japan
| | - Kouji Yamashita
- Department of Radiology, Kyorin University School of Medicine, 6-20-2 Shimorenjaku, Mitaka-shi, Tokyo, Japan 181-8611 Japan
| | - Jun Shudo
- Department of Radiology, Kyorin University School of Medicine, 6-20-2 Shimorenjaku, Mitaka-shi, Tokyo, Japan 181-8611 Japan
| | - Akihito Nakanishi
- Department of Radiology, Kyorin University School of Medicine, 6-20-2 Shimorenjaku, Mitaka-shi, Tokyo, Japan 181-8611 Japan
| | - Sadato Akahori
- FUJIFILM Corporation, 2-26-30 Nishiazabu, Minato-ku, Tokyo, Japan
| | - Kenichi Yokoyama
- Department of Radiology, Kyorin University School of Medicine, 6-20-2 Shimorenjaku, Mitaka-shi, Tokyo, Japan 181-8611 Japan
| |
Collapse
|
4
|
Rockall AG, Li X, Johnson N, Lavdas I, Santhakumaran S, Prevost AT, Punwani S, Goh V, Barwick TD, Bharwani N, Sandhu A, Sidhu H, Plumb A, Burn J, Fagan A, Wengert GJ, Koh DM, Reczko K, Dou Q, Warwick J, Liu X, Messiou C, Tunariu N, Boavida P, Soneji N, Johnston EW, Kelly-Morland C, De Paepe KN, Sokhi H, Wallitt K, Lakhani A, Russell J, Salib M, Vinnicombe S, Haq A, Aboagye EO, Taylor S, Glocker B. Development and Evaluation of Machine Learning in Whole-Body Magnetic Resonance Imaging for Detecting Metastases in Patients With Lung or Colon Cancer: A Diagnostic Test Accuracy Study. Invest Radiol 2023; 58:823-831. [PMID: 37358356 PMCID: PMC10662596 DOI: 10.1097/rli.0000000000000996] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/22/2023] [Accepted: 05/01/2023] [Indexed: 06/27/2023]
Abstract
OBJECTIVES Whole-body magnetic resonance imaging (WB-MRI) has been demonstrated to be efficient and cost-effective for cancer staging. The study aim was to develop a machine learning (ML) algorithm to improve radiologists' sensitivity and specificity for metastasis detection and reduce reading times. MATERIALS AND METHODS A retrospective analysis of 438 prospectively collected WB-MRI scans from multicenter Streamline studies (February 2013-September 2016) was undertaken. Disease sites were manually labeled using Streamline reference standard. Whole-body MRI scans were randomly allocated to training and testing sets. A model for malignant lesion detection was developed based on convolutional neural networks and a 2-stage training strategy. The final algorithm generated lesion probability heat maps. Using a concurrent reader paradigm, 25 radiologists (18 experienced, 7 inexperienced in WB-/MRI) were randomly allocated WB-MRI scans with or without ML support to detect malignant lesions over 2 or 3 reading rounds. Reads were undertaken in the setting of a diagnostic radiology reading room between November 2019 and March 2020. Reading times were recorded by a scribe. Prespecified analysis included sensitivity, specificity, interobserver agreement, and reading time of radiology readers to detect metastases with or without ML support. Reader performance for detection of the primary tumor was also evaluated. RESULTS Four hundred thirty-three evaluable WB-MRI scans were allocated to algorithm training (245) or radiology testing (50 patients with metastases, from primary 117 colon [n = 117] or lung [n = 71] cancer). Among a total 562 reads by experienced radiologists over 2 reading rounds, per-patient specificity was 86.2% (ML) and 87.7% (non-ML) (-1.5% difference; 95% confidence interval [CI], -6.4%, 3.5%; P = 0.39). Sensitivity was 66.0% (ML) and 70.0% (non-ML) (-4.0% difference; 95% CI, -13.5%, 5.5%; P = 0.344). Among 161 reads by inexperienced readers, per-patient specificity in both groups was 76.3% (0% difference; 95% CI, -15.0%, 15.0%; P = 0.613), with sensitivity of 73.3% (ML) and 60.0% (non-ML) (13.3% difference; 95% CI, -7.9%, 34.5%; P = 0.313). Per-site specificity was high (>90%) for all metastatic sites and experience levels. There was high sensitivity for the detection of primary tumors (lung cancer detection rate of 98.6% with and without ML [0.0% difference; 95% CI, -2.0%, 2.0%; P = 1.00], colon cancer detection rate of 89.0% with and 90.6% without ML [-1.7% difference; 95% CI, -5.6%, 2.2%; P = 0.65]). When combining all reads from rounds 1 and 2, reading times fell by 6.2% (95% CI, -22.8%, 10.0%) when using ML. Round 2 read-times fell by 32% (95% CI, 20.8%, 42.8%) compared with round 1. Within round 2, there was a significant decrease in read-time when using ML support, estimated as 286 seconds (or 11%) quicker ( P = 0.0281), using regression analysis to account for reader experience, read round, and tumor type. Interobserver variance suggests moderate agreement, Cohen κ = 0.64; 95% CI, 0.47, 0.81 (with ML), and Cohen κ = 0.66; 95% CI, 0.47, 0.81 (without ML). CONCLUSIONS There was no evidence of a significant difference in per-patient sensitivity and specificity for detecting metastases or the primary tumor using concurrent ML compared with standard WB-MRI. Radiology read-times with or without ML support fell for round 2 reads compared with round 1, suggesting that readers familiarized themselves with the study reading method. During the second reading round, there was a significant reduction in reading time when using ML support.
Collapse
|
5
|
Gao Y, Zhang B, Zhao D, Li S, Rong C, Sun M, Wu X. Automatic Segmentation and Radiomics for Identification and Activity Assessment of CTE Lesions in Crohn's Disease. Inflamm Bowel Dis 2023:izad285. [PMID: 38011673 DOI: 10.1093/ibd/izad285] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 06/14/2023] [Indexed: 11/29/2023]
Abstract
BACKGROUND The purpose of this article is to develop a deep learning automatic segmentation model for the segmentation of Crohn's disease (CD) lesions in computed tomography enterography (CTE) images. Additionally, the radiomics features extracted from the segmented CD lesions will be analyzed and multiple machine learning classifiers will be built to distinguish CD activity. METHODS This was a retrospective study with 2 sets of CTE image data. Segmentation datasets were used to establish nnU-Net neural network's automatic segmentation model. The classification dataset was processed using the automatic segmentation model to obtain segmentation results and extract radiomics features. The most optimal features were then selected to build 5 machine learning classifiers to distinguish CD activity. The performance of the automatic segmentation model was evaluated using the Dice similarity coefficient, while the performance of the machine learning classifier was evaluated using the area under the curve, sensitivity, specificity, and accuracy. RESULTS The segmentation dataset had 84 CTE examinations of CD patients (mean age 31 ± 13 years , 60 males), and the classification dataset had 193 (mean age 31 ± 12 years , 136 males). The deep learning segmentation model achieved a Dice similarity coefficient of 0.824 on the testing set. The logistic regression model showed the best performance among the 5 classifiers in the testing set, with an area under the curve, sensitivity, specificity, and accuracy of 0.862, 0.697, 0.840, and 0.759, respectively. CONCLUSION The automated segmentation model accurately segments CD lesions, and machine learning classifier distinguishes CD activity well. This method can assist radiologists in promptly and precisely evaluating CD activity.
Collapse
Affiliation(s)
- Yankun Gao
- Department of Radiology, The First Affiliated Hospital of Anhui Medical University, Hefei, China
| | - Bo Zhang
- Department of Radiology, The First Affiliated Hospital of Anhui Medical University, Hefei, China
| | - Dehan Zhao
- Department of Precision Machinery and Precision Instruments, University of Science and Technology of China, Hefei, China
| | - Shuai Li
- Department of Radiology, The First Affiliated Hospital of Anhui Medical University, Hefei, China
| | - Chang Rong
- Department of Radiology, The First Affiliated Hospital of Anhui Medical University, Hefei, China
| | - Mingzhai Sun
- Suzhou Institute for Advanced Research, University of Science and Technology of China, Suzhou, China
| | - Xingwang Wu
- Department of Radiology, The First Affiliated Hospital of Anhui Medical University, Hefei, China
| |
Collapse
|
6
|
Du Y, Greuter MJW, Prokop MW, de Bock GH. Pricing and cost-saving potential for deep-learning computer-aided lung nodule detection software in CT lung cancer screening. Insights Imaging 2023; 14:208. [PMID: 38010436 PMCID: PMC10682324 DOI: 10.1186/s13244-023-01561-z] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/24/2023] [Accepted: 10/28/2023] [Indexed: 11/29/2023] Open
Abstract
OBJECTIVE An increasing number of commercial deep learning computer-aided detection (DL-CAD) systems are available but their cost-saving potential is largely unknown. This study aimed to gain insight into appropriate pricing for DL-CAD in different reading modes to be cost-saving and to determine the potentially most cost-effective reading mode for lung cancer screening. METHODS In three representative settings, DL-CAD was evaluated as a concurrent, pre-screening, and second reader. Scoping review was performed to estimate radiologist reading time with and without DL-CAD. Hourly cost of radiologist time was collected for the USA (€196), UK (€127), and Poland (€45), and monetary equivalence of saved time was calculated. The minimum number of screening CTs to reach break-even was calculated for one-time investment of €51,616 for DL-CAD. RESULTS Mean reading time was 162 (95% CI: 111-212) seconds per case without DL-CAD, which decreased by 77 (95% CI: 47-107) and 104 (95% CI: 71-136) seconds for DL-CAD as concurrent and pre-screening reader, respectively, and increased by 33-41 s for DL-CAD as second reader. This translates into €1.0-4.3 per-case cost for concurrent reading and €0.8-5.7 for pre-screening reading in the USA, UK, and Poland. To achieve break-even with a one-time investment, the minimum number of CT scans was 12,300-53,600 for concurrent reader, and 9400-65,000 for pre-screening reader in the three countries. CONCLUSIONS Given current pricing, DL-CAD must be priced substantially below €6 in a pay-per-case setting or used in a high-workload environment to reach break-even in lung cancer screening. DL-CAD as pre-screening reader shows the largest potential to be cost-saving. CRITICAL RELEVANCE STATEMENT Deep-learning computer-aided lung nodule detection (DL-CAD) software must be priced substantially below 6 euro in a pay-per-case setting or must be used in high-workload environments with one-time investment in order to achieve break-even. DL-CAD as a pre-screening reader has the greatest cost savings potential. KEY POINTS • DL-CAD must be substantially below €6 in a pay-per-case setting to reach break-even. • DL-CAD must be used in a high-workload screening environment to achieve break-even. • DL-CAD as a pre-screening reader shows the largest potential to be cost-saving.
Collapse
Affiliation(s)
- Yihui Du
- Department of Epidemiology and Health Statistics, School of Public Health, Hangzhou Normal University, Hangzhou, China
- Department of Epidemiology, University Medical Center Groningen, University of Groningen, Groningen, The Netherlands
| | - Marcel J W Greuter
- Department of Radiology, University Medical Center Groningen, University of Groningen, Groningen, The Netherlands
| | - Mathias W Prokop
- Department of Radiology, University Medical Center Groningen, University of Groningen, Groningen, The Netherlands
- Department of Medical Imaging, Radboud University Medical Center, Nijmegen, The Netherlands
| | - Geertruida H de Bock
- Department of Epidemiology, University Medical Center Groningen, University of Groningen, Groningen, The Netherlands.
| |
Collapse
|
7
|
Hendrix W, Hendrix N, Scholten ET, Mourits M, Trap-de Jong J, Schalekamp S, Korst M, van Leuken M, van Ginneken B, Prokop M, Rutten M, Jacobs C. Deep learning for the detection of benign and malignant pulmonary nodules in non-screening chest CT scans. COMMUNICATIONS MEDICINE 2023; 3:156. [PMID: 37891360 PMCID: PMC10611755 DOI: 10.1038/s43856-023-00388-5] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/31/2023] [Accepted: 10/12/2023] [Indexed: 10/29/2023] Open
Abstract
BACKGROUND Outside a screening program, early-stage lung cancer is generally diagnosed after the detection of incidental nodules in clinically ordered chest CT scans. Despite the advances in artificial intelligence (AI) systems for lung cancer detection, clinical validation of these systems is lacking in a non-screening setting. METHOD We developed a deep learning-based AI system and assessed its performance for the detection of actionable benign nodules (requiring follow-up), small lung cancers, and pulmonary metastases in CT scans acquired in two Dutch hospitals (internal and external validation). A panel of five thoracic radiologists labeled all nodules, and two additional radiologists verified the nodule malignancy status and searched for any missed cancers using data from the national Netherlands Cancer Registry. The detection performance was evaluated by measuring the sensitivity at predefined false positive rates on a free receiver operating characteristic curve and was compared with the panel of radiologists. RESULTS On the external test set (100 scans from 100 patients), the sensitivity of the AI system for detecting benign nodules, primary lung cancers, and metastases is respectively 94.3% (82/87, 95% CI: 88.1-98.8%), 96.9% (31/32, 95% CI: 91.7-100%), and 92.0% (104/113, 95% CI: 88.5-95.5%) at a clinically acceptable operating point of 1 false positive per scan (FP/s). These sensitivities are comparable to or higher than the radiologists, albeit with a slightly higher FP/s (average difference of 0.6). CONCLUSIONS The AI system reliably detects benign and malignant pulmonary nodules in clinically indicated CT scans and can potentially assist radiologists in this setting.
Collapse
Affiliation(s)
- Ward Hendrix
- Diagnostic Imaging Analysis Group, Radiology and Nuclear Medicine Department, Radboud University Medical Center, Geert Grooteplein Zuid 10, 6525 GA, Nijmegen, The Netherlands
- Radiology Department, Jeroen Bosch Hospital, Henri Dunantstraat 1, 5223 GZ, 's-Hertogenbosch, The Netherlands
| | - Nils Hendrix
- Diagnostic Imaging Analysis Group, Radiology and Nuclear Medicine Department, Radboud University Medical Center, Geert Grooteplein Zuid 10, 6525 GA, Nijmegen, The Netherlands
- Radiology Department, Jeroen Bosch Hospital, Henri Dunantstraat 1, 5223 GZ, 's-Hertogenbosch, The Netherlands
- Jheronimus Academy of Data Science, Sint Janssingel 92, 5211 DA, 's-Hertogenbosch, The Netherlands
| | - Ernst T Scholten
- Diagnostic Imaging Analysis Group, Radiology and Nuclear Medicine Department, Radboud University Medical Center, Geert Grooteplein Zuid 10, 6525 GA, Nijmegen, The Netherlands
| | - Mariëlle Mourits
- Radiology Department, Canisius Wilhelmina Hospital, Weg door Jonkerbos 100, 6532 SZ, Nijmegen, The Netherlands
| | - Joline Trap-de Jong
- Radiology Department, St. Antonius Hospital, Koekoekslaan 1, 3435 CM, Nieuwegein, The Netherlands
| | - Steven Schalekamp
- Diagnostic Imaging Analysis Group, Radiology and Nuclear Medicine Department, Radboud University Medical Center, Geert Grooteplein Zuid 10, 6525 GA, Nijmegen, The Netherlands
| | - Mike Korst
- Radiology Department, Jeroen Bosch Hospital, Henri Dunantstraat 1, 5223 GZ, 's-Hertogenbosch, The Netherlands
| | - Maarten van Leuken
- Radiology Department, Canisius Wilhelmina Hospital, Weg door Jonkerbos 100, 6532 SZ, Nijmegen, The Netherlands
| | - Bram van Ginneken
- Diagnostic Imaging Analysis Group, Radiology and Nuclear Medicine Department, Radboud University Medical Center, Geert Grooteplein Zuid 10, 6525 GA, Nijmegen, The Netherlands
| | - Mathias Prokop
- Diagnostic Imaging Analysis Group, Radiology and Nuclear Medicine Department, Radboud University Medical Center, Geert Grooteplein Zuid 10, 6525 GA, Nijmegen, The Netherlands
- Radiology Department, University Medical Center Groningen, Hanzeplein 1, 9713 GZ, Groningen, The Netherlands
| | - Matthieu Rutten
- Diagnostic Imaging Analysis Group, Radiology and Nuclear Medicine Department, Radboud University Medical Center, Geert Grooteplein Zuid 10, 6525 GA, Nijmegen, The Netherlands
- Radiology Department, Jeroen Bosch Hospital, Henri Dunantstraat 1, 5223 GZ, 's-Hertogenbosch, The Netherlands
| | - Colin Jacobs
- Diagnostic Imaging Analysis Group, Radiology and Nuclear Medicine Department, Radboud University Medical Center, Geert Grooteplein Zuid 10, 6525 GA, Nijmegen, The Netherlands.
| |
Collapse
|
8
|
Thanoon MA, Zulkifley MA, Mohd Zainuri MAA, Abdani SR. A Review of Deep Learning Techniques for Lung Cancer Screening and Diagnosis Based on CT Images. Diagnostics (Basel) 2023; 13:2617. [PMID: 37627876 PMCID: PMC10453592 DOI: 10.3390/diagnostics13162617] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/30/2023] [Revised: 07/26/2023] [Accepted: 08/02/2023] [Indexed: 08/27/2023] Open
Abstract
One of the most common and deadly diseases in the world is lung cancer. Only early identification of lung cancer can increase a patient's probability of survival. A frequently used modality for the screening and diagnosis of lung cancer is computed tomography (CT) imaging, which provides a detailed scan of the lung. In line with the advancement of computer-assisted systems, deep learning techniques have been extensively explored to help in interpreting the CT images for lung cancer identification. Hence, the goal of this review is to provide a detailed review of the deep learning techniques that were developed for screening and diagnosing lung cancer. This review covers an overview of deep learning (DL) techniques, the suggested DL techniques for lung cancer applications, and the novelties of the reviewed methods. This review focuses on two main methodologies of deep learning in screening and diagnosing lung cancer, which are classification and segmentation methodologies. The advantages and shortcomings of current deep learning models will also be discussed. The resultant analysis demonstrates that there is a significant potential for deep learning methods to provide precise and effective computer-assisted lung cancer screening and diagnosis using CT scans. At the end of this review, a list of potential future works regarding improving the application of deep learning is provided to spearhead the advancement of computer-assisted lung cancer diagnosis systems.
Collapse
Affiliation(s)
- Mohammad A. Thanoon
- Department of Electrical, Electronic and Systems Engineering, Faculty of Engineering and Built Environment, University Kebangsaan Malaysia, Bangi 43600, Malaysia;
- System and Control Engineering Department, College of Electronics Engineering, Ninevah University, Mosul 41002, Iraq
| | - Mohd Asyraf Zulkifley
- Department of Electrical, Electronic and Systems Engineering, Faculty of Engineering and Built Environment, University Kebangsaan Malaysia, Bangi 43600, Malaysia;
| | - Muhammad Ammirrul Atiqi Mohd Zainuri
- Department of Electrical, Electronic and Systems Engineering, Faculty of Engineering and Built Environment, University Kebangsaan Malaysia, Bangi 43600, Malaysia;
| | - Siti Raihanah Abdani
- School of Computing Sciences, College of Computing, Informatics and Media, Universiti Teknologi MARA, Shah Alam 40450, Malaysia;
| |
Collapse
|
9
|
Nuutinen M, Leskelä RL. Systematic review of the performance evaluation of clinicians with or without the aid of machine learning clinical decision support system. HEALTH AND TECHNOLOGY 2023; 13:1-14. [PMID: 37363342 PMCID: PMC10262137 DOI: 10.1007/s12553-023-00763-1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/13/2023] [Accepted: 06/01/2023] [Indexed: 06/28/2023]
Abstract
Background For the adoption of machine learning clinical decision support systems (ML-CDSS) it is critical to understand the performance aid of the ML-CDSS. However, it is not trivial, how the performance aid should be evaluated. To design reliable performance evaluation study, both the knowledge from the practical framework of experimental study design and the understanding of domain specific design factors are required. Objective The aim of this review study was to form a practical framework and identify key design factors for experimental design in evaluating the performance of clinicians with or without the aid of ML-CDSS. Methods The study was based on published ML-CDSS performance evaluation studies. We systematically searched articles published between January 2016 and December 2022. From the articles we collected a set of design factors. Only the articles comparing the performance of clinicians with or without the aid of ML-CDSS using experimental study methods were considered. Results The identified key design factors for the practical framework of ML-CDSS experimental study design were performance measures, user interface, ground truth data and the selection of samples and participants. In addition, we identified the importance of randomization, crossover design and training and practice rounds. Previous studies had shortcomings in the rationale and documentation of choices regarding the number of participants and the duration of the experiment. Conclusion The design factors of ML-CDSS experimental study are interdependent and all factors must be considered in individual choices. Supplementary Information The online version contains supplementary material available at 10.1007/s12553-023-00763-1.
Collapse
Affiliation(s)
- Mikko Nuutinen
- Nordic Healthcare Group, Helsinki, Finland
- Haartman Institute, University of Helsinki, Helsinki, Finland
| | | |
Collapse
|
10
|
Ewals LJS, van der Wulp K, van den Borne BEEM, Pluyter JR, Jacobs I, Mavroeidis D, van der Sommen F, Nederend J. The Effects of Artificial Intelligence Assistance on the Radiologists' Assessment of Lung Nodules on CT Scans: A Systematic Review. J Clin Med 2023; 12:jcm12103536. [PMID: 37240643 DOI: 10.3390/jcm12103536] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/15/2023] [Revised: 04/19/2023] [Accepted: 05/16/2023] [Indexed: 05/28/2023] Open
Abstract
To reduce the number of missed or misdiagnosed lung nodules on CT scans by radiologists, many Artificial Intelligence (AI) algorithms have been developed. Some algorithms are currently being implemented in clinical practice, but the question is whether radiologists and patients really benefit from the use of these novel tools. This study aimed to review how AI assistance for lung nodule assessment on CT scans affects the performances of radiologists. We searched for studies that evaluated radiologists' performances in the detection or malignancy prediction of lung nodules with and without AI assistance. Concerning detection, radiologists achieved with AI assistance a higher sensitivity and AUC, while the specificity was slightly lower. Concerning malignancy prediction, radiologists achieved with AI assistance generally a higher sensitivity, specificity and AUC. The radiologists' workflows of using the AI assistance were often only described in limited detail in the papers. As recent studies showed improved performances of radiologists with AI assistance, AI assistance for lung nodule assessment holds great promise. To achieve added value of AI tools for lung nodule assessment in clinical practice, more research is required on the clinical validation of AI tools, impact on follow-up recommendations and ways of using AI tools.
Collapse
Affiliation(s)
- Lotte J S Ewals
- Department of Radiology, Catharina Cancer Institute, Catharina Hospital Eindhoven, 5623 EJ Eindhoven, The Netherlands
- Department of Electrical Engineering, Eindhoven University of Technology, 5612 AZ Eindhoven, The Netherlands
| | - Kasper van der Wulp
- Department of Radiology, Catharina Cancer Institute, Catharina Hospital Eindhoven, 5623 EJ Eindhoven, The Netherlands
| | - Ben E E M van den Borne
- Department of Pulmonology, Catharina Cancer Institute, Catharina Hospital Eindhoven, 5623 EJ Eindhoven, The Netherlands
| | - Jon R Pluyter
- Department of Experience Design, Royal Philips, 5656 AE Eindhoven, The Netherlands
| | - Igor Jacobs
- Department of Hospital Services and Informatics, Philips Research, 5656 AE Eindhoven, The Netherlands
| | - Dimitrios Mavroeidis
- Department of Data Science, Philips Research, 5656 AE Eindhoven, The Netherlands
| | - Fons van der Sommen
- Department of Electrical Engineering, Eindhoven University of Technology, 5612 AZ Eindhoven, The Netherlands
| | - Joost Nederend
- Department of Radiology, Catharina Cancer Institute, Catharina Hospital Eindhoven, 5623 EJ Eindhoven, The Netherlands
| |
Collapse
|
11
|
Xue P, Si M, Qin D, Wei B, Seery S, Ye Z, Chen M, Wang S, Song C, Zhang B, Ding M, Zhang W, Bai A, Yan H, Dang L, Zhao Y, Rezhake R, Zhang S, Qiao Y, Qu Y, Jiang Y. Unassisted Clinicians Versus Deep Learning-Assisted Clinicians in Image-Based Cancer Diagnostics: Systematic Review With Meta-analysis. J Med Internet Res 2023; 25:e43832. [PMID: 36862499 PMCID: PMC10020907 DOI: 10.2196/43832] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/26/2022] [Revised: 01/19/2023] [Accepted: 02/13/2023] [Indexed: 02/16/2023] Open
Abstract
BACKGROUND A number of publications have demonstrated that deep learning (DL) algorithms matched or outperformed clinicians in image-based cancer diagnostics, but these algorithms are frequently considered as opponents rather than partners. Despite the clinicians-in-the-loop DL approach having great potential, no study has systematically quantified the diagnostic accuracy of clinicians with and without the assistance of DL in image-based cancer identification. OBJECTIVE We systematically quantified the diagnostic accuracy of clinicians with and without the assistance of DL in image-based cancer identification. METHODS PubMed, Embase, IEEEXplore, and the Cochrane Library were searched for studies published between January 1, 2012, and December 7, 2021. Any type of study design was permitted that focused on comparing unassisted clinicians and DL-assisted clinicians in cancer identification using medical imaging. Studies using medical waveform-data graphics material and those investigating image segmentation rather than classification were excluded. Studies providing binary diagnostic accuracy data and contingency tables were included for further meta-analysis. Two subgroups were defined and analyzed, including cancer type and imaging modality. RESULTS In total, 9796 studies were identified, of which 48 were deemed eligible for systematic review. Twenty-five of these studies made comparisons between unassisted clinicians and DL-assisted clinicians and provided sufficient data for statistical synthesis. We found a pooled sensitivity of 83% (95% CI 80%-86%) for unassisted clinicians and 88% (95% CI 86%-90%) for DL-assisted clinicians. Pooled specificity was 86% (95% CI 83%-88%) for unassisted clinicians and 88% (95% CI 85%-90%) for DL-assisted clinicians. The pooled sensitivity and specificity values for DL-assisted clinicians were higher than for unassisted clinicians, at ratios of 1.07 (95% CI 1.05-1.09) and 1.03 (95% CI 1.02-1.05), respectively. Similar diagnostic performance by DL-assisted clinicians was also observed across the predefined subgroups. CONCLUSIONS The diagnostic performance of DL-assisted clinicians appears better than unassisted clinicians in image-based cancer identification. However, caution should be exercised, because the evidence provided in the reviewed studies does not cover all the minutiae involved in real-world clinical practice. Combining qualitative insights from clinical practice with data-science approaches may improve DL-assisted practice, although further research is required. TRIAL REGISTRATION PROSPERO CRD42021281372; https://www.crd.york.ac.uk/prospero/display_record.php?RecordID=281372.
Collapse
Affiliation(s)
- Peng Xue
- Department of Epidemiology and Biostatistics, School of Population Medicine and Public Health, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, China
| | - Mingyu Si
- Department of Epidemiology and Biostatistics, School of Population Medicine and Public Health, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, China
| | - Dongxu Qin
- Department of Epidemiology and Biostatistics, School of Population Medicine and Public Health, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, China
| | - Bingrui Wei
- Department of Epidemiology and Biostatistics, School of Population Medicine and Public Health, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, China
| | - Samuel Seery
- Faculty of Health and Medicine, Division of Health Research, Lancaster University, Lancaster, United Kingdom
| | - Zichen Ye
- Department of Epidemiology and Biostatistics, School of Population Medicine and Public Health, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, China
| | - Mingyang Chen
- Department of Epidemiology and Biostatistics, School of Population Medicine and Public Health, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, China
| | - Sumeng Wang
- Department of Cancer Epidemiology, National Cancer Center/National Clinical Research Center for Cancer/Cancer Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, China
| | - Cheng Song
- Department of Epidemiology and Biostatistics, School of Population Medicine and Public Health, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, China
| | - Bo Zhang
- Department of Epidemiology and Biostatistics, School of Population Medicine and Public Health, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, China
| | - Ming Ding
- Department of Epidemiology and Biostatistics, School of Population Medicine and Public Health, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, China
| | - Wenling Zhang
- Department of Epidemiology and Biostatistics, School of Population Medicine and Public Health, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, China
| | - Anying Bai
- Department of Epidemiology and Biostatistics, School of Population Medicine and Public Health, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, China
| | - Huijiao Yan
- Department of Epidemiology and Biostatistics, School of Population Medicine and Public Health, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, China
| | - Le Dang
- Peking Union Medical College Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, China
| | - Yuqian Zhao
- Sichuan Cancer Hospital & Institute, Sichuan Cancer Center, School of Medicine, University of Electronic Science & Technology of China, Sichuan, China
| | - Remila Rezhake
- Affiliated Cancer Hospital, The 3rd Affiliated Teaching Hospital of Xinjiang Medical University, Xinjiang, China
| | - Shaokai Zhang
- Henan Cancer Hospital, Affiliated Cancer Hospital of Zhengzhou University, Henan, China
| | - Youlin Qiao
- Center for Global Health, School of Population Medicine and Public Health, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, China
| | - Yimin Qu
- Department of Epidemiology and Biostatistics, School of Population Medicine and Public Health, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, China
| | - Yu Jiang
- Department of Epidemiology and Biostatistics, School of Population Medicine and Public Health, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, China
| |
Collapse
|
12
|
Nuutinen M, Hiltunen AM, Korhonen S, Haavisto I, Poikonen-Saksela P, Mattson J, Manikis G, Kondylakis H, Simos P, Mazzocco K, Pat-Horenczyk R, Sousa B, Cardoso F, Manica I, Kudel I, Leskelä RL. Aid of a machine learning algorithm can improve clinician predictions of patient quality of life during breast cancer treatments. HEALTH AND TECHNOLOGY 2023. [DOI: 10.1007/s12553-023-00733-7] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 02/12/2023]
|
13
|
Radiologists with and without deep learning-based computer-aided diagnosis: comparison of performance and interobserver agreement for characterizing and diagnosing pulmonary nodules/masses. Eur Radiol 2022; 33:348-359. [PMID: 35751697 DOI: 10.1007/s00330-022-08948-4] [Citation(s) in RCA: 9] [Impact Index Per Article: 4.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/16/2022] [Revised: 05/01/2022] [Accepted: 06/08/2022] [Indexed: 11/04/2022]
Abstract
OBJECTIVES To compare the performance of radiologists in characterizing and diagnosing pulmonary nodules/masses with and without deep learning (DL)-based computer-aided diagnosis (CAD). METHODS We studied a total of 101 nodules/masses detected on CT performed between January and March 2018 at Osaka University Hospital (malignancy: 55 cases). SYNAPSE SAI Viewer V1.4 was used to analyze the nodules/masses. In total, 15 independent radiologists were grouped (n = 5 each) according to their experience: L (< 3 years), M (3-5 years), and H (> 5 years). The likelihoods of 15 characteristics, such as cavitation and calcification, and the diagnosis (malignancy) were evaluated by each radiologist with and without CAD, and the assessment time was recorded. The AUCs compared with the reference standard set by two board-certified chest radiologists were analyzed following the multi-reader multi-case method. Furthermore, interobserver agreement was compared using intraclass correlation coefficients (ICCs). RESULTS The AUCs for ill-defined boundary, irregular margin, irregular shape, calcification, pleural contact, and malignancy in all 15 radiologists, irregular margin and irregular shape in L and ill-defined boundary and irregular margin in M improved significantly (p < 0.05); no significant improvements were found in H. L showed the greatest increase in the AUC for malignancy (not significant). The ICCs improved in all groups and for nearly all items. The median assessment time was not prolonged by CAD. CONCLUSIONS DL-based CAD helps radiologists, particularly those with < 5 years of experience, to accurately characterize and diagnose pulmonary nodules/masses, and improves the reproducibility of findings among radiologists. KEY POINTS • Deep learning-based computer-aided diagnosis improves the accuracy of characterizing nodules/masses and diagnosing malignancy, particularly by radiologists with < 5 years of experience. • Computer-aided diagnosis increases not only the accuracy but also the reproducibility of the findings across radiologists.
Collapse
|
14
|
Wang Y, Cai H, Pu Y, Li J, Yang F, Yang C, Chen L, Hu Z. The value of AI in the Diagnosis, Treatment, and Prognosis of Malignant Lung Cancer. FRONTIERS IN RADIOLOGY 2022; 2:810731. [PMID: 37492685 PMCID: PMC10365105 DOI: 10.3389/fradi.2022.810731] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 11/07/2021] [Accepted: 03/30/2022] [Indexed: 07/27/2023]
Abstract
Malignant tumors is a serious public health threat. Among them, lung cancer, which has the highest fatality rate globally, has significantly endangered human health. With the development of artificial intelligence (AI) and its integration with medicine, AI research in malignant lung tumors has become critical. This article reviews the value of CAD, computer neural network deep learning, radiomics, molecular biomarkers, and digital pathology for the diagnosis, treatment, and prognosis of malignant lung tumors.
Collapse
Affiliation(s)
- Yue Wang
- Department of PET/CT Center, Cancer Center of Yunnan Province, Yunnan Cancer Hospital, The Third Affiliated Hospital of Kunming Medical University, Kunming, China
| | - Haihua Cai
- Lauterbur Research Center for Biomedical Imaging, Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences, Shenzhen, China
| | - Yongzhu Pu
- Department of PET/CT Center, Cancer Center of Yunnan Province, Yunnan Cancer Hospital, The Third Affiliated Hospital of Kunming Medical University, Kunming, China
| | - Jindan Li
- Department of PET/CT Center, Cancer Center of Yunnan Province, Yunnan Cancer Hospital, The Third Affiliated Hospital of Kunming Medical University, Kunming, China
| | - Fake Yang
- Department of PET/CT Center, Cancer Center of Yunnan Province, Yunnan Cancer Hospital, The Third Affiliated Hospital of Kunming Medical University, Kunming, China
| | - Conghui Yang
- Department of PET/CT Center, Cancer Center of Yunnan Province, Yunnan Cancer Hospital, The Third Affiliated Hospital of Kunming Medical University, Kunming, China
| | - Long Chen
- Department of PET/CT Center, Cancer Center of Yunnan Province, Yunnan Cancer Hospital, The Third Affiliated Hospital of Kunming Medical University, Kunming, China
| | - Zhanli Hu
- Lauterbur Research Center for Biomedical Imaging, Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences, Shenzhen, China
| |
Collapse
|
15
|
Hu Q, Liu Y, Chen C, Kang S, Sun Z, Wang Y, Xiang M, Guan H, Xia L. Application of computer-aided detection (CAD) software to automatically detect nodules under SDCT and LDCT scans with different parameters. Comput Biol Med 2022; 146:105538. [DOI: 10.1016/j.compbiomed.2022.105538] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/22/2022] [Revised: 03/26/2022] [Accepted: 04/14/2022] [Indexed: 11/03/2022]
|
16
|
Comparison of CO-RADS Scores Based on Visual and Artificial Intelligence Assessments in a Non-Endemic Area. Diagnostics (Basel) 2022; 12:diagnostics12030738. [PMID: 35328290 PMCID: PMC8946998 DOI: 10.3390/diagnostics12030738] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/26/2022] [Revised: 03/08/2022] [Accepted: 03/11/2022] [Indexed: 11/17/2022] Open
Abstract
In this study, we first developed an artificial intelligence (AI)-based algorithm for classifying chest computed tomography (CT) images using the coronavirus disease 2019 Reporting and Data System (CO-RADS). Subsequently, we evaluated its accuracy by comparing the calculated scores with those assigned by radiologists with varying levels of experience. This study included patients with suspected SARS-CoV-2 infection who underwent chest CT imaging between February and October 2020 in Japan, a non-endemic area. For each chest CT, the CO-RADS scores, determined by consensus among three experienced chest radiologists, were used as the gold standard. Images from 412 patients were used to train the model, whereas images from 83 patients were tested to obtain AI-based CO-RADS scores for each image. Six independent raters (one medical student, two residents, and three board-certified radiologists) evaluated the test images. Intraclass correlation coefficients (ICC) and weighted kappa values were calculated to determine the inter-rater agreement with the gold standard. The mean ICC and weighted kappa were 0.754 and 0.752 for the medical student and residents (taken together), 0.851 and 0.850 for the diagnostic radiologists, and 0.913 and 0.912 for AI, respectively. The CO-RADS scores calculated using our AI-based algorithm were comparable to those assigned by radiologists, indicating the accuracy and high reproducibility of our model. Our study findings would enable accurate reading, particularly in areas where radiologists are unavailable, and contribute to improvements in patient management and workflow.
Collapse
|
17
|
Nagaoka T, Kozuka T, Yamada T, Habe H, Nemoto M, Tada M, Abe K, Handa H, Yoshida H, Ishii K, Kimura Y. A Deep Learning System to Diagnose COVID-19 Pneumonia Using Masked Lung CT Images to Avoid AI-generated COVID-19 Diagnoses that Include Data outside the Lungs. ADVANCED BIOMEDICAL ENGINEERING 2022. [DOI: 10.14326/abe.11.76] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/04/2022] Open
Affiliation(s)
- Takashi Nagaoka
- Department of Computational Systems Biology, Faculty of Biology-Oriented Science and Technology, Kindai University
| | - Takenori Kozuka
- Department of Radiology, Kindai University Faculty of Medicine
| | - Takahiro Yamada
- Institute of Advanced Clinical Medicine, Kindai University Hospital
| | - Hitoshi Habe
- Department of Informatics, Faculty of Science and Technology, Kindai University
| | - Mitsutaka Nemoto
- Department of Biomedical Engineering, Faculty of Biology-Oriented Science and Technology, Kindai University
| | - Masahiro Tada
- Department of Informatics, Faculty of Science and Technology, Kindai University
| | - Koji Abe
- Department of Informatics, Faculty of Science and Technology, Kindai University
| | - Hisashi Handa
- Department of Informatics, Faculty of Science and Technology, Kindai University
| | - Hisashi Yoshida
- Department of Computational Systems Biology, Faculty of Biology-Oriented Science and Technology, Kindai University
| | - Kazunari Ishii
- Institute of Advanced Clinical Medicine, Kindai University Hospital
| | - Yuichi Kimura
- Department of Computational Systems Biology, Faculty of Biology-Oriented Science and Technology, Kindai University
| |
Collapse
|
18
|
Li D, Pehrson LM, Lauridsen CA, Tøttrup L, Fraccaro M, Elliott D, Zając HD, Darkner S, Carlsen JF, Nielsen MB. The Added Effect of Artificial Intelligence on Physicians' Performance in Detecting Thoracic Pathologies on CT and Chest X-ray: A Systematic Review. Diagnostics (Basel) 2021; 11:diagnostics11122206. [PMID: 34943442 PMCID: PMC8700414 DOI: 10.3390/diagnostics11122206] [Citation(s) in RCA: 15] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/20/2021] [Revised: 11/18/2021] [Accepted: 11/23/2021] [Indexed: 12/20/2022] Open
Abstract
Our systematic review investigated the additional effect of artificial intelligence-based devices on human observers when diagnosing and/or detecting thoracic pathologies using different diagnostic imaging modalities, such as chest X-ray and CT. Peer-reviewed, original research articles from EMBASE, PubMed, Cochrane library, SCOPUS, and Web of Science were retrieved. Included articles were published within the last 20 years and used a device based on artificial intelligence (AI) technology to detect or diagnose pulmonary findings. The AI-based device had to be used in an observer test where the performance of human observers with and without addition of the device was measured as sensitivity, specificity, accuracy, AUC, or time spent on image reading. A total of 38 studies were included for final assessment. The quality assessment tool for diagnostic accuracy studies (QUADAS-2) was used for bias assessment. The average sensitivity increased from 67.8% to 74.6%; specificity from 82.2% to 85.4%; accuracy from 75.4% to 81.7%; and Area Under the ROC Curve (AUC) from 0.75 to 0.80. Generally, a faster reading time was reported when radiologists were aided by AI-based devices. Our systematic review showed that performance generally improved for the physicians when assisted by AI-based devices compared to unaided interpretation.
Collapse
Affiliation(s)
- Dana Li
- Department of Diagnostic Radiology, Copenhagen University Hospital, Rigshospitalet, 2100 Copenhagen, Denmark; (L.M.P.); (C.A.L.); (J.F.C.); (M.B.N.)
- Department of Clinical Medicine, University of Copenhagen, 2100 Copenhagen, Denmark
- Correspondence:
| | - Lea Marie Pehrson
- Department of Diagnostic Radiology, Copenhagen University Hospital, Rigshospitalet, 2100 Copenhagen, Denmark; (L.M.P.); (C.A.L.); (J.F.C.); (M.B.N.)
| | - Carsten Ammitzbøl Lauridsen
- Department of Diagnostic Radiology, Copenhagen University Hospital, Rigshospitalet, 2100 Copenhagen, Denmark; (L.M.P.); (C.A.L.); (J.F.C.); (M.B.N.)
- Department of Technology, Faculty of Health and Technology, University College Copenhagen, 2200 Copenhagen, Denmark
| | - Lea Tøttrup
- Unumed Aps, 1055 Copenhagen, Denmark; (L.T.); (M.F.)
| | | | - Desmond Elliott
- Department of Computer Science, University of Copenhagen, 2100 Copenhagen, Denmark; (D.E.); (H.D.Z.); (S.D.)
| | - Hubert Dariusz Zając
- Department of Computer Science, University of Copenhagen, 2100 Copenhagen, Denmark; (D.E.); (H.D.Z.); (S.D.)
| | - Sune Darkner
- Department of Computer Science, University of Copenhagen, 2100 Copenhagen, Denmark; (D.E.); (H.D.Z.); (S.D.)
| | - Jonathan Frederik Carlsen
- Department of Diagnostic Radiology, Copenhagen University Hospital, Rigshospitalet, 2100 Copenhagen, Denmark; (L.M.P.); (C.A.L.); (J.F.C.); (M.B.N.)
| | - Michael Bachmann Nielsen
- Department of Diagnostic Radiology, Copenhagen University Hospital, Rigshospitalet, 2100 Copenhagen, Denmark; (L.M.P.); (C.A.L.); (J.F.C.); (M.B.N.)
- Department of Clinical Medicine, University of Copenhagen, 2100 Copenhagen, Denmark
| |
Collapse
|
19
|
Zhang Y, Jiang B, Zhang L, Greuter MJW, de Bock GH, Zhang H, Xie X. Lung Nodule Detectability of Artificial Intelligence-assisted CT Image Reading in Lung Cancer Screening. Curr Med Imaging 2021; 18:327-334. [PMID: 34365951 DOI: 10.2174/1573405617666210806125953] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/01/2021] [Revised: 06/11/2021] [Accepted: 06/17/2021] [Indexed: 11/22/2022]
Abstract
BACKGROUND Artificial intelligence (AI)-based automatic lung nodule detection system improves the detection rate of nodules. It is important to evaluate the clinical value of AI system by comparing AI-assisted nodule detection with actu-al radiology reports. OBJECTIVE To compare the detection rate of lung nodules between the actual radiology reports and AI-assisted reading in lung cancer CT screening. METHODS Participants in chest CT screening from November to December 2019 were retrospectively included. In the real-world radiologist observation, 14 residents and 15 radiologists participated to finalize radiology reports. In AI-assisted reading, one resident and one radiologist reevaluated all subjects with the assistance of an AI system to lo-cate and measure the detected lung nodules. A reading panel determined the type and number of detected lung nodules between these two methods. RESULTS In 860 participants (57±7 years), the reading panel confirmed 250 patients with >1 solid nodule, while radiolo-gists observed 131, lower than 247 by AI-assisted reading (p<0.001). The panel confirmed 111 patients with >1 non-solid nodule, whereas radiologist observation identified 28, lower than 110 by AI-assisted reading (p<0.001). The accuracy and sensitivity of radiologist observation for solid nodules were 86.2% and 52.4%, lower than 99.1% and 98.8% by AI-assisted reading, respectively. These metrics were 90.4% and 25.2% for non-solid nodules, lower than 98.8% and 99.1% by AI-assisted reading, respectively. CONCLUSION Comparing with the actual radiology reports, AI-assisted reading greatly improves the accuracy and sensi-tivity of nodule detection in chest CT, which benefits lung nodule detection, especially for non-solid nodules.
Collapse
Affiliation(s)
- Yaping Zhang
- Department of Radiology, Shanghai General Hospital of Nanjing Medical University, Haining Rd.100, Shanghai 200080. China
| | - Beibei Jiang
- Department of Radiology, Shanghai General Hospital, Shanghai Jiao Tong University School of Medicine, Haining Rd.100, Shanghai 200080. 0
| | - Lu Zhang
- Department of Radiology, Shanghai General Hospital, Shanghai Jiao Tong University School of Medicine, Haining Rd.100, Shanghai 200080. 0
| | - Marcel J W Greuter
- Department of Radiology, University of Groningen, University Medical Center Groningen, Hanzeplein 1, 9713GZ Gro-ningen. Netherlands
| | - Geertruida H de Bock
- Department of Epidemiology, University of Groningen, University Medical Center Groningen, Hanzeplein 1, 9713GZ Groningen. Netherlands
| | - Hao Zhang
- Department of Radiology, Shanghai General Hospital of Nanjing Medical University, Haining Rd.100, Shanghai 200080. China
| | - Xueqian Xie
- Department of Radiology, Shanghai General Hospital of Nanjing Medical University, Haining Rd.100, Shanghai 200080. China
| |
Collapse
|
20
|
Takaishi T, Ozawa Y, Bando Y, Yamamoto A, Okochi S, Suzuki H, Shibamoto Y. Incorporation of a computer-aided vessel-suppression system to detect lung nodules in CT images: effect on sensitivity and reading time in routine clinical settings. Jpn J Radiol 2020; 39:159-164. [PMID: 32940850 DOI: 10.1007/s11604-020-01043-y] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/30/2020] [Accepted: 09/10/2020] [Indexed: 11/24/2022]
Abstract
PURPOSE To evaluate whether a computer-aided vessel-suppression system improves lung nodule detection in routine clinical settings. MATERIALS AND METHODS We used computer software that automatically suppresses pulmonary vessels on chest CT while preserving pulmonary nodules. Sixty-one chest CT images were included in our study. Three radiologists independently read either standard CT images alone or both computer-aided CT and standard CT images randomly to detect a pulmonary nodule ≥ 4 mm in diameter. After an interval of at least 15 days to avoid recall bias, the three radiologists interpreted the counterpart images of the same patients. The reference standard was decided by an expert panel. The primary endpoint was sensitivity. The secondary endpoint was interpretation time. RESULTS The average sensitivity improved with computer-aided CT (72% for standard CT vs. 84% for computer-aided CT, p = 0.02). There was no difference in the false-positive rate (21% for both standard CT and computer-aided CT, p = 0.98). Although the average reading time was 9.5% longer for computer-aided plus standard CT compared with standard CT alone, the difference was not significant (p = 0.11). CONCLUSION Vessel-suppressed CT images helped radiologists to improve the sensitivity of pulmonary nodule detection without compromising the false-positive rate.
Collapse
Affiliation(s)
- Taku Takaishi
- Konan Kosei Hospital, Takayacho-Omatsubara 137, Konan, Aichi, Japan.
| | - Yoshiyuki Ozawa
- Department of Radiology, Nagoya City University Graduate School of Medical Sciences, Nagoya, Japan
| | - Yuya Bando
- Konan Kosei Hospital, Takayacho-Omatsubara 137, Konan, Aichi, Japan
| | - Akiko Yamamoto
- Konan Kosei Hospital, Takayacho-Omatsubara 137, Konan, Aichi, Japan
| | - Sachiko Okochi
- Konan Kosei Hospital, Takayacho-Omatsubara 137, Konan, Aichi, Japan
| | - Hirochika Suzuki
- Konan Kosei Hospital, Takayacho-Omatsubara 137, Konan, Aichi, Japan
| | - Yuta Shibamoto
- Department of Radiology, Nagoya City University Graduate School of Medical Sciences, Nagoya, Japan
| |
Collapse
|