1
|
Kuwabara M, Ikawa F, Sakamoto S, Okazaki T, Ishii D, Hosogai M, Maeda Y, Chiku M, Kitamura N, Choppin A, Takamiya D, Shimahara Y, Nakayama T, Kurisu K, Horie N. Effectiveness of tuning an artificial intelligence algorithm for cerebral aneurysm diagnosis: a study of 10,000 consecutive cases. Sci Rep 2023; 13:16202. [PMID: 37758849 PMCID: PMC10533861 DOI: 10.1038/s41598-023-43418-x] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/17/2023] [Accepted: 09/23/2023] [Indexed: 09/29/2023] Open
Abstract
Diagnostic image analysis for unruptured cerebral aneurysms using artificial intelligence has a very high sensitivity. However, further improvement is needed because of a relatively high number of false positives. This study aimed to confirm the clinical utility of tuning an artificial intelligence algorithm for cerebral aneurysm diagnosis. We extracted 10,000 magnetic resonance imaging scans of participants who underwent brain screening using the "Brain Dock" system. The sensitivity and false positives/case for aneurysm detection were compared before and after tuning the algorithm. The initial diagnosis included only cases for which feedback to the algorithm was provided. In the primary analysis, the sensitivity of aneurysm diagnosis decreased from 96.5 to 90% and the false positives/case improved from 2.06 to 0.99 after tuning the algorithm (P < 0.001). In the secondary analysis, the sensitivity of aneurysm diagnosis decreased from 98.8 to 94.6% and the false positives/case improved from 1.99 to 1.03 after tuning the algorithm (P < 0.001). The false positives/case reduced without a significant decrease in sensitivity. Using large clinical datasets, we demonstrated that by tuning the algorithm, we could significantly reduce false positives with a minimal decline in sensitivity.
Collapse
Affiliation(s)
- Masashi Kuwabara
- Department of Neurosurgery, Graduate School of Biomedical and Health Sciences, Hiroshima University, 1-2-3 Kasumi, Minami-ku, Hiroshima, Hiroshima, 734-8551, Japan
| | - Fusao Ikawa
- Department of Neurosurgery, Graduate School of Biomedical and Health Sciences, Hiroshima University, 1-2-3 Kasumi, Minami-ku, Hiroshima, Hiroshima, 734-8551, Japan.
- Department of Neurosurgery, Shimane Prefectural Central Hospital, 4-1-1 Himebara, Izumo, Shimane, 693-8555, Japan.
| | - Shigeyuki Sakamoto
- Department of Neurosurgery, Graduate School of Biomedical and Health Sciences, Hiroshima University, 1-2-3 Kasumi, Minami-ku, Hiroshima, Hiroshima, 734-8551, Japan
| | - Takahito Okazaki
- Department of Neurosurgery, Graduate School of Biomedical and Health Sciences, Hiroshima University, 1-2-3 Kasumi, Minami-ku, Hiroshima, Hiroshima, 734-8551, Japan
| | - Daizo Ishii
- Department of Neurosurgery, Graduate School of Biomedical and Health Sciences, Hiroshima University, 1-2-3 Kasumi, Minami-ku, Hiroshima, Hiroshima, 734-8551, Japan
| | - Masahiro Hosogai
- Department of Neurosurgery, Graduate School of Biomedical and Health Sciences, Hiroshima University, 1-2-3 Kasumi, Minami-ku, Hiroshima, Hiroshima, 734-8551, Japan
| | - Yuyo Maeda
- Department of Neurosurgery, Graduate School of Biomedical and Health Sciences, Hiroshima University, 1-2-3 Kasumi, Minami-ku, Hiroshima, Hiroshima, 734-8551, Japan
| | - Masaaki Chiku
- Department of Neurosurgery, Medical Check Studio, Tokyo Ginza Clinic, 1-2-4 Ginza, Chuo-ku, Tokyo, 104-0061, Japan
| | - Naoyuki Kitamura
- Department of Diagnostic Radiology, Kasumi Clinic, 1-2-27 Shinonomehommachi, Minami-ku, Hiroshima, Hiroshima, 734-0023, Japan
| | - Antoine Choppin
- LPIXEL Inc., 1-6-1 Otemachi, Chiyoda-ku, Tokyo, 100-0004, Japan
| | | | - Yuki Shimahara
- LPIXEL Inc., 1-6-1 Otemachi, Chiyoda-ku, Tokyo, 100-0004, Japan
| | - Takeo Nakayama
- Department of Health Informatics, School of Public Health, Graduate School of Medicine, Kyoto University, Yoshida-Konoe, Sakyo-ku, Kyoto, Kyoto, 606-8501, Japan
| | - Kaoru Kurisu
- Chugoku Rosai Hospital, 1-5-1 Hirotagaya, Kure, Hiroshima, 737-0193, Japan
| | - Nobutaka Horie
- Department of Neurosurgery, Graduate School of Biomedical and Health Sciences, Hiroshima University, 1-2-3 Kasumi, Minami-ku, Hiroshima, Hiroshima, 734-8551, Japan
| |
Collapse
|
2
|
Din M, Agarwal S, Grzeda M, Wood DA, Modat M, Booth TC. Detection of cerebral aneurysms using artificial intelligence: a systematic review and meta-analysis. J Neurointerv Surg 2023; 15:262-271. [PMID: 36375834 PMCID: PMC9985742 DOI: 10.1136/jnis-2022-019456] [Citation(s) in RCA: 17] [Impact Index Per Article: 17.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/29/2022] [Accepted: 10/11/2022] [Indexed: 11/16/2022]
Abstract
BACKGROUND Subarachnoid hemorrhage from cerebral aneurysm rupture is a major cause of morbidity and mortality. Early aneurysm identification, aided by automated systems, may improve patient outcomes. Therefore, a systematic review and meta-analysis of the diagnostic accuracy of artificial intelligence (AI) algorithms in detecting cerebral aneurysms using CT, MRI or DSA was performed. METHODS MEDLINE, Embase, Cochrane Library and Web of Science were searched until August 2021. Eligibility criteria included studies using fully automated algorithms to detect cerebral aneurysms using MRI, CT or DSA. Following Preferred Reporting Items for Systematic Reviews and Meta-Analysis: Diagnostic Test Accuracy (PRISMA-DTA), articles were assessed using Quality Assessment of Diagnostic Accuracy Studies 2 (QUADAS-2). Meta-analysis included a bivariate random-effect model to determine pooled sensitivity, specificity, and area under the receiver operator characteristic curve (ROC-AUC). PROSPERO CRD42021278454. RESULTS 43 studies were included, and 41/43 (95%) were retrospective. 34/43 (79%) used AI as a standalone tool, while 9/43 (21%) used AI assisting a reader. 23/43 (53%) used deep learning. Most studies had high bias risk and applicability concerns, limiting conclusions. Six studies in the standalone AI meta-analysis gave (pooled) 91.2% (95% CI 82.2% to 95.8%) sensitivity; 16.5% (95% CI 9.4% to 27.1%) false-positive rate (1-specificity); 0.936 ROC-AUC. Five reader-assistive AI studies gave (pooled) 90.3% (95% CI 88.0% - 92.2%) sensitivity; 7.9% (95% CI 3.5% to 16.8%) false-positive rate; 0.910 ROC-AUC. CONCLUSION AI has the potential to support clinicians in detecting cerebral aneurysms. Interpretation is limited due to high risk of bias and poor generalizability. Multicenter, prospective studies are required to assess AI in clinical practice.
Collapse
Affiliation(s)
- Munaib Din
- School of Biomedical Engineering & Imaging Sciences, King's College London, London, UK
| | - Siddharth Agarwal
- School of Biomedical Engineering & Imaging Sciences, King's College London, London, UK
| | - Mariusz Grzeda
- School of Biomedical Engineering & Imaging Sciences, King's College London, London, UK
| | - David A Wood
- School of Biomedical Engineering & Imaging Sciences, King's College London, London, UK
| | - Marc Modat
- School of Biomedical Engineering & Imaging Sciences, King's College London, London, UK
| | - Thomas C Booth
- School of Biomedical Engineering & Imaging Sciences, King's College London, London, UK
- Department of Neuroradiology, King's College Hospital NHS Foundation Trust, London, UK
| |
Collapse
|
3
|
Alwalid O, Long X, Xie M, Han P. Artificial Intelligence Applications in Intracranial Aneurysm: Achievements, Challenges and Opportunities. Acad Radiol 2022; 29 Suppl 3:S201-S214. [PMID: 34376335 DOI: 10.1016/j.acra.2021.06.013] [Citation(s) in RCA: 6] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/06/2021] [Revised: 06/22/2021] [Accepted: 06/29/2021] [Indexed: 01/10/2023]
Abstract
Intracranial aneurysms present in about 3% of the general population and the number of detected aneurysms is continuously rising with the advances in imaging techniques. Intracranial aneurysm rupture carries a high risk of death or permanent disabilities; therefore assessment of the intracranial aneurysm along the entire course is of great clinical importance. Given the outstanding performance of artificial intelligence (AI) in image-based tasks, many AI-based applications have emerged in recent years for the assessment of intracranial aneurysms. In this review we will summarize the state-of-the-art of AI applications in intracranial aneurysms, emphasizing the achievements, and exploring the challenges. We will also discuss the future prospects and potential opportunities. This article provides an updated view of the AI applications in intracranial aneurysms and may act as a basis for guiding the related future works.
Collapse
Affiliation(s)
- Osamah Alwalid
- Department of Radiology, Union Hospital, Tongji Medical College, Huazhong University of Science and Technology, 1277 Jiefang Avenue, Wuhan 430022, China; Hubei Province Key Laboratory of Molecular Imaging, Wuhan 430022, China
| | - Xi Long
- Department of Radiology, Union Hospital, Tongji Medical College, Huazhong University of Science and Technology, 1277 Jiefang Avenue, Wuhan 430022, China; Hubei Province Key Laboratory of Molecular Imaging, Wuhan 430022, China
| | - Mingfei Xie
- Department of Radiology, Union Hospital, Tongji Medical College, Huazhong University of Science and Technology, 1277 Jiefang Avenue, Wuhan 430022, China; Hubei Province Key Laboratory of Molecular Imaging, Wuhan 430022, China
| | - Ping Han
- Department of Radiology, Union Hospital, Tongji Medical College, Huazhong University of Science and Technology, 1277 Jiefang Avenue, Wuhan 430022, China; Hubei Province Key Laboratory of Molecular Imaging, Wuhan 430022, China.
| |
Collapse
|
4
|
Nomura Y, Hanaoka S, Nakao T, Hayashi N, Yoshikawa T, Miki S, Watadani T, Abe O. Performance changes due to differences in training data for cerebral aneurysm detection in head MR angiography images. Jpn J Radiol 2021; 39:1039-1048. [PMID: 34125368 DOI: 10.1007/s11604-021-01153-1] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/25/2021] [Accepted: 06/08/2021] [Indexed: 01/10/2023]
Abstract
PURPOSE The performance of computer-aided detection (CAD) software depends on the quality and quantity of the dataset used for machine learning. If the data characteristics in development and practical use are different, the performance of CAD software degrades. In this study, we investigated changes in detection performance due to differences in training data for cerebral aneurysm detection software in head magnetic resonance angiography images. MATERIALS AND METHODS We utilized three types of CAD software for cerebral aneurysm detection in MRA images, which were based on 3D local intensity structure analysis, graph-based features, and convolutional neural network. For each type of CAD software, we compared three types of training pattern, which were two types of training using single-site data and one type of training using multisite data. We also carried out internal and external evaluations. RESULTS In training using single-site data, the performance of CAD software largely and unpredictably fluctuated when the training dataset was changed. Training using multisite data did not show the lowest performance among the three training patterns for any CAD software and dataset. CONCLUSION The training of cerebral aneurysm detection software using data collected from multiple sites is desirable to ensure the stable performance of the software.
Collapse
Affiliation(s)
- Yukihiro Nomura
- Department of Computational Diagnostic Radiology and Preventive Medicine, The University of Tokyo Hospital, 7-3-1 Hongo, Bunkyo-ku, Tokyo, 113-8655, Japan.
| | - Shouhei Hanaoka
- Department of Radiology, The University of Tokyo Hospital, 7-3-1 Hongo, Bunkyo-ku, Tokyo, 113-8655, Japan
| | - Takahiro Nakao
- Department of Computational Diagnostic Radiology and Preventive Medicine, The University of Tokyo Hospital, 7-3-1 Hongo, Bunkyo-ku, Tokyo, 113-8655, Japan
| | - Naoto Hayashi
- Department of Computational Diagnostic Radiology and Preventive Medicine, The University of Tokyo Hospital, 7-3-1 Hongo, Bunkyo-ku, Tokyo, 113-8655, Japan
| | - Takeharu Yoshikawa
- Department of Computational Diagnostic Radiology and Preventive Medicine, The University of Tokyo Hospital, 7-3-1 Hongo, Bunkyo-ku, Tokyo, 113-8655, Japan
| | - Soichiro Miki
- Department of Computational Diagnostic Radiology and Preventive Medicine, The University of Tokyo Hospital, 7-3-1 Hongo, Bunkyo-ku, Tokyo, 113-8655, Japan
| | - Takeyuki Watadani
- Department of Radiology, The University of Tokyo Hospital, 7-3-1 Hongo, Bunkyo-ku, Tokyo, 113-8655, Japan
| | - Osamu Abe
- Department of Radiology, The University of Tokyo Hospital, 7-3-1 Hongo, Bunkyo-ku, Tokyo, 113-8655, Japan
| |
Collapse
|
5
|
Detection of clustered anomalies in single-voxel morphometry as a rapid automated method for identifying intracranial aneurysms. Comput Med Imaging Graph 2021; 89:101888. [PMID: 33690001 DOI: 10.1016/j.compmedimag.2021.101888] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/28/2020] [Revised: 01/18/2021] [Accepted: 01/24/2021] [Indexed: 12/13/2022]
Abstract
Unruptured intracranial aneurysms (UIAs) are prevalent neurovascular anomalies which, in rare circumstances, rupture to cause a catastrophic subarachnoid haemorrhage. Although surgical management can reduce rupture risk, the majority of UIAs exist undiscovered until rupture. Current clinical practice in the detection of UIAs relies heavily on manual radiological review of standard imaging modalities. Recent computer-aided UIA diagnoses can sensitively detect and measure UIAs within cranial angiograms but remain limited to low specificities whose output also requires considerable radiologist interpretation not amenable to broad screening efforts. To address these limitations, we have developed a novel automatic pipeline algorithm which inputs medical images and outputs detected UIAs by characterising single-voxel morphometry of segmented neurovasculature. Once neurovascular anatomy of a specified resolution is segmented, correlations between voxel-specific morphometries are estimated and spatially-clustered outliers are identified as UIA candidates. Our automated solution detects UIAs within magnetic resonance angiograms (MRA) at unmatched 86% specificity and 81% sensitivity using 3 min on a conventional laptop. Our approach does not rely on interpatient comparisons or training datasets which could be difficult to amass and process for rare incidentally discovered UIAs within large MRA files, and in doing so, is versatile to user-defined segmentation quality, to detection sensitivity, and across a range of imaging resolutions and modalities. We propose this method as a unique tool to aid UIA screening, characterisation of abnormal vasculature in at-risk patients, morphometry-based rupture risk prediction, and identification of other vascular abnormalities.
Collapse
|
6
|
Shi Z, Hu B, Schoepf UJ, Savage RH, Dargis DM, Pan CW, Li XL, Ni QQ, Lu GM, Zhang LJ. Artificial Intelligence in the Management of Intracranial Aneurysms: Current Status and Future Perspectives. AJNR Am J Neuroradiol 2020; 41:373-379. [PMID: 32165361 DOI: 10.3174/ajnr.a6468] [Citation(s) in RCA: 40] [Impact Index Per Article: 10.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/30/2019] [Accepted: 12/16/2019] [Indexed: 12/13/2022]
Abstract
Intracranial aneurysms with subarachnoid hemorrhage lead to high morbidity and mortality. It is of critical importance to detect aneurysms, identify risk factors of rupture, and predict treatment response of aneurysms to guide clinical interventions. Artificial intelligence has received worldwide attention for its impressive performance in image-based tasks. Artificial intelligence serves as an adjunct to physicians in a series of clinical settings, which substantially improves diagnostic accuracy while reducing physicians' workload. Computer-assisted diagnosis systems of aneurysms based on MRA and CTA using deep learning have been evaluated, and excellent performances have been reported. Artificial intelligence has also been used in automated morphologic calculation, rupture risk stratification, and outcomes prediction with the implementation of machine learning methods, which have exhibited incremental value. This review summarizes current advances of artificial intelligence in the management of aneurysms, including detection and prediction. The challenges and future directions of clinical implementations of artificial intelligence are briefly discussed.
Collapse
Affiliation(s)
- Z Shi
- From the Department of Medical Imaging (Z.S., B.H., Q.Q.N., G.M.L., L.J.Z.), Jinling Hospital, Medical School of Nanjing University, Nanjing, Jiangsu, China
| | - B Hu
- From the Department of Medical Imaging (Z.S., B.H., Q.Q.N., G.M.L., L.J.Z.), Jinling Hospital, Medical School of Nanjing University, Nanjing, Jiangsu, China
| | - U J Schoepf
- Division of Cardiovascular Imaging (U.J.S., R.H.S., D.M.D.), Department of Radiology and Radiological Science, Medical University of South Carolina, Charleston, South Carolina
| | - R H Savage
- Division of Cardiovascular Imaging (U.J.S., R.H.S., D.M.D.), Department of Radiology and Radiological Science, Medical University of South Carolina, Charleston, South Carolina
| | - D M Dargis
- Division of Cardiovascular Imaging (U.J.S., R.H.S., D.M.D.), Department of Radiology and Radiological Science, Medical University of South Carolina, Charleston, South Carolina
| | - C W Pan
- DeepWise AI Lab (C.W.P., X.L.L.), Beijing, China
| | - X L Li
- DeepWise AI Lab (C.W.P., X.L.L.), Beijing, China.,Peng Cheng Laboratory (X.L.L.), Vanke Cloud City Phase I, Nanshan District, Shenzhen, Guangdong, China
| | - Q Q Ni
- From the Department of Medical Imaging (Z.S., B.H., Q.Q.N., G.M.L., L.J.Z.), Jinling Hospital, Medical School of Nanjing University, Nanjing, Jiangsu, China
| | - G M Lu
- From the Department of Medical Imaging (Z.S., B.H., Q.Q.N., G.M.L., L.J.Z.), Jinling Hospital, Medical School of Nanjing University, Nanjing, Jiangsu, China
| | - L J Zhang
- From the Department of Medical Imaging (Z.S., B.H., Q.Q.N., G.M.L., L.J.Z.), Jinling Hospital, Medical School of Nanjing University, Nanjing, Jiangsu, China
| |
Collapse
|