1
|
Akpinar MH, Sengur A, Faust O, Tong L, Molinari F, Acharya UR. Artificial intelligence in retinal screening using OCT images: A review of the last decade (2013-2023). COMPUTER METHODS AND PROGRAMS IN BIOMEDICINE 2024; 254:108253. [PMID: 38861878 DOI: 10.1016/j.cmpb.2024.108253] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 12/22/2023] [Revised: 04/22/2024] [Accepted: 05/25/2024] [Indexed: 06/13/2024]
Abstract
BACKGROUND AND OBJECTIVES Optical coherence tomography (OCT) has ushered in a transformative era in the domain of ophthalmology, offering non-invasive imaging with high resolution for ocular disease detection. OCT, which is frequently used in diagnosing fundamental ocular pathologies, such as glaucoma and age-related macular degeneration (AMD), plays an important role in the widespread adoption of this technology. Apart from glaucoma and AMD, we will also investigate pertinent pathologies, such as epiretinal membrane (ERM), macular hole (MH), macular dystrophy (MD), vitreomacular traction (VMT), diabetic maculopathy (DMP), cystoid macular edema (CME), central serous chorioretinopathy (CSC), diabetic macular edema (DME), diabetic retinopathy (DR), drusen, glaucomatous optic neuropathy (GON), neovascular AMD (nAMD), myopia macular degeneration (MMD) and choroidal neovascularization (CNV) diseases. This comprehensive review examines the role that OCT-derived images play in detecting, characterizing, and monitoring eye diseases. METHOD The 2020 PRISMA guideline was used to structure a systematic review of research on various eye conditions using machine learning (ML) or deep learning (DL) techniques. A thorough search across IEEE, PubMed, Web of Science, and Scopus databases yielded 1787 publications, of which 1136 remained after removing duplicates. Subsequent exclusion of conference papers, review papers, and non-open-access articles reduced the selection to 511 articles. Further scrutiny led to the exclusion of 435 more articles due to lower-quality indexing or irrelevance, resulting in 76 journal articles for the review. RESULTS During our investigation, we found that a major challenge for ML-based decision support is the abundance of features and the determination of their significance. In contrast, DL-based decision support is characterized by a plug-and-play nature rather than relying on a trial-and-error approach. Furthermore, we observed that pre-trained networks are practical and especially useful when working on complex images such as OCT. Consequently, pre-trained deep networks were frequently utilized for classification tasks. Currently, medical decision support aims to reduce the workload of ophthalmologists and retina specialists during routine tasks. In the future, it might be possible to create continuous learning systems that can predict ocular pathologies by identifying subtle changes in OCT images.
Collapse
Affiliation(s)
- Muhammed Halil Akpinar
- Department of Electronics and Automation, Vocational School of Technical Sciences, Istanbul University-Cerrahpasa, Istanbul, Turkey
| | - Abdulkadir Sengur
- Electrical-Electronics Engineering Department, Technology Faculty, Firat University, Elazig, Turkey.
| | - Oliver Faust
- School of Computing and Information Science, Anglia Ruskin University Cambridge Campus, United Kingdom
| | - Louis Tong
- Singapore Eye Research Institute, Singapore, Singapore
| | - Filippo Molinari
- Biolab, PolitoBIOMedLab, Department of Electronics and Telecommunications, Politecnico di Torino, Turin, Italy
| | - U Rajendra Acharya
- School of Mathematics, Physics and Computing, University of Southern Queensland, Springfield, Australia
| |
Collapse
|
2
|
Bacon EJ, He D, Achi NAD, Wang L, Li H, Yao-Digba PDZ, Monkam P, Qi S. Neuroimage analysis using artificial intelligence approaches: a systematic review. Med Biol Eng Comput 2024; 62:2599-2627. [PMID: 38664348 DOI: 10.1007/s11517-024-03097-w] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/10/2023] [Accepted: 04/14/2024] [Indexed: 08/18/2024]
Abstract
In the contemporary era, artificial intelligence (AI) has undergone a transformative evolution, exerting a profound influence on neuroimaging data analysis. This development has significantly elevated our comprehension of intricate brain functions. This study investigates the ramifications of employing AI techniques on neuroimaging data, with a specific objective to improve diagnostic capabilities and contribute to the overall progress of the field. A systematic search was conducted in prominent scientific databases, including PubMed, IEEE Xplore, and Scopus, meticulously curating 456 relevant articles on AI-driven neuroimaging analysis spanning from 2013 to 2023. To maintain rigor and credibility, stringent inclusion criteria, quality assessments, and precise data extraction protocols were consistently enforced throughout this review. Following a rigorous selection process, 104 studies were selected for review, focusing on diverse neuroimaging modalities with an emphasis on mental and neurological disorders. Among these, 19.2% addressed mental illness, and 80.7% focused on neurological disorders. It is found that the prevailing clinical tasks are disease classification (58.7%) and lesion segmentation (28.9%), whereas image reconstruction constituted 7.3%, and image regression and prediction tasks represented 9.6%. AI-driven neuroimaging analysis holds tremendous potential, transforming both research and clinical applications. Machine learning and deep learning algorithms outperform traditional methods, reshaping the field significantly.
Collapse
Affiliation(s)
- Eric Jacob Bacon
- College of Medicine and Biological Information Engineering, Northeastern University, Shenyang, China
- Key Laboratory of Intelligent Computing in Medical Image, Ministry of Education, Northeastern University, Shenyang, China
| | - Dianning He
- College of Medicine and Biological Information Engineering, Northeastern University, Shenyang, China
| | | | - Lanbo Wang
- Department of Radiology, Shengjing Hospital of China Medical University, Shenyang, China
| | - Han Li
- Department of Neurosurgery, Shengjing Hospital of China Medical University, Shenyang, China
| | | | - Patrice Monkam
- College of Medicine and Biological Information Engineering, Northeastern University, Shenyang, China.
- Key Laboratory of Intelligent Computing in Medical Image, Ministry of Education, Northeastern University, Shenyang, China.
| | - Shouliang Qi
- College of Medicine and Biological Information Engineering, Northeastern University, Shenyang, China.
- Key Laboratory of Intelligent Computing in Medical Image, Ministry of Education, Northeastern University, Shenyang, China.
| |
Collapse
|
3
|
Guo R, Wei J, Sun L, Yu B, Chang G, Liu D, Zhang S, Yao Z, Xu M, Bu L. A survey on advancements in image-text multimodal models: From general techniques to biomedical implementations. Comput Biol Med 2024; 178:108709. [PMID: 38878398 DOI: 10.1016/j.compbiomed.2024.108709] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/27/2023] [Revised: 06/01/2024] [Accepted: 06/03/2024] [Indexed: 07/24/2024]
Abstract
With the significant advancements of Large Language Models (LLMs) in the field of Natural Language Processing (NLP), the development of image-text multimodal models has garnered widespread attention. Current surveys on image-text multimodal models mainly focus on representative models or application domains, but lack a review on how general technical models influence the development of domain-specific models, which is crucial for domain researchers. Based on this, this paper first reviews the technological evolution of image-text multimodal models, from early explorations of feature space to visual language encoding structures, and then to the latest large model architectures. Next, from the perspective of technological evolution, we explain how the development of general image-text multimodal technologies promotes the progress of multimodal technologies in the biomedical field, as well as the importance and complexity of specific datasets in the biomedical domain. Then, centered on the tasks of image-text multimodal models, we analyze their common components and challenges. After that, we summarize the architecture, components, and data of general image-text multimodal models, and introduce the applications and improvements of image-text multimodal models in the biomedical field. Finally, we categorize the challenges faced in the development and application of general models into external factors and intrinsic factors, further refining them into 2 external factors and 5 intrinsic factors, and propose targeted solutions, providing guidance for future research directions. For more details and data, please visit our GitHub page: https://github.com/i2vec/A-survey-on-image-text-multimodal-models.
Collapse
Affiliation(s)
- Ruifeng Guo
- Shenyang Institute of Computing Technology, Chinese Academy of Sciences, Shenyang, 110168, China; University of Chinese Academy of Sciences, Beijing, 100049, China.
| | - Jingxuan Wei
- Shenyang Institute of Computing Technology, Chinese Academy of Sciences, Shenyang, 110168, China; University of Chinese Academy of Sciences, Beijing, 100049, China.
| | - Linzhuang Sun
- Shenyang Institute of Computing Technology, Chinese Academy of Sciences, Shenyang, 110168, China; University of Chinese Academy of Sciences, Beijing, 100049, China.
| | - Bihui Yu
- Shenyang Institute of Computing Technology, Chinese Academy of Sciences, Shenyang, 110168, China; University of Chinese Academy of Sciences, Beijing, 100049, China.
| | - Guiyong Chang
- Shenyang Institute of Computing Technology, Chinese Academy of Sciences, Shenyang, 110168, China; University of Chinese Academy of Sciences, Beijing, 100049, China.
| | - Dawei Liu
- Shenyang Institute of Computing Technology, Chinese Academy of Sciences, Shenyang, 110168, China; University of Chinese Academy of Sciences, Beijing, 100049, China.
| | - Sibo Zhang
- Shenyang Institute of Computing Technology, Chinese Academy of Sciences, Shenyang, 110168, China; University of Chinese Academy of Sciences, Beijing, 100049, China.
| | - Zhengbing Yao
- Shenyang Institute of Computing Technology, Chinese Academy of Sciences, Shenyang, 110168, China; University of Chinese Academy of Sciences, Beijing, 100049, China.
| | - Mingjun Xu
- Shenyang Institute of Computing Technology, Chinese Academy of Sciences, Shenyang, 110168, China; University of Chinese Academy of Sciences, Beijing, 100049, China.
| | - Liping Bu
- Shenyang Institute of Computing Technology, Chinese Academy of Sciences, Shenyang, 110168, China; University of Chinese Academy of Sciences, Beijing, 100049, China.
| |
Collapse
|
4
|
Abbasian Ardakani A, Airom O, Khorshidi H, Bureau NJ, Salvi M, Molinari F, Acharya UR. Interpretation of Artificial Intelligence Models in Healthcare: A Pictorial Guide for Clinicians. JOURNAL OF ULTRASOUND IN MEDICINE : OFFICIAL JOURNAL OF THE AMERICAN INSTITUTE OF ULTRASOUND IN MEDICINE 2024. [PMID: 39032010 DOI: 10.1002/jum.16524] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Received: 04/13/2024] [Revised: 06/19/2024] [Accepted: 07/01/2024] [Indexed: 07/22/2024]
Abstract
Artificial intelligence (AI) models can play a more effective role in managing patients with the explosion of digital health records available in the healthcare industry. Machine-learning (ML) and deep-learning (DL) techniques are two methods used to develop predictive models that serve to improve the clinical processes in the healthcare industry. These models are also implemented in medical imaging machines to empower them with an intelligent decision system to aid physicians in their decisions and increase the efficiency of their routine clinical practices. The physicians who are going to work with these machines need to have an insight into what happens in the background of the implemented models and how they work. More importantly, they need to be able to interpret their predictions, assess their performance, and compare them to find the one with the best performance and fewer errors. This review aims to provide an accessible overview of key evaluation metrics for physicians without AI expertise. In this review, we developed four real-world diagnostic AI models (two ML and two DL models) for breast cancer diagnosis using ultrasound images. Then, 23 of the most commonly used evaluation metrics were reviewed uncomplicatedly for physicians. Finally, all metrics were calculated and used practically to interpret and evaluate the outputs of the models. Accessible explanations and practical applications empower physicians to effectively interpret, evaluate, and optimize AI models to ensure safety and efficacy when integrated into clinical practice.
Collapse
Affiliation(s)
- Ali Abbasian Ardakani
- Department of Radiology Technology, School of Allied Medical Sciences, Shahid Beheshti University of Medical Sciences, Tehran, Iran
| | - Omid Airom
- Department of Mathematics, University of Padova, Padova, Italy
| | - Hamid Khorshidi
- Department of Information Engineering, University of Padova, Padova, Italy
| | - Nathalie J Bureau
- Department of Radiology, Centre Hospitalier de l'Université de Montréal, Montreal, Quebec, Canada
| | - Massimo Salvi
- Biolab, PolitoBIOMedLab, Department of Electronics and Telecommunications, Politecnico di Torino, Turin, Italy
| | - Filippo Molinari
- Biolab, PolitoBIOMedLab, Department of Electronics and Telecommunications, Politecnico di Torino, Turin, Italy
| | - U Rajendra Acharya
- School of Mathematics, Physics and Computing, University of Southern Queensland, Springfield, Queensland, Australia
- Centre for Health Research, University of Southern Queensland, Springfield, Queensland, Australia
| |
Collapse
|
5
|
Shi S, Fu L, Yi J, Yang Z, Zhang X, Deng Y, Wang W, Wu C, Zhao W, Hou T, Zeng X, Lyu A, Cao D. ChemFH: an integrated tool for screening frequent false positives in chemical biology and drug discovery. Nucleic Acids Res 2024; 52:W439-W449. [PMID: 38783035 PMCID: PMC11223804 DOI: 10.1093/nar/gkae424] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/06/2024] [Revised: 04/25/2024] [Accepted: 05/10/2024] [Indexed: 05/25/2024] Open
Abstract
High-throughput screening rapidly tests an extensive array of chemical compounds to identify hit compounds for specific biological targets in drug discovery. However, false-positive results disrupt hit compound screening, leading to wastage of time and resources. To address this, we propose ChemFH, an integrated online platform facilitating rapid virtual evaluation of potential false positives, including colloidal aggregators, spectroscopic interference compounds, firefly luciferase inhibitors, chemical reactive compounds, promiscuous compounds, and other assay interferences. By leveraging a dataset containing 823 391 compounds, we constructed high-quality prediction models using multi-task directed message-passing network (DMPNN) architectures combining uncertainty estimation, yielding an average AUC value of 0.91. Furthermore, ChemFH incorporated 1441 representative alert substructures derived from the collected data and ten commonly used frequent hitter screening rules. ChemFH was validated with an external set of 75 compounds. Subsequently, the virtual screening capability of ChemFH was successfully confirmed through its application to five virtual screening libraries. Furthermore, ChemFH underwent additional validation on two natural products and FDA-approved drugs, yielding reliable and accurate results. ChemFH is a comprehensive, reliable, and computationally efficient screening pipeline that facilitates the identification of true positive results in assays, contributing to enhanced efficiency and success rates in drug discovery. ChemFH is freely available via https://chemfh.scbdd.com/.
Collapse
Affiliation(s)
- Shaohua Shi
- Xiangya School of Pharmaceutical Sciences, Central South University, Changsha, Hunan 410013, P.R. China
- School of Chinese Medicine, Hong Kong Baptist University, Kowloon, Hong Kong SAR, 999077, P.R. China
| | - Li Fu
- Xiangya School of Pharmaceutical Sciences, Central South University, Changsha, Hunan 410013, P.R. China
| | - Jiacai Yi
- School of Computer Science, National University of Defense Technology, Changsha, Hunan 410073, P.R. China
| | - Ziyi Yang
- Xiangya School of Pharmaceutical Sciences, Central South University, Changsha, Hunan 410013, P.R. China
| | - Xiaochen Zhang
- School of Information Technology, Shangqiu Normal University, Shangqiu, Henan 476000, P.R. China
| | - Youchao Deng
- Xiangya School of Pharmaceutical Sciences, Central South University, Changsha, Hunan 410013, P.R. China
| | - Wenxuan Wang
- Xiangya School of Pharmaceutical Sciences, Central South University, Changsha, Hunan 410013, P.R. China
| | - Chengkun Wu
- School of Computer Science, National University of Defense Technology, Changsha, Hunan 410073, P.R. China
| | - Wentao Zhao
- School of Computer Science, National University of Defense Technology, Changsha, Hunan 410073, P.R. China
| | - Tingjun Hou
- College of Pharmaceutical Sciences, Zhejiang University, Hangzhou, Zhejiang 310058, P.R. China
| | - Xiangxiang Zeng
- College of Computer Science and Electronic Engineering, Hunan University, Changsha, Hunan 410082, P.R. China
| | - Aiping Lyu
- School of Chinese Medicine, Hong Kong Baptist University, Kowloon, Hong Kong SAR, 999077, P.R. China
| | - Dongsheng Cao
- Xiangya School of Pharmaceutical Sciences, Central South University, Changsha, Hunan 410013, P.R. China
| |
Collapse
|
6
|
Li J, Tang T, Wu E, Zhao J, Zong H, Wu R, Feng W, Zhang K, Wang D, Qin Y, Shen Z, Qin Y, Ren S, Zhan C, Yang L, Wei Q, Shen B. RARPKB: a knowledge-guide decision support platform for personalized robot-assisted surgery in prostate cancer. Int J Surg 2024; 110:3412-3424. [PMID: 38498357 PMCID: PMC11175739 DOI: 10.1097/js9.0000000000001290] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/10/2023] [Accepted: 02/22/2024] [Indexed: 03/20/2024]
Abstract
BACKGROUND Robot-assisted radical prostatectomy (RARP) has emerged as a pivotal surgical intervention for the treatment of prostate cancer (PCa). However, the complexity of clinical cases, heterogeneity of PCa, and limitations in physician expertise pose challenges to rational decision-making in RARP. To address these challenges, the authors aimed to organize the knowledge of previously complex cohorts and establish an online platform named the RARP knowledge base (RARPKB) to provide reference evidence for personalized treatment plans. MATERIALS AND METHODS PubMed searches over the past two decades were conducted to identify publications describing RARP. The authors collected, classified, and structured surgical details, patient information, surgical data, and various statistical results from the literature. A knowledge-guided decision-support tool was established using MySQL, DataTable, ECharts, and JavaScript. ChatGPT-4 and two assessment scales were used to validate and compare the platform. RESULTS The platform comprised 583 studies, 1589 cohorts, 1 911 968 patients, and 11 986 records, resulting in 54 834 data entries. The knowledge-guided decision support tool provide personalized surgical plan recommendations and potential complications on the basis of patients' baseline and surgical information. Compared with ChatGPT-4, RARPKB outperformed in authenticity (100% vs. 73%), matching (100% vs. 53%), personalized recommendations (100% vs. 20%), matching of patients (100% vs. 0%), and personalized recommendations for complications (100% vs. 20%). Postuse, the average System Usability Scale score was 88.88±15.03, and the Net Promoter Score of RARPKB was 85. The knowledge base is available at: http://rarpkb.bioinf.org.cn . CONCLUSIONS The authors introduced the pioneering RARPKB, the first knowledge base for robot-assisted surgery, with an emphasis on PCa. RARPKB can assist in personalized and complex surgical planning for PCa to improve its efficacy. RARPKB provides a reference for the future applications of artificial intelligence in clinical practice.
Collapse
Affiliation(s)
- Jiakun Li
- Department of Urology, West China Hospital, Sichuan University
- Institutes for Systems Genetics, Frontiers Science Center for Disease-related Molecular Network, West China Hospital, Sichuan University
| | - Tong Tang
- Institutes for Systems Genetics, Frontiers Science Center for Disease-related Molecular Network, West China Hospital, Sichuan University
- Department of Computer Science and Information Technologies, Elviña Campus, University of A Coruña, A Coruña, Spain
| | - Erman Wu
- Institutes for Systems Genetics, Frontiers Science Center for Disease-related Molecular Network, West China Hospital, Sichuan University
| | - Jing Zhao
- Institutes for Systems Genetics, Frontiers Science Center for Disease-related Molecular Network, West China Hospital, Sichuan University
| | - Hui Zong
- Institutes for Systems Genetics, Frontiers Science Center for Disease-related Molecular Network, West China Hospital, Sichuan University
| | - Rongrong Wu
- Institutes for Systems Genetics, Frontiers Science Center for Disease-related Molecular Network, West China Hospital, Sichuan University
| | - Weizhe Feng
- Institutes for Systems Genetics, Frontiers Science Center for Disease-related Molecular Network, West China Hospital, Sichuan University
| | - Ke Zhang
- Institutes for Systems Genetics, Frontiers Science Center for Disease-related Molecular Network, West China Hospital, Sichuan University
- Chengdu Aixam Medical Technology Co. Ltd, Chengdu
| | - Dongyue Wang
- Department of Ophthalmology, West China Hospital, Sichuan University
| | - Yawen Qin
- Clinical Medical College, Southwest Medical University, Luzhou, Sichuan Province
| | | | - Yi Qin
- Institutes for Systems Genetics, Frontiers Science Center for Disease-related Molecular Network, West China Hospital, Sichuan University
| | - Shumin Ren
- Institutes for Systems Genetics, Frontiers Science Center for Disease-related Molecular Network, West China Hospital, Sichuan University
- Department of Computer Science and Information Technologies, Elviña Campus, University of A Coruña, A Coruña, Spain
| | - Chaoying Zhan
- Institutes for Systems Genetics, Frontiers Science Center for Disease-related Molecular Network, West China Hospital, Sichuan University
| | - Lu Yang
- Department of Urology, West China Hospital, Sichuan University
| | - Qiang Wei
- Department of Urology, West China Hospital, Sichuan University
| | - Bairong Shen
- Institutes for Systems Genetics, Frontiers Science Center for Disease-related Molecular Network, West China Hospital, Sichuan University
| |
Collapse
|
7
|
Scalco E, Pozzi S, Rizzo G, Lanzarone E. Uncertainty quantification in multi-class segmentation: Comparison between Bayesian and non-Bayesian approaches in a clinical perspective. Med Phys 2024. [PMID: 38808956 DOI: 10.1002/mp.17189] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/12/2024] [Revised: 04/17/2024] [Accepted: 05/12/2024] [Indexed: 05/30/2024] Open
Abstract
BACKGROUND Automatic segmentation techniques based on Convolutional Neural Networks (CNNs) are widely adopted to automatically identify any structure of interest from a medical image, as they are not time consuming and not subject to high intra- and inter-operator variability. However, the adoption of these approaches in clinical practice is slowed down by some factors, such as the difficulty in providing an accurate quantification of their uncertainty. PURPOSE This work aims to evaluate the uncertainty quantification provided by two Bayesian and two non-Bayesian approaches for a multi-class segmentation problem, and to compare the risk propensity among these approaches, considering CT images of patients affected by renal cancer (RC). METHODS Four uncertainty quantification approaches were implemented in this work, based on a benchmark CNN currently employed in medical image segmentation: two Bayesian CNNs with different regularizations (Dropout and DropConnect), named BDR and BDC, an ensemble method (Ens) and a test-time augmentation (TTA) method. They were compared in terms of segmentation accuracy, using the Dice score, uncertainty quantification, using the ratio of correct-certain pixels (RCC) and incorrect-uncertain pixels (RIU), and with respect to inter-observer variability in manual segmentation. They were trained with the Kidney and Kidney Tumor Segmentation Challenge launched in 2021 (Kits21), for which multi-class segmentations of kidney, RC, and cyst on 300 CT volumes are available. Moreover, they were tested considering this and other two public renal CT datasets. RESULTS Accuracy results achieved large differences across the structures of interest for all approaches, with an average Dice score of 0.92, 0.58, and 0.21 for kidney, tumor, and cyst, respectively. In terms of uncertainties, TTA provided the highest uncertainty, followed by Ens and BDC, whereas BDR provided the lowest, and minimized the number of incorrect certain pixels worse than the other approaches. Again, large differences were seen across the three structures in terms of RCC and RIU. These metrics were associated with different risk propensity, as BDR was the most risk-taking approach, able to provide higher accuracy in its prediction, but failing to assign uncertainty on incorrect segmentation in every case. The other three approaches were more conservative, providing large uncertainty regions, with the drawback of giving alert also on correct areas. Finally, the analysis of the inter-observer segmentation variability showed a significant variation among the four approaches on the external dataset, with BDR reporting the lowest agreement (Dice = 0.82), and TTA obtaining the highest score (Dice = 0.94). CONCLUSIONS Our outcomes highlight the importance of quantifying the segmentation uncertainty and that decision-makers can choose the approach most in line with the risk propensity degree required by the application and their policy.
Collapse
Affiliation(s)
- Elisa Scalco
- Institute of Biomedical Technologies (ITB), National Research Council (CNR), Segrate, Milan, Italy
| | - Silvia Pozzi
- Department of Management, Information and Production Engineering, University of Bergamo, Bergamo, Italy
| | - Giovanna Rizzo
- Institute Of Intelligent Industrial Technologies and Systems (STIIMA), National Research Council (CNR), Milan, Italy
| | - Ettore Lanzarone
- Department of Management, Information and Production Engineering, University of Bergamo, Bergamo, Italy
| |
Collapse
|
8
|
Wahid KA, Kaffey ZY, Farris DP, Humbert-Vidan L, Moreno AC, Rasmussen M, Ren J, Naser MA, Netherton TJ, Korreman S, Balakrishnan G, Fuller CD, Fuentes D, Dohopolski MJ. Artificial Intelligence Uncertainty Quantification in Radiotherapy Applications - A Scoping Review. MEDRXIV : THE PREPRINT SERVER FOR HEALTH SCIENCES 2024:2024.05.13.24307226. [PMID: 38798581 PMCID: PMC11118597 DOI: 10.1101/2024.05.13.24307226] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/29/2024]
Abstract
Background/purpose The use of artificial intelligence (AI) in radiotherapy (RT) is expanding rapidly. However, there exists a notable lack of clinician trust in AI models, underscoring the need for effective uncertainty quantification (UQ) methods. The purpose of this study was to scope existing literature related to UQ in RT, identify areas of improvement, and determine future directions. Methods We followed the PRISMA-ScR scoping review reporting guidelines. We utilized the population (human cancer patients), concept (utilization of AI UQ), context (radiotherapy applications) framework to structure our search and screening process. We conducted a systematic search spanning seven databases, supplemented by manual curation, up to January 2024. Our search yielded a total of 8980 articles for initial review. Manuscript screening and data extraction was performed in Covidence. Data extraction categories included general study characteristics, RT characteristics, AI characteristics, and UQ characteristics. Results We identified 56 articles published from 2015-2024. 10 domains of RT applications were represented; most studies evaluated auto-contouring (50%), followed by image-synthesis (13%), and multiple applications simultaneously (11%). 12 disease sites were represented, with head and neck cancer being the most common disease site independent of application space (32%). Imaging data was used in 91% of studies, while only 13% incorporated RT dose information. Most studies focused on failure detection as the main application of UQ (60%), with Monte Carlo dropout being the most commonly implemented UQ method (32%) followed by ensembling (16%). 55% of studies did not share code or datasets. Conclusion Our review revealed a lack of diversity in UQ for RT applications beyond auto-contouring. Moreover, there was a clear need to study additional UQ methods, such as conformal prediction. Our results may incentivize the development of guidelines for reporting and implementation of UQ in RT.
Collapse
Affiliation(s)
- Kareem A. Wahid
- Department of Imaging Physics, The University of Texas MD Anderson Cancer Center, Houston, Texas, USA
- Department of Radiation Oncology, The University of Texas MD Anderson Cancer Center, Houston, Texas, USA
| | - Zaphanlene Y. Kaffey
- Department of Radiation Oncology, The University of Texas MD Anderson Cancer Center, Houston, Texas, USA
| | - David P. Farris
- Research Medical Library, The University of Texas MD Anderson Cancer Center, Houston, Texas, USA
| | - Laia Humbert-Vidan
- Department of Radiation Oncology, The University of Texas MD Anderson Cancer Center, Houston, Texas, USA
| | - Amy C. Moreno
- Department of Radiation Oncology, The University of Texas MD Anderson Cancer Center, Houston, Texas, USA
| | | | - Jintao Ren
- Department of Oncology, Aarhus University Hospital, Denmark
| | - Mohamed A. Naser
- Department of Radiation Oncology, The University of Texas MD Anderson Cancer Center, Houston, Texas, USA
| | - Tucker J. Netherton
- Department of Radiation Physics, University of Texas MD Anderson Cancer Center, Houston, TX, USA
| | - Stine Korreman
- Department of Oncology, Aarhus University Hospital, Denmark
| | | | - Clifton D. Fuller
- Department of Radiation Oncology, The University of Texas MD Anderson Cancer Center, Houston, Texas, USA
| | - David Fuentes
- Department of Imaging Physics, The University of Texas MD Anderson Cancer Center, Houston, Texas, USA
| | - Michael J. Dohopolski
- Department of Radiation Oncology, The University of Texas Southwestern Medical Center, Dallas, Texas, USA
| |
Collapse
|
9
|
Tuncer T, Dogan S, Baygin M, Barua PD, Palmer EE, March S, Ciaccio EJ, Tan RS, Acharya UR. FLP: Factor lattice pattern-based automated detection of Parkinson's disease and specific language impairment using recorded speech. Comput Biol Med 2024; 173:108280. [PMID: 38547655 DOI: 10.1016/j.compbiomed.2024.108280] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/08/2023] [Revised: 02/18/2024] [Accepted: 03/09/2024] [Indexed: 04/17/2024]
Abstract
BACKGROUND Timely detection of neurodevelopmental and neurological conditions is crucial for early intervention. Specific Language Impairment (SLI) in children and Parkinson's disease (PD) manifests in speech disturbances that may be exploited for diagnostic screening using recorded speech signals. We were motivated to develop an accurate yet computationally lightweight model for speech-based detection of SLI and PD, employing novel feature engineering techniques to mimic the adaptable dynamic weight assignment network capability of deep learning architectures. MATERIALS AND METHODS In this research, we have introduced an advanced feature engineering model incorporating a novel feature extraction function, the Factor Lattice Pattern (FLP), which is a quantum-inspired method and uses a superposition-like mechanism, making it dynamic in nature. The FLP encompasses eight distinct patterns, from which the most appropriate pattern was discerned based on the data structure. Through the implementation of the FLP, we automatically extracted signal-specific textural features. Additionally, we developed a new feature engineering model to assess the efficacy of the FLP. This model is self-organizing, producing nine potential results and subsequently choosing the optimal one. Our speech classification framework consists of (1) feature extraction using the proposed FLP and a statistical feature extractor; (2) feature selection employing iterative neighborhood component analysis and an intersection-based feature selector; (3) classification via support vector machine and k-nearest neighbors; and (4) outcome determination using combinational majority voting to select the most favorable results. RESULTS To validate the classification capabilities of our proposed feature engineering model, designed to automatically detect PD and SLI, we employed three speech datasets of PD and SLI patients. Our presented FLP-centric model achieved classification accuracy of more than 95% and 99.79% for all PD and SLI datasets, respectively. CONCLUSIONS Our results indicate that the proposed model is an accurate alternative to deep learning models in classifying neurological conditions using speech signals.
Collapse
Affiliation(s)
- Turker Tuncer
- Department of Digital Forensics Engineering, Technology Faculty, Firat University, Elazig, Turkey.
| | - Sengul Dogan
- Department of Digital Forensics Engineering, Technology Faculty, Firat University, Elazig, Turkey.
| | - Mehmet Baygin
- Department of Computer Engineering, Faculty of Engineering and Architecture, Erzurum Technical University, Erzurum, Turkey.
| | - Prabal Datta Barua
- School of Business (Information System), University of Southern Queensland, Australia.
| | - Elizabeth Emma Palmer
- Centre of Clinical Genetics, Sydney Children's Hospitals Network, Randwick, 2031, Australia; School of Women's and Children's Health, University of New South Wales, Randwick, 2031, Australia.
| | - Sonja March
- School of Psychology and Counselling and Centre for Health Research, University of Southern Queensland, Springfield, Australia.
| | - Edward J Ciaccio
- Department of Medicine, Columbia University Irving Medical Center, USA.
| | - Ru-San Tan
- Department of Cardiology, National Heart Centre Singapore, Singapore; Duke-NUS Medical School, Singapore
| | - U Rajendra Acharya
- School of Mathematics, Physics and Computing, University of Southern Queensland, Springfield, Australia; Centre for Health Research, University of Southern Queensland, Australia.
| |
Collapse
|
10
|
Yang X, Zheng Y, Mei C, Jiang G, Tian B, Wang L. UGLS: an uncertainty guided deep learning strategy for accurate image segmentation. Front Physiol 2024; 15:1362386. [PMID: 38651048 PMCID: PMC11033460 DOI: 10.3389/fphys.2024.1362386] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/05/2024] [Accepted: 03/26/2024] [Indexed: 04/25/2024] Open
Abstract
Accurate image segmentation plays a crucial role in computer vision and medical image analysis. In this study, we developed a novel uncertainty guided deep learning strategy (UGLS) to enhance the performance of an existing neural network (i.e., U-Net) in segmenting multiple objects of interest from images with varying modalities. In the developed UGLS, a boundary uncertainty map was introduced for each object based on its coarse segmentation (obtained by the U-Net) and then combined with input images for the fine segmentation of the objects. We validated the developed method by segmenting optic cup (OC) regions from color fundus images and left and right lung regions from Xray images. Experiments on public fundus and Xray image datasets showed that the developed method achieved a average Dice Score (DS) of 0.8791 and a sensitivity (SEN) of 0.8858 for the OC segmentation, and 0.9605, 0.9607, 0.9621, and 0.9668 for the left and right lung segmentation, respectively. Our method significantly improved the segmentation performance of the U-Net, making it comparable or superior to five sophisticated networks (i.e., AU-Net, BiO-Net, AS-Net, Swin-Unet, and TransUNet).
Collapse
Affiliation(s)
- Xiaoguo Yang
- Wenzhou People’s Hospital, The Third Affiliated Hospital of Shanghai University, Wenzhou, China
| | - Yanyan Zheng
- Wenzhou People’s Hospital, The Third Affiliated Hospital of Shanghai University, Wenzhou, China
| | - Chenyang Mei
- School of Ophthalmology and Optometry, Eye Hospital, Wenzhou Medical University, Wenzhou, China
| | - Gaoqiang Jiang
- School of Ophthalmology and Optometry, Eye Hospital, Wenzhou Medical University, Wenzhou, China
| | - Bihan Tian
- School of Ophthalmology and Optometry, Eye Hospital, Wenzhou Medical University, Wenzhou, China
| | - Lei Wang
- School of Ophthalmology and Optometry, Eye Hospital, Wenzhou Medical University, Wenzhou, China
| |
Collapse
|
11
|
Baygin M, Barua PD, Dogan S, Tuncer T, Hong TJ, March S, Tan RS, Molinari F, Acharya UR. Automated anxiety detection using probabilistic binary pattern with ECG signals. COMPUTER METHODS AND PROGRAMS IN BIOMEDICINE 2024; 247:108076. [PMID: 38422891 DOI: 10.1016/j.cmpb.2024.108076] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 11/13/2023] [Revised: 01/22/2024] [Accepted: 02/09/2024] [Indexed: 03/02/2024]
Abstract
BACKGROUND AND AIM Anxiety disorder is common; early diagnosis is crucial for management. Anxiety can induce physiological changes in the brain and heart. We aimed to develop an efficient and accurate handcrafted feature engineering model for automated anxiety detection using ECG signals. MATERIALS AND METHODS We studied open-access electrocardiography (ECG) data of 19 subjects collected via wearable sensors while they were shown videos that might induce anxiety. Using the Hamilton Anxiety Rating Scale, subjects are categorized into normal, light anxiety, moderate anxiety, and severe anxiety groups. ECGs were divided into non-overlapping 4- (Case 1), 5- (Case 2), and 6-second (Case 3) segments for analysis. We proposed a self-organized dynamic pattern-based feature extraction function-probabilistic binary pattern (PBP)-in which patterns within the function were determined by the probabilities of the input signal-dependent values. This was combined with tunable q-factor wavelet transform to facilitate multileveled generation of feature vectors in both spatial and frequency domains. Neighborhood component analysis and Chi2 functions were used to select features and reduce data dimensionality. Shallow k-nearest neighbors and support vector machine classifiers were used to calculate four (=2 × 2) classifier-wise results per input signal. From the latter, novel self-organized combinational majority voting was applied to calculate an additional five voted results. The optimal final model outcome was chosen from among the nine (classifier-wise and voted) results using a greedy algorithm. RESULTS Our model achieved classification accuracies of over 98.5 % for all three cases. Ablation studies confirmed the incremental accuracy of PBP-based feature engineering over traditional local binary pattern feature extraction. CONCLUSIONS The results demonstrated the feasibility and accuracy of our PBP-based feature engineering model for anxiety classification using ECG signals.
Collapse
Affiliation(s)
- Mehmet Baygin
- Department of Computer Engineering, Faculty of Engineering and Architecture, Erzurum Technical University, Erzurum, Turkey
| | - Prabal Datta Barua
- School of Business (Information System), University of Southern Queensland, Australia
| | - Sengul Dogan
- Department of Digital Forensics Engineering, College of Technology, Firat University, 23119, Elazig, Turkey.
| | - Turker Tuncer
- Department of Digital Forensics Engineering, College of Technology, Firat University, 23119, Elazig, Turkey
| | - Tan Jen Hong
- Data Science and Artificial Intelligence Lab, Singapore General Hospital, Singapore
| | - Sonja March
- Centre for Health Research, University of Southern Queensland, Australia; School of Psychology and Wellbeing, University of Southern Queensland, Australia
| | - Ru-San Tan
- Department of Cardiology, National Heart Centre Singapore, Singapore; Duke-NUS Medical School, Singapore
| | - Filippo Molinari
- Biolab, PolitoBIOMed Lab, Department of Electronics and Telecommunications, Politecnico di Torino, Corso Duca degli Abruzzi 24, 10129 Turin, Italy
| | - U Rajendra Acharya
- Centre for Health Research, University of Southern Queensland, Australia; School of Mathematics, Physics and Computing, University of Southern Queensland, Springfield, Australia
| |
Collapse
|
12
|
Zhong C, Darbandi M, Nassr M, Latifian A, Hosseinzadeh M, Jafari Navimipour N. A new cloud-based method for composition of healthcare services using deep reinforcement learning and Kalman filtering. Comput Biol Med 2024; 172:108152. [PMID: 38452470 DOI: 10.1016/j.compbiomed.2024.108152] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/24/2022] [Revised: 01/06/2024] [Accepted: 02/12/2024] [Indexed: 03/09/2024]
Abstract
Healthcare has significantly contributed to the well-being of individuals around the globe; nevertheless, further benefits could be derived from a more streamlined healthcare system without incurring additional costs. Recently, the main attributes of cloud computing, such as on-demand service, high scalability, and virtualization, have brought many benefits across many areas, especially in medical services. It is considered an important element in healthcare services, enhancing the performance and efficacy of the services. The current state of the healthcare industry requires the supply of healthcare products and services, increasing its viability for everyone involved. Developing new approaches for discovering and selecting healthcare services in the cloud has become more critical due to the rising popularity of these kinds of services. As a result of the diverse array of healthcare services, service composition enables the execution of intricate operations by integrating multiple services' functionalities into a single procedure. However, many methods in this field encounter several issues, such as high energy consumption, cost, and response time. This article introduces a novel layered method for selecting and evaluating healthcare services to find optimal service selection and composition solutions based on Deep Reinforcement Learning (Deep RL), Kalman filtering, and repeated training, addressing the aforementioned issues. The results revealed that the proposed method has achieved acceptable results in terms of availability, reliability, energy consumption, and response time when compared to other methods.
Collapse
Affiliation(s)
- Chongzhou Zhong
- School of Public Health & Management, Wenzhou Medical University, Wenzhou, Zhejiang, 325035, China
| | | | - Mohammad Nassr
- Communication Technology Engineering Department, Tartous University, Syria; Department of Mathematics and Natural Sciences, Gulf University for Science and Technology, Mishref Campus, Kuwait.
| | - Ahmad Latifian
- Department of Management, Faculty of Economics and Administrative Sciences, Ferdowsi University of Mashhad, Iran.
| | - Mehdi Hosseinzadeh
- Institute of Research and Development, Duy Tan University, Da Nang, Viet Nam; School of Medicine and Pharmacy, Duy Tan University, Da Nang, Viet Nam.
| | - Nima Jafari Navimipour
- Department of Computer Engineering, Faculty of Engineering and Natural Sciences, Kadir Has University, Istanbul, Turkey; Future Technology Research Center, National Yunlin University of Science and Technology, Douliou, Yunlin, 64002, Taiwan.
| |
Collapse
|
13
|
Gudigar A, Kadri NA, Raghavendra U, Samanth J, Maithri M, Inamdar MA, Prabhu MA, Hegde A, Salvi M, Yeong CH, Barua PD, Molinari F, Acharya UR. Automatic identification of hypertension and assessment of its secondary effects using artificial intelligence: A systematic review (2013-2023). Comput Biol Med 2024; 172:108207. [PMID: 38489986 DOI: 10.1016/j.compbiomed.2024.108207] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/27/2023] [Revised: 02/09/2024] [Accepted: 02/12/2024] [Indexed: 03/17/2024]
Abstract
Artificial Intelligence (AI) techniques are increasingly used in computer-aided diagnostic tools in medicine. These techniques can also help to identify Hypertension (HTN) in its early stage, as it is a global health issue. Automated HTN detection uses socio-demographic, clinical data, and physiological signals. Additionally, signs of secondary HTN can also be identified using various imaging modalities. This systematic review examines related work on automated HTN detection. We identify datasets, techniques, and classifiers used to develop AI models from clinical data, physiological signals, and fused data (a combination of both). Image-based models for assessing secondary HTN are also reviewed. The majority of the studies have primarily utilized single-modality approaches, such as biological signals (e.g., electrocardiography, photoplethysmography), and medical imaging (e.g., magnetic resonance angiography, ultrasound). Surprisingly, only a small portion of the studies (22 out of 122) utilized a multi-modal fusion approach combining data from different sources. Even fewer investigated integrating clinical data, physiological signals, and medical imaging to understand the intricate relationships between these factors. Future research directions are discussed that could build better healthcare systems for early HTN detection through more integrated modeling of multi-modal data sources.
Collapse
Affiliation(s)
- Anjan Gudigar
- Department of Instrumentation and Control Engineering, Manipal Institute of Technology, Manipal Academy of Higher Education, Manipal, 576104, India
| | - Nahrizul Adib Kadri
- Department of Biomedical Engineering, Faculty of Engineering, University of Malaya, Kuala Lumpur, 50603, Malaysia
| | - U Raghavendra
- Department of Instrumentation and Control Engineering, Manipal Institute of Technology, Manipal Academy of Higher Education, Manipal, 576104, India.
| | - Jyothi Samanth
- Department of Cardiovascular Technology, Manipal College of Health Professions, Manipal Academy of Higher Education, Manipal, 576104, India
| | - M Maithri
- Department of Mechatronics, Manipal Institute of Technology, Manipal Academy of Higher Education, Manipal, 576104, India
| | - Mahesh Anil Inamdar
- Department of Mechatronics, Manipal Institute of Technology, Manipal Academy of Higher Education, Manipal, 576104, India
| | - Mukund A Prabhu
- Department of Cardiology, Kasturba Medical College, Manipal Academy of Higher Education, Manipal, 576104, India
| | - Ajay Hegde
- Manipal Hospitals, Bengaluru, Karnataka, 560102, India
| | - Massimo Salvi
- Biolab, PolitoBIOMedLab, Department of Electronics and Telecommunications, Politecnicodi Torino, Turin, Italy
| | - Chai Hong Yeong
- School of Medicine, Faculty of Health and Medical Sciences, Taylor's University, 47500, Subang Jaya, Malaysia
| | - Prabal Datta Barua
- Cogninet Brain Team, Cogninet Australia, Sydney, NSW, 2010, Australia; School of Business (Information Systems), Faculty of Business, Education, Law & Arts, University of Southern Queensland, Toowoomba, QLD, 4350, Australia; Faculty of Engineering and Information Technology, University of Technology Sydney, Sydney, NSW, 2007, Australia
| | - Filippo Molinari
- Biolab, PolitoBIOMedLab, Department of Electronics and Telecommunications, Politecnicodi Torino, Turin, Italy
| | - U Rajendra Acharya
- School of Mathematics, Physics, and Computing, University of Southern Queensland, Springfield, QLD, 4300, Australia; Centre for Health Research, University of Southern Queensland, Toowoomba, QLD, 4350, Australia
| |
Collapse
|
14
|
Zamanian H, Shalbaf A, Zali MR, Khalaj AR, Dehghan P, Tabesh M, Hatami B, Alizadehsani R, Tan RS, Acharya UR. Application of artificial intelligence techniques for non-alcoholic fatty liver disease diagnosis: A systematic review (2005-2023). COMPUTER METHODS AND PROGRAMS IN BIOMEDICINE 2024; 244:107932. [PMID: 38008040 DOI: 10.1016/j.cmpb.2023.107932] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 08/19/2023] [Revised: 11/13/2023] [Accepted: 11/15/2023] [Indexed: 11/28/2023]
Abstract
BACKGROUND AND OBJECTIVES Non-alcoholic fatty liver disease (NAFLD) is a common liver disease with a rapidly growing incidence worldwide. For prognostication and therapeutic decisions, it is important to distinguish the pathological stages of NAFLD: steatosis, steatohepatitis, and liver fibrosis, which are definitively diagnosed on invasive biopsy. Non-invasive ultrasound (US) imaging, including US elastography technique, and clinical parameters can be used to diagnose and grade NAFLD and its complications. Artificial intelligence (AI) is increasingly being harnessed for developing NAFLD diagnostic models based on clinical, biomarker, or imaging data. In this work, we systemically reviewed the literature for AI-enabled NAFLD diagnostic models based on US (including elastography) and clinical (including serological) data. METHODS We performed a comprehensive search on Google Scholar, Scopus, and PubMed search engines for articles published between January 2005 and June 2023 related to AI models for NAFLD diagnosis based on US and/or clinical parameters using the following search terms: "non-alcoholic fatty liver disease", "non-alcoholic steatohepatitis", "deep learning", "machine learning", "artificial intelligence", "ultrasound imaging", "sonography", "clinical information". RESULTS We reviewed 64 published models that used either US (including elastography) or clinical data input to detect the presence of NAFLD, non-alcoholic steatohepatitis, and/or fibrosis, and in some cases, the severity of steatosis, inflammation, and/or fibrosis as well. The performances of the published models were summarized, and stratified by data input and algorithms used, which could be broadly divided into machine and deep learning approaches. CONCLUSION AI models based on US imaging and clinical data can reliably detect NAFLD and its complications, thereby reducing diagnostic costs and the need for invasive liver biopsy. The models offer advantages of efficiency, accuracy, and accessibility, and serve as virtual assistants for specialists to accelerate disease diagnosis and reduce treatment costs for patients and healthcare systems.
Collapse
Affiliation(s)
- H Zamanian
- Department of Biomedical Engineering and Medical Physics, School of Medicine, Shahid Beheshti University of Medical Sciences, Tehran, Iran
| | - A Shalbaf
- Department of Biomedical Engineering and Medical Physics, School of Medicine, Shahid Beheshti University of Medical Sciences, Tehran, Iran.
| | - M R Zali
- Gastroenterology and Liver Diseases Research Center, Research Institute for Gastroenterology and Liver Diseases, Shahid Beheshti University of Medical Sciences, Tehran, Iran
| | - A R Khalaj
- Tehran obesity treatment center, Department of Surgery, Faculty of Medicine, Shahed University, Tehran, Iran
| | - P Dehghan
- Department of Radiology, Imaging Department, Taleghani Hospital, Shahid Beheshti University of Medical Sciences, Tehran, Iran
| | - M Tabesh
- Cardiac Primary Prevention Research Center, Cardiovascular Diseases Research, Tehran University of Medical Sciences, Tehran, Iran
| | - B Hatami
- Gastroenterology and Liver Diseases Research Center, Research Institute for Gastroenterology and Liver Diseases, Shahid Beheshti University of Medical Sciences, Tehran, Iran
| | - R Alizadehsani
- Institute for Intelligent Systems Research and Innovation (IISRI), Deakin University, Waurn Ponds, VIC, Australia
| | - Ru-San Tan
- National Heart Centre Singapore, Singapore 169609, Singapore; Duke-NUS Medical School, Singapore
| | - U Rajendra Acharya
- School of Mathematics, Physics and Computing, University of Southern Queensland, Toowoomba, QLD, Australia; Centre for Health Research, University of Southern Queensland, Australia
| |
Collapse
|
15
|
Maruccio FC, Eppinga W, Laves MH, Navarro RF, Salvi M, Molinari F, Papaconstadopoulos P. Clinical assessment of deep learning-based uncertainty maps in lung cancer segmentation. Phys Med Biol 2024; 69:035007. [PMID: 38171012 DOI: 10.1088/1361-6560/ad1a26] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/24/2023] [Accepted: 01/03/2024] [Indexed: 01/05/2024]
Abstract
Objective. Prior to radiation therapy planning, accurate delineation of gross tumour volume (GTVs) and organs at risk (OARs) is crucial. In the current clinical practice, tumour delineation is performed manually by radiation oncologists, which is time-consuming and prone to large inter-observer variability. With the advent of deep learning (DL) models, automated contouring has become possible, speeding up procedures and assisting clinicians. However, these tools are currently used in the clinic mostly for contouring OARs, since these systems are not reliable yet for contouring GTVs. To improve the reliability of these systems, researchers have started exploring the topic of probabilistic neural networks. However, there is still limited knowledge of the practical implementation of such networks in real clinical settings.Approach. In this work, we developed a 3D probabilistic system that generates DL-based uncertainty maps for lung cancer CT segmentations. We employed the Monte Carlo (MC) dropout technique to generate probabilistic and uncertainty maps, while the model calibration was evaluated by using reliability diagrams. A clinical validation was conducted in collaboration with a radiation oncologist to qualitatively assess the value of the uncertainty estimates. We also proposed two novel metrics, namely mean uncertainty (MU) and relative uncertainty volume (RUV), as potential indicators for clinicians to assess the need for independent visual checks of the DL-based segmentation. Main results. Our study showed that uncertainty mapping effectively identified cases of under or over-contouring. Although the overconfidence of the model, a strong correlation was observed between the clinical opinion and MU metric. Moreover, both MU and RUV revealed high AUC values in discretising between low and high uncertainty cases.Significance. Our study is one of the first attempts to clinically validate uncertainty estimates in DL-based contouring. The two proposed metrics exhibited promising potential as indicators for clinicians to independently assess the quality of tumour delineation.
Collapse
Affiliation(s)
| | - Wietse Eppinga
- University Medical Centre Utrecht, Department of Radiotherapy, Heidelberglaan 100 Utrecht, 3584 CX, NL, The Netherlands
| | | | | | - Massimo Salvi
- Politecnico di Torino, Department of Electronics and Telecommunications, Corso Duca degli Abruzzi 24 Torino, Piemonte, I-10129, IT, Italy
| | - Filippo Molinari
- Politecnico di Torino, Department of Electronics and Telecommunications, Corso Duca degli Abruzzi 24 Torino, Piemonte, I-10129, IT, Italy
| | | |
Collapse
|
16
|
Fallahpoor M, Chakraborty S, Pradhan B, Faust O, Barua PD, Chegeni H, Acharya R. Deep learning techniques in PET/CT imaging: A comprehensive review from sinogram to image space. COMPUTER METHODS AND PROGRAMS IN BIOMEDICINE 2024; 243:107880. [PMID: 37924769 DOI: 10.1016/j.cmpb.2023.107880] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 07/31/2023] [Revised: 10/16/2023] [Accepted: 10/21/2023] [Indexed: 11/06/2023]
Abstract
Positron emission tomography/computed tomography (PET/CT) is increasingly used in oncology, neurology, cardiology, and emerging medical fields. The success stems from the cohesive information that hybrid PET/CT imaging offers, surpassing the capabilities of individual modalities when used in isolation for different malignancies. However, manual image interpretation requires extensive disease-specific knowledge, and it is a time-consuming aspect of physicians' daily routines. Deep learning algorithms, akin to a practitioner during training, extract knowledge from images to facilitate the diagnosis process by detecting symptoms and enhancing images. This acquired knowledge aids in supporting the diagnosis process through symptom detection and image enhancement. The available review papers on PET/CT imaging have a drawback as they either included additional modalities or examined various types of AI applications. However, there has been a lack of comprehensive investigation specifically focused on the highly specific use of AI, and deep learning, on PET/CT images. This review aims to fill that gap by investigating the characteristics of approaches used in papers that employed deep learning for PET/CT imaging. Within the review, we identified 99 studies published between 2017 and 2022 that applied deep learning to PET/CT images. We also identified the best pre-processing algorithms and the most effective deep learning models reported for PET/CT while highlighting the current limitations. Our review underscores the potential of deep learning (DL) in PET/CT imaging, with successful applications in lesion detection, tumor segmentation, and disease classification in both sinogram and image spaces. Common and specific pre-processing techniques are also discussed. DL algorithms excel at extracting meaningful features, and enhancing accuracy and efficiency in diagnosis. However, limitations arise from the scarcity of annotated datasets and challenges in explainability and uncertainty. Recent DL models, such as attention-based models, generative models, multi-modal models, graph convolutional networks, and transformers, are promising for improving PET/CT studies. Additionally, radiomics has garnered attention for tumor classification and predicting patient outcomes. Ongoing research is crucial to explore new applications and improve the accuracy of DL models in this rapidly evolving field.
Collapse
Affiliation(s)
- Maryam Fallahpoor
- Centre for Advanced Modelling and Geospatial Information Systems (CAMGIS), School of Civil and Environmental Engineering, University of Technology Sydney, Ultimo, NSW 2007, Australia
| | - Subrata Chakraborty
- Centre for Advanced Modelling and Geospatial Information Systems (CAMGIS), School of Civil and Environmental Engineering, University of Technology Sydney, Ultimo, NSW 2007, Australia; School of Science and Technology, Faculty of Science, Agriculture, Business and Law, University of New England, Armidale, NSW 2351, Australia
| | - Biswajeet Pradhan
- Centre for Advanced Modelling and Geospatial Information Systems (CAMGIS), School of Civil and Environmental Engineering, University of Technology Sydney, Ultimo, NSW 2007, Australia; Earth Observation Centre, Institute of Climate Change, Universiti Kebangsaan Malaysia, Bangi 43600, Malaysia.
| | - Oliver Faust
- School of Computing and Information Science, Anglia Ruskin University Cambridge Campus, United Kingdom
| | - Prabal Datta Barua
- School of Science and Technology, Faculty of Science, Agriculture, Business and Law, University of New England, Armidale, NSW 2351, Australia; Faculty of Engineering and Information Technology, University of Technology Sydney, Australia; School of Business (Information Systems), Faculty of Business, Education, Law & Arts, University of Southern Queensland, Australia
| | | | - Rajendra Acharya
- School of Mathematics, Physics and Computing, University of Southern Queensland, Toowoomba, QLD, Australia
| |
Collapse
|
17
|
Howard A, Aston S, Gerada A, Reza N, Bincalar J, Mwandumba H, Butterworth T, Hope W, Buchan I. Antimicrobial learning systems: an implementation blueprint for artificial intelligence to tackle antimicrobial resistance. Lancet Digit Health 2024; 6:e79-e86. [PMID: 38123255 DOI: 10.1016/s2589-7500(23)00221-2] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/29/2023] [Revised: 10/17/2023] [Accepted: 10/21/2023] [Indexed: 12/23/2023]
Abstract
The proliferation of various forms of artificial intelligence (AI) brings many opportunities to improve health care. AI models can harness complex evolving data, inform and augment human actions, and learn from health outcomes such as morbidity and mortality. The global public health challenge of antimicrobial resistance (AMR) needs large-scale optimisation of antimicrobial use and wider infection care, which could be enabled by carefully constructed AI models. As AI models become increasingly useful and robust, health-care systems remain challenging places for their deployment. An implementation gap exists between the promise of AI models and their use in patient and population care. Here, we outline an adaptive implementation and maintenance framework for AI models to improve antimicrobial use and infection care as a learning system. The roles of AMR problem identification, law and regulation, organisational support, data processing, and AI development, assessment, maintenance, and scalability in the implementation of AMR-targeted AI models are considered.
Collapse
Affiliation(s)
- Alex Howard
- Department of Antimicrobial Pharmacodynamics and Therapeutics, Institute of Systems, Molecular and Integrative Biology, University of Liverpool, Liverpool, UK; Liverpool University Hospitals NHS Foundation Trust, Liverpool, UK.
| | - Stephen Aston
- Department of Antimicrobial Pharmacodynamics and Therapeutics, Institute of Systems, Molecular and Integrative Biology, University of Liverpool, Liverpool, UK; Liverpool University Hospitals NHS Foundation Trust, Liverpool, UK
| | - Alessandro Gerada
- Department of Antimicrobial Pharmacodynamics and Therapeutics, Institute of Systems, Molecular and Integrative Biology, University of Liverpool, Liverpool, UK; Liverpool University Hospitals NHS Foundation Trust, Liverpool, UK
| | - Nada Reza
- Department of Antimicrobial Pharmacodynamics and Therapeutics, Institute of Systems, Molecular and Integrative Biology, University of Liverpool, Liverpool, UK; Liverpool University Hospitals NHS Foundation Trust, Liverpool, UK
| | - Jason Bincalar
- Department of Health Data Science, University of Liverpool, Liverpool, UK; Liverpool University Hospitals NHS Foundation Trust, Liverpool, UK
| | - Henry Mwandumba
- Malawi Liverpool Wellcome Programme, Kamuzu University of Health Sciences, Blantyre, Malawi; Department of Clinical Sciences, Liverpool School of Tropical Medicine, Liverpool, UK
| | - Tom Butterworth
- Combined Intelligence for Public Health Action, NHS Cheshire and Merseyside, Warrington, UK
| | - William Hope
- Department of Antimicrobial Pharmacodynamics and Therapeutics, Institute of Systems, Molecular and Integrative Biology, University of Liverpool, Liverpool, UK; Liverpool University Hospitals NHS Foundation Trust, Liverpool, UK
| | - Iain Buchan
- Department of Public Health, Policy and Systems, Institute of Population Health, University of Liverpool, Liverpool, UK; Combined Intelligence for Public Health Action, NHS Cheshire and Merseyside, Warrington, UK
| |
Collapse
|
18
|
Zhang B, Li J, Bai Y, Jiang Q, Yan B, Wang Z. An Improved Microaneurysm Detection Model Based on SwinIR and YOLOv8. Bioengineering (Basel) 2023; 10:1405. [PMID: 38135996 PMCID: PMC10740408 DOI: 10.3390/bioengineering10121405] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/09/2023] [Revised: 11/23/2023] [Accepted: 11/26/2023] [Indexed: 12/24/2023] Open
Abstract
Diabetic retinopathy (DR) is a microvascular complication of diabetes. Microaneurysms (MAs) are often observed in the retinal vessels of diabetic patients and represent one of the earliest signs of DR. Accurate and efficient detection of MAs is crucial for the diagnosis of DR. In this study, an automatic model (MA-YOLO) is proposed for MA detection in fluorescein angiography (FFA) images. To obtain detailed features and improve the discriminability of MAs in FFA images, SwinIR was utilized to reconstruct super-resolution images. To solve the problems of missed detection of small features and feature information loss, an MA detection layer was added between the neck and the head sections of YOLOv8. To enhance the generalization ability of the MA-YOLO model, transfer learning was conducted between high-resolution images and low-resolution images. To avoid excessive penalization due to geometric factors and address sample distribution imbalance, the loss function was optimized by taking the Wise-IoU loss as a bounding box regression loss. The performance of the MA-YOLO model in MA detection was compared with that of other state-of-the-art models, including SSD, RetinaNet, YOLOv5, YOLOX, and YOLOv7. The results showed that the MA-YOLO model had the best performance in MA detection, as shown by its optimal metrics, including recall, precision, F1 score, and AP, which were 88.23%, 97.98%, 92.85%, and 94.62%, respectively. Collectively, the proposed MA-YOLO model is suitable for the automatic detection of MAs in FFA images, which can assist ophthalmologists in the diagnosis of the progression of DR.
Collapse
Affiliation(s)
- Bowei Zhang
- College of Information Science, Shanghai Ocean University, Shanghai 201306, China; (B.Z.); (Y.B.)
| | - Jing Li
- Department of Ophthalmology, Eye Institute, Eye and ENT Hospital, Fudan University, Shanghai 201114, China (B.Y.)
| | - Yun Bai
- College of Information Science, Shanghai Ocean University, Shanghai 201306, China; (B.Z.); (Y.B.)
| | - Qing Jiang
- The Affiliated Eye Hospital, Nanjing Medical University, Nanjing 211166, China
| | - Biao Yan
- Department of Ophthalmology, Eye Institute, Eye and ENT Hospital, Fudan University, Shanghai 201114, China (B.Y.)
| | - Zhenhua Wang
- College of Information Science, Shanghai Ocean University, Shanghai 201306, China; (B.Z.); (Y.B.)
| |
Collapse
|
19
|
Komasawa N, Yokohira M. Learner-Centered Experience-Based Medical Education in an AI-Driven Society: A Literature Review. Cureus 2023; 15:e46883. [PMID: 37954813 PMCID: PMC10636515 DOI: 10.7759/cureus.46883] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 10/12/2023] [Indexed: 11/14/2023] Open
Abstract
This review proposes and explores the significance of "experience-based medical education" (EXPBME) in the context of an artificial intelligence (AI)-driven society. The rapid advancements in AI, particularly driven by deep learning, have revolutionized medical practices by replicating human cognitive functions, such as image analysis and data interpretation, significantly enhancing efficiency and precision across medical settings. The integration of AI into healthcare presents substantial potential, ranging from precise diagnostics to streamlined data management. However, non-technical skills, such as situational awareness on recognizing AI's fallibility or inherent risks, are critical for future healthcare professionals. EXPBME in a clinical or simulation environment plays a vital role, allowing medical practitioners to navigate AI failures through sufficient reflections. As AI continues to evolve, aligning educational frameworks to nurture these fundamental non-technical skills is paramount to adequately prepare healthcare professionals. Learner-centered EXPBME, combined with the AI literacy acquirement, stands as a key pillar in shaping the future of medical education.
Collapse
Affiliation(s)
- Nobuyasu Komasawa
- Community Medicine Education Promotion Office, Faculty of Medicine, Kagawa University, Takamatsu, JPN
| | - Masanao Yokohira
- Department of Medical Education, Kagawa University, Takamatsu, JPN
| |
Collapse
|