1
|
Wills MF, Alejo CB, Hundt N, Hudson AJ, Eperon IC. FluoroTensor: Identification and tracking of colocalised molecules and their stoichiometries in multi-colour single molecule imaging via deep learning. Comput Struct Biotechnol J 2024; 23:918-928. [PMID: 38375530 PMCID: PMC10875188 DOI: 10.1016/j.csbj.2024.02.004] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/17/2023] [Revised: 02/06/2024] [Accepted: 02/06/2024] [Indexed: 02/21/2024] Open
Abstract
The identification of photobleaching steps in single molecule fluorescence imaging is a well-established procedure for analysing the stoichiometries of molecular complexes. Nonetheless, the method is challenging with protein fluorophores because of the high levels of noise, rapid bleaching and highly variable signal intensities, all of which complicate methods based on statistical analyses of intensities to identify bleaching steps. It has recently been shown that deep learning by convolutional neural networks can yield an accurate analysis with a relatively short computational time. We describe here an improved use of such an approach that detects bleaching events even in the first time point of observation, and we have included this within an integrated software package incorporating fluorescence spot detection, colocalisation, tracking, FRET and photobleaching step analyses of single molecules or complexes. This package, known as FluoroTensor, is written in Python with a self-explanatory user interface.
Collapse
Affiliation(s)
- Max F.K. Wills
- Institute for Structural and Chemical Biology, University of Leicester, UK
- Department of Molecular and Cell Biology, University of Leicester, UK
| | - Carlos Bueno Alejo
- Institute for Structural and Chemical Biology, University of Leicester, UK
- Department of Chemistry, University of Leicester, UK
| | - Nikolas Hundt
- Department of Cellular Physiology, Ludwig-Maximilians-Universität München, Germany
| | - Andrew J. Hudson
- Institute for Structural and Chemical Biology, University of Leicester, UK
- Department of Chemistry, University of Leicester, UK
| | - Ian C. Eperon
- Institute for Structural and Chemical Biology, University of Leicester, UK
- Department of Molecular and Cell Biology, University of Leicester, UK
| |
Collapse
|
2
|
Poimala J, Cox B, Hauptmann A. Compensating unknown speed of sound in learned fast 3D limited-view photoacoustic tomography. Photoacoustics 2024; 37:100597. [PMID: 38425677 PMCID: PMC10901832 DOI: 10.1016/j.pacs.2024.100597] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 04/29/2022] [Revised: 08/15/2023] [Accepted: 02/16/2024] [Indexed: 03/02/2024]
Abstract
Real-time applications in three-dimensional photoacoustic tomography from planar sensors rely on fast reconstruction algorithms that assume the speed of sound (SoS) in the tissue is homogeneous. Moreover, the reconstruction quality depends on the correct choice for the constant SoS. In this study, we discuss the possibility of ameliorating the problem of unknown or heterogeneous SoS distributions by using learned reconstruction methods. This can be done by modelling the uncertainties in the training data. In addition, a correction term can be included in the learned reconstruction method. We investigate the influence of both and while a learned correction component can improve reconstruction quality further, we show that a careful choice of uncertainties in the training data is the primary factor to overcome unknown SoS. We support our findings with simulated and in vivo measurements in 3D.
Collapse
Affiliation(s)
- Jenni Poimala
- Research Unit of Mathematical Sciences, University of Oulu, Finland
| | - Ben Cox
- Department of Medical Physics and Biomedical Engineering, University College London, UK
| | - Andreas Hauptmann
- Research Unit of Mathematical Sciences, University of Oulu, Finland
- Department of Computer Science, University College London, UK
| |
Collapse
|
3
|
Zhang Y, Fang Z, Fan J. Generalization analysis of deep CNNs under maximum correntropy criterion. Neural Netw 2024; 174:106226. [PMID: 38490117 DOI: 10.1016/j.neunet.2024.106226] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/08/2023] [Revised: 02/01/2024] [Accepted: 03/04/2024] [Indexed: 03/17/2024]
Abstract
Convolutional neural networks (CNNs) have gained immense popularity in recent years, finding their utility in diverse fields such as image recognition, natural language processing, and bio-informatics. Despite the remarkable progress made in deep learning theory, most studies on CNNs, especially in regression tasks, tend to heavily rely on the least squares loss function. However, there are situations where such learning algorithms may not suffice, particularly in the presence of heavy-tailed noises or outliers. This predicament emphasizes the necessity of exploring alternative loss functions that can handle such scenarios more effectively, thereby unleashing the true potential of CNNs. In this paper, we investigate the generalization error of deep CNNs with the rectified linear unit (ReLU) activation function for robust regression problems within an information-theoretic learning framework. Our study demonstrates that when the regression function exhibits an additive ridge structure and the noise possesses a finite pth moment, the empirical risk minimization scheme, generated by the maximum correntropy criterion and deep CNNs, achieves fast convergence rates. Notably, these rates align with the mini-max optimal convergence rates attained by fully connected neural network model with the Huber loss function up to a logarithmic factor. Additionally, we further establish the convergence rates of deep CNNs under the maximum correntropy criterion when the regression function resides in a Sobolev space on the sphere.
Collapse
Affiliation(s)
- Yingqiao Zhang
- Department of Mathematics, Hong Kong Baptist University, Kowloon, Hong Kong, China.
| | - Zhiying Fang
- Institute of Applied Mathematics, Shenzhen Polytechnic University, Shahexi Road 4089, Shenzhen, 518000, Guangdong, China.
| | - Jun Fan
- Department of Mathematics, Hong Kong Baptist University, Kowloon, Hong Kong, China.
| |
Collapse
|
4
|
Lu Y, Cao Y, Tang X, Hu N, Wang Z, Xu P, Hua Z, Wang Y, Su Y, Guo Y. Deep learning-assisted mass spectrometry imaging for preliminary screening and pre-classification of psychoactive substances. Talanta 2024; 272:125757. [PMID: 38368831 DOI: 10.1016/j.talanta.2024.125757] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/25/2023] [Revised: 01/28/2024] [Accepted: 02/05/2024] [Indexed: 02/20/2024]
Abstract
Currently, it is of great urgency to develop a rapid pre-classification and screening method for suspected drugs as the constantly springing up of new psychoactive substances. In most researches, psychoactive substances classification approaches depended on the similar chemical structures and pharmacological action with known drugs. Such approaches could not face the complicated circumstance of emerging new psychoactive substances. Herein, mass spectrometry imaging and convolutional neural networks (CNN) were used for preliminary screening and pre-classification of suspected psychoactive substances. Mass spectrometry imaging was performed simultaneously on two brain slices as one was from blank group and another one was from psychoactive substance-induced group. Then, fused neurotransmitter variation mass spectrometry images (Nv-MSIs) reflecting the difference of neurotransmitters between two slices were achieved through two homemade programs. A CNN model was developed to classify the Nv-MSIs. Compared with traditional classification methods, CNN achieved better estimation accuracy and required minimal data preprocessing. Also, the specific region on Nv-MSIs and weight of each neurotransmitter that affected the classification most could be unraveled by CNN. Finally, the method was successfully applied to assist the identification of a new psychoactive substance seized recently. This sample was identified as cannabinoids, which greatly promoted the screening process.
Collapse
Affiliation(s)
- Yingjie Lu
- Institute of Interdisciplinary Integrative Medicine Research, Shanghai University of Traditional Chinese Medicine, 1200 Cailun Road, Shanghai, 201203, China; Department of Pharmacognosy, School of Pharmacy, Naval Medical University, Shanghai, 200433, China
| | - Yuqi Cao
- Technical Centre, Shanghai Tobacco (Group) Corp., Shanghai, 200082, China
| | - Xiaohang Tang
- Institute of Interdisciplinary Integrative Medicine Research, Shanghai University of Traditional Chinese Medicine, 1200 Cailun Road, Shanghai, 201203, China
| | - Na Hu
- Institute of Interdisciplinary Integrative Medicine Research, Shanghai University of Traditional Chinese Medicine, 1200 Cailun Road, Shanghai, 201203, China
| | - Zhengyong Wang
- Institute of Interdisciplinary Integrative Medicine Research, Shanghai University of Traditional Chinese Medicine, 1200 Cailun Road, Shanghai, 201203, China
| | - Peng Xu
- Key Laboratory of Drug Monitoring and Control, Drug Intelligence and Forensic Center, Ministry of Public Security, Beijing, 100193, China
| | - Zhendong Hua
- Key Laboratory of Drug Monitoring and Control, Drug Intelligence and Forensic Center, Ministry of Public Security, Beijing, 100193, China
| | - Youmei Wang
- Key Laboratory of Drug Monitoring and Control, Drug Intelligence and Forensic Center, Ministry of Public Security, Beijing, 100193, China.
| | - Yue Su
- Institute of Interdisciplinary Integrative Medicine Research, Shanghai University of Traditional Chinese Medicine, 1200 Cailun Road, Shanghai, 201203, China.
| | - Yinlong Guo
- State Key Laboratory of Organometallic Chemistry and National Center for Organic Mass Spectrometry in Shanghai, Center for Excellence in Molecular Synthesis, Shanghai Institute of Organic Chemistry, University of Chinese Academy of Sciences, Chinese Academy of Sciences, 345 Lingling Road, Shanghai, 200032, China.
| |
Collapse
|
5
|
Urbańska M, Sofińska K, Czaja M, Szymoński K, Skirlińska-Nosek K, Seweryn S, Lupa D, Szymoński M, Lipiec E. Molecular alterations in metaphase chromosomes induced by bleomycin. Spectrochim Acta A Mol Biomol Spectrosc 2024; 312:124026. [PMID: 38368817 DOI: 10.1016/j.saa.2024.124026] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 03/20/2023] [Revised: 12/22/2023] [Accepted: 02/07/2024] [Indexed: 02/20/2024]
Abstract
Chromosomes are intranuclear structures, their main function is to store and transmit genetic information during cell division. They are composed of tightly packed DNA in the form of chromatin, which is constantly exposed to various damaging factors. The resulting changes in DNA can have serious consequences (e.g. mutations) if they are not repaired or repaired incorrectly. In this article, we studied chromosomes isolated from human cervical cancer cells (HeLa) exposed to a genotoxic drug causing both single- and double-strand breaks. Specifically, we used bleomycin to induce DNA damage. We followed morphological and chemical changes in chromosomes upon damage induction. Atomic force microscopy was used to visualize the morphology of chromosomes, while Raman microspectroscopy enabled the detection of changes in the chemical structure of chromatin with the resolution close to the diffraction limit. Additionally, we extracted spectra corresponding to chromosome I or chromatin from hyperspectral Raman maps with convolutional neural networks (CNN), which were further analysed with the principal component analysis (PCA) algorithm to reveal molecular markers of DNA damage in chromosomes. The applied multimodal approach revealed simultaneous morphological and molecular changes, including chromosomal aberrations, alterations in DNA conformation, methylation pattern, and increased protein expression upon the bleomycin treatment at the level of the single chromosome.
Collapse
Affiliation(s)
- Marta Urbańska
- Jagiellonian University, Faculty of Physics, Astronomy and Applied Computer Science, M. Smoluchowski Institute of Physics, Łojasiewicza 11, 30-348 Krakow, Poland; Jagiellonian University, Doctoral School of Exact and Natural Sciences, Krakow, Poland
| | - Kamila Sofińska
- Jagiellonian University, Faculty of Physics, Astronomy and Applied Computer Science, M. Smoluchowski Institute of Physics, Łojasiewicza 11, 30-348 Krakow, Poland
| | - Michał Czaja
- Jagiellonian University, Faculty of Physics, Astronomy and Applied Computer Science, M. Smoluchowski Institute of Physics, Łojasiewicza 11, 30-348 Krakow, Poland; Jagiellonian University, Doctoral School of Exact and Natural Sciences, Krakow, Poland
| | - Krzysztof Szymoński
- Jagiellonian University Medical College, Department of Pathomorphology, Grzegorzecka 16, 31-531, Krakow, Poland; University Hospital, Department of Pathomorphology, Krakow, Poland
| | - Katarzyna Skirlińska-Nosek
- Jagiellonian University, Faculty of Physics, Astronomy and Applied Computer Science, M. Smoluchowski Institute of Physics, Łojasiewicza 11, 30-348 Krakow, Poland; Jagiellonian University, Doctoral School of Exact and Natural Sciences, Krakow, Poland
| | - Sara Seweryn
- Jagiellonian University, Faculty of Physics, Astronomy and Applied Computer Science, M. Smoluchowski Institute of Physics, Łojasiewicza 11, 30-348 Krakow, Poland; Jagiellonian University, Doctoral School of Exact and Natural Sciences, Krakow, Poland
| | - Dawid Lupa
- Jagiellonian University, Faculty of Physics, Astronomy and Applied Computer Science, M. Smoluchowski Institute of Physics, Łojasiewicza 11, 30-348 Krakow, Poland
| | - Marek Szymoński
- Jagiellonian University, Faculty of Physics, Astronomy and Applied Computer Science, M. Smoluchowski Institute of Physics, Łojasiewicza 11, 30-348 Krakow, Poland
| | - Ewelina Lipiec
- Jagiellonian University, Faculty of Physics, Astronomy and Applied Computer Science, M. Smoluchowski Institute of Physics, Łojasiewicza 11, 30-348 Krakow, Poland.
| |
Collapse
|
6
|
Binson VA, Thomas S, Subramoniam M, Arun J, Naveen S, Madhu S. A Review of Machine Learning Algorithms for Biomedical Applications. Ann Biomed Eng 2024; 52:1159-1183. [PMID: 38383870 DOI: 10.1007/s10439-024-03459-3] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/30/2023] [Accepted: 01/24/2024] [Indexed: 02/23/2024]
Abstract
As the amount and complexity of biomedical data continue to increase, machine learning methods are becoming a popular tool in creating prediction models for the underlying biomedical processes. Although all machine learning methods aim to fit models to data, the methodologies used can vary greatly and may seem daunting at first. A comprehensive review of various machine learning algorithms per biomedical applications is presented. The key concepts of machine learning are supervised and unsupervised learning, feature selection, and evaluation metrics. Technical insights on the major machine learning methods such as decision trees, random forests, support vector machines, and k-nearest neighbors are analyzed. Next, the dimensionality reduction methods like principal component analysis and t-distributed stochastic neighbor embedding methods, and their applications in biomedical data analysis were reviewed. Moreover, in biomedical applications predominantly feedforward neural networks, convolutional neural networks, and recurrent neural networks are utilized. In addition, the identification of emerging directions in machine learning methodology will serve as a useful reference for individuals involved in biomedical research, clinical practice, and related professions who are interested in understanding and applying machine learning algorithms in their research or practice.
Collapse
Affiliation(s)
- V A Binson
- Department of Electronics Engineering, Saintgits College of Engineering, Kottayam, India
| | - Sania Thomas
- Department of Computer Science and Engineering, Saintgits College of Engineering, Kottayam, India
| | - M Subramoniam
- Department of Electronics Engineering, Sathyabama Institute of Science and Technology, Chennai, Tamil Nadu, India
| | - J Arun
- Centre for Waste Management-International Research Centre, Sathyabama Institute of Science and Technology, Chennai, 600119, India
| | - S Naveen
- Department of Automobile Engineering, Saveetha School of Engineering, Saveetha Institute of Medical and Technical Sciences, Saveetha University, Chennai, Tamil Nadu, India
| | - S Madhu
- Department of Automobile Engineering, Saveetha School of Engineering, Saveetha Institute of Medical and Technical Sciences, Saveetha University, Chennai, Tamil Nadu, India.
| |
Collapse
|
7
|
Yap MH, Cassidy B, Byra M, Liao TY, Yi H, Galdran A, Chen YH, Brüngel R, Koitka S, Friedrich CM, Lo YW, Yang CH, Li K, Lao Q, Ballester MAG, Carneiro G, Ju YJ, Huang JD, Pappachan JM, Reeves ND, Chandrabalan V, Dancey D, Kendrick C. Diabetic foot ulcers segmentation challenge report: Benchmark and analysis. Med Image Anal 2024; 94:103153. [PMID: 38569380 DOI: 10.1016/j.media.2024.103153] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/27/2023] [Revised: 01/30/2024] [Accepted: 03/20/2024] [Indexed: 04/05/2024]
Abstract
Monitoring the healing progress of diabetic foot ulcers is a challenging process. Accurate segmentation of foot ulcers can help podiatrists to quantitatively measure the size of wound regions to assist prediction of healing status. The main challenge in this field is the lack of publicly available manual delineation, which can be time consuming and laborious. Recently, methods based on deep learning have shown excellent results in automatic segmentation of medical images, however, they require large-scale datasets for training, and there is limited consensus on which methods perform the best. The 2022 Diabetic Foot Ulcers segmentation challenge was held in conjunction with the 2022 International Conference on Medical Image Computing and Computer Assisted Intervention, which sought to address these issues and stimulate progress in this research domain. A training set of 2000 images exhibiting diabetic foot ulcers was released with corresponding segmentation ground truth masks. Of the 72 (approved) requests from 47 countries, 26 teams used this data to develop fully automated systems to predict the true segmentation masks on a test set of 2000 images, with the corresponding ground truth segmentation masks kept private. Predictions from participating teams were scored and ranked according to their average Dice similarity coefficient of the ground truth masks and prediction masks. The winning team achieved a Dice of 0.7287 for diabetic foot ulcer segmentation. This challenge has now entered a live leaderboard stage where it serves as a challenging benchmark for diabetic foot ulcer segmentation.
Collapse
Affiliation(s)
- Moi Hoon Yap
- Department of Computing and Mathematics, Manchester Metropolitan University, John Dalton Building, Chester Street, Manchester M1 5GD, United Kingdom; Lancashire Teaching Hospitals NHS Trust, Preston, PR2 9HT, United Kingdom.
| | - Bill Cassidy
- Department of Computing and Mathematics, Manchester Metropolitan University, John Dalton Building, Chester Street, Manchester M1 5GD, United Kingdom
| | - Michal Byra
- Institute of Fundamental Technological Research, Polish Academy of Sciences, Warsaw, Poland; RIKEN Center for Brain Science, Wako, Japan
| | - Ting-Yu Liao
- Department of Computer Science, National Tsing Hua University, No. 101, Section 2, Kuang-Fu Road, Hsinchu, Taiwan
| | - Huahui Yi
- West China Biomedical Big Data Center, West China Hospital, Sichuan University, Chengdu, China
| | - Adrian Galdran
- BCN Medtech, Universitat Pompeu Fabra, Barcelona, Spain; AIML, University of Adelaide, Australia
| | - Yung-Han Chen
- Institute of Electronics, National Yang Ming Chiao Tung University, No. 1001, University Road, Hsinchu 300, Taiwan
| | - Raphael Brüngel
- Department of Computer Science, University of Applied Sciences and Arts Dortmund (FH Dortmund), Emil-Figge-Str. 42, 44227 Dortmund, Germany; Institute for Medical Informatics, Biometry and Epidemiology (IMIBE), University Hospital Essen, Zweigertstr. 37, 45130 Essen, Germany; Institute for Artificial Intelligence in Medicine (IKIM), University Hospital Essen, Girardetstr. 2, 45131 Essen, Germany
| | - Sven Koitka
- Institute for Artificial Intelligence in Medicine (IKIM), University Hospital Essen, Girardetstr. 2, 45131 Essen, Germany; Institute of Diagnostic and Interventional Radiology and Neuroradiology, University Hospital Essen, Hufelandstr. 55, 45147 Essen, Germany
| | - Christoph M Friedrich
- Department of Computer Science, University of Applied Sciences and Arts Dortmund (FH Dortmund), Emil-Figge-Str. 42, 44227 Dortmund, Germany; Institute for Medical Informatics, Biometry and Epidemiology (IMIBE), University Hospital Essen, Zweigertstr. 37, 45130 Essen, Germany
| | - Yu-Wen Lo
- Department of Computer Science, National Tsing Hua University, No. 101, Section 2, Kuang-Fu Road, Hsinchu, Taiwan
| | - Ching-Hui Yang
- Department of Computer Science, National Tsing Hua University, No. 101, Section 2, Kuang-Fu Road, Hsinchu, Taiwan
| | - Kang Li
- West China Biomedical Big Data Center, West China Hospital, Sichuan University, Chengdu, China; Shanghai Artificial Intelligence Laboratory, Shanghai, China
| | - Qicheng Lao
- School of Artificial Intelligence, Beijing University of Posts and Telecommunications, Beijing, China; Shanghai Artificial Intelligence Laboratory, Shanghai, China
| | | | | | - Yi-Jen Ju
- Institute of Electronics, National Yang Ming Chiao Tung University, No. 1001, University Road, Hsinchu 300, Taiwan
| | - Juinn-Dar Huang
- Institute of Electronics, National Yang Ming Chiao Tung University, No. 1001, University Road, Hsinchu 300, Taiwan
| | - Joseph M Pappachan
- Lancashire Teaching Hospitals NHS Trust, Preston, PR2 9HT, United Kingdom; Department of Life Sciences, Manchester Metropolitan University, Manchester, M1 5GD, United Kingdom
| | - Neil D Reeves
- Department of Life Sciences, Manchester Metropolitan University, Manchester, M1 5GD, United Kingdom
| | | | - Darren Dancey
- Department of Computing and Mathematics, Manchester Metropolitan University, John Dalton Building, Chester Street, Manchester M1 5GD, United Kingdom
| | - Connah Kendrick
- Department of Computing and Mathematics, Manchester Metropolitan University, John Dalton Building, Chester Street, Manchester M1 5GD, United Kingdom
| |
Collapse
|
8
|
Deng H, Li M, Li J, Guo M, Xu G. A robust multi-branch multi-attention-mechanism EEGNet for motor imagery BCI decoding. J Neurosci Methods 2024; 405:110108. [PMID: 38458260 DOI: 10.1016/j.jneumeth.2024.110108] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/05/2023] [Revised: 02/28/2024] [Accepted: 03/05/2024] [Indexed: 03/10/2024]
Abstract
BACKGROUND Motor-Imagery-based Brain-Computer Interface (MI-BCI) is a promising technology to assist communication, movement, and neurological rehabilitation for motor-impaired individuals. Electroencephalography (EEG) decoding techniques using deep learning (DL) possess noteworthy advantages due to automatic feature extraction and end-to-end learning. However, the DL-based EEG decoding models tend to show large variations due to intersubject variability of EEG, which results from inconsistencies of different subjects' optimal hyperparameters. NEW METHODS This study proposes a multi-branch multi-attention mechanism EEGNet model (MBMANet) for robust decoding. It applies the multi-branch EEGNet structure to achieve various feature extractions. Further, the different attention mechanisms introduced in each branch attain diverse adaptive weight adjustments. This combination of multi-branch and multi-attention mechanisms allows for multi-level feature fusion to provide robust decoding for different subjects. RESULTS The MBMANet model has a four-classification accuracy of 83.18% and kappa of 0.776 on the BCI Competition IV-2a dataset, which outperforms other eight CNN-based decoding models. This consistently satisfactory performance across all nine subjects indicates that the proposed model is robust. CONCLUSIONS The combine of multi-branch and multi-attention mechanisms empowers the DL-based models to adaptively learn different EEG features, which provides a feasible solution for dealing with data variability. It also gives the MBMANet model more accurate decoding of motion intentions and lower training costs, thus improving the MI-BCI's utility and robustness.
Collapse
Affiliation(s)
- Haodong Deng
- State Key Laboratory of Reliability and Intelligence of Electrical Equipment, School of Health Sciences and Biomedical Engineering, Hebei University of Technology, Tianjin 300132, China; Hebei Key Laboratory of Bioelectromagnetics and Neuroengineering, Tianjin 300132, China; Tianjin Key Laboratory of Bioelectromagnetic Technology and Intelligent Health, Hebei University of Technology, Tianjin 300132, China; School of Health Sciences and Biomedical Engineering, Hebei University of Technology, Tianjin 300132, China
| | - Mengfan Li
- State Key Laboratory of Reliability and Intelligence of Electrical Equipment, School of Health Sciences and Biomedical Engineering, Hebei University of Technology, Tianjin 300132, China; Hebei Key Laboratory of Bioelectromagnetics and Neuroengineering, Tianjin 300132, China; Tianjin Key Laboratory of Bioelectromagnetic Technology and Intelligent Health, Hebei University of Technology, Tianjin 300132, China; School of Health Sciences and Biomedical Engineering, Hebei University of Technology, Tianjin 300132, China.
| | - Jundi Li
- State Key Laboratory of Reliability and Intelligence of Electrical Equipment, School of Health Sciences and Biomedical Engineering, Hebei University of Technology, Tianjin 300132, China; Hebei Key Laboratory of Bioelectromagnetics and Neuroengineering, Tianjin 300132, China; Tianjin Key Laboratory of Bioelectromagnetic Technology and Intelligent Health, Hebei University of Technology, Tianjin 300132, China; School of Health Sciences and Biomedical Engineering, Hebei University of Technology, Tianjin 300132, China
| | - Miaomiao Guo
- State Key Laboratory of Reliability and Intelligence of Electrical Equipment, School of Health Sciences and Biomedical Engineering, Hebei University of Technology, Tianjin 300132, China; Hebei Key Laboratory of Bioelectromagnetics and Neuroengineering, Tianjin 300132, China; Tianjin Key Laboratory of Bioelectromagnetic Technology and Intelligent Health, Hebei University of Technology, Tianjin 300132, China; School of Health Sciences and Biomedical Engineering, Hebei University of Technology, Tianjin 300132, China
| | - Guizhi Xu
- State Key Laboratory of Reliability and Intelligence of Electrical Equipment, School of Health Sciences and Biomedical Engineering, Hebei University of Technology, Tianjin 300132, China; Hebei Key Laboratory of Bioelectromagnetics and Neuroengineering, Tianjin 300132, China; Tianjin Key Laboratory of Bioelectromagnetic Technology and Intelligent Health, Hebei University of Technology, Tianjin 300132, China; School of Health Sciences and Biomedical Engineering, Hebei University of Technology, Tianjin 300132, China
| |
Collapse
|
9
|
Yang T, Wang Y, He Y. TEC-miTarget: enhancing microRNA target prediction based on deep learning of ribonucleic acid sequences. BMC Bioinformatics 2024; 25:159. [PMID: 38643080 PMCID: PMC11032603 DOI: 10.1186/s12859-024-05780-z] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/16/2023] [Accepted: 04/12/2024] [Indexed: 04/22/2024] Open
Abstract
BACKGROUND MicroRNAs play a critical role in regulating gene expression by binding to specific target sites within gene transcripts, making the identification of microRNA targets a prominent focus of research. Conventional experimental methods for identifying microRNA targets are both time-consuming and expensive, prompting the development of computational tools for target prediction. However, the existing computational tools exhibit limited performance in meeting the demands of practical applications, highlighting the need to improve the performance of microRNA target prediction models. RESULTS In this paper, we utilize the most popular natural language processing and computer vision technologies to propose a novel approach, called TEC-miTarget, for microRNA target prediction based on transformer encoder and convolutional neural networks. TEC-miTarget treats RNA sequences as a natural language and encodes them using a transformer encoder, a widely used encoder in natural language processing. It then combines the representations of a pair of microRNA and its candidate target site sequences into a contact map, which is a three-dimensional array similar to a multi-channel image. Therefore, the contact map's features are extracted using a four-layer convolutional neural network, enabling the prediction of interactions between microRNA and its candidate target sites. We applied a series of comparative experiments to demonstrate that TEC-miTarget significantly improves microRNA target prediction, compared with existing state-of-the-art models. Our approach is the first approach to perform comparisons with other approaches at both sequence and transcript levels. Furthermore, it is the first approach compared with both deep learning-based and seed-match-based methods. We first compared TEC-miTarget's performance with approaches at the sequence level, and our approach delivers substantial improvements in performance using the same datasets and evaluation metrics. Moreover, we utilized TEC-miTarget to predict microRNA targets in long mRNA sequences, which involves two steps: selecting candidate target site sequences and applying sequence-level predictions. We finally showed that TEC-miTarget outperforms other approaches at the transcript level, including the popular seed match methods widely used in previous years. CONCLUSIONS We propose a novel approach for predicting microRNA targets at both sequence and transcript levels, and demonstrate that our approach outperforms other methods based on deep learning or seed match. We also provide our approach as an easy-to-use software, TEC-miTarget, at https://github.com/tingpeng17/TEC-miTarget . Our results provide new perspectives for microRNA target prediction.
Collapse
Affiliation(s)
- Tingpeng Yang
- Peng Cheng Laboratory, Shenzhen, 518055, China
- Tsinghua Shenzhen International Graduate School, Shenzhen, 518055, China
| | - Yu Wang
- Peng Cheng Laboratory, Shenzhen, 518055, China.
| | - Yonghong He
- Peng Cheng Laboratory, Shenzhen, 518055, China.
- Tsinghua Shenzhen International Graduate School, Shenzhen, 518055, China.
| |
Collapse
|
10
|
da Costa D, Kornemann L, Goebel R, Senden M. Convolutional neural networks develop major organizational principles of early visual cortex when enhanced with retinal sampling. Sci Rep 2024; 14:8980. [PMID: 38637554 PMCID: PMC11026486 DOI: 10.1038/s41598-024-59376-x] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/02/2023] [Accepted: 04/09/2024] [Indexed: 04/20/2024] Open
Abstract
Primate visual cortex exhibits key organizational principles: cortical magnification, eccentricity-dependent receptive field size and spatial frequency tuning as well as radial bias. We provide compelling evidence that these principles arise from the interplay of the non-uniform distribution of retinal ganglion cells, and a quasi-uniform convergence rate from the retina to the cortex. We show that convolutional neural networks outfitted with a retinal sampling layer, which resamples images according to retinal ganglion cell density, develop these organizational principles. Surprisingly, our results indicate that radial bias is spatial-frequency dependent and only manifests for high spatial frequencies. For low spatial frequencies, the bias shifts towards orthogonal orientations. These findings introduce a novel hypothesis about the origin of radial bias. Quasi-uniform convergence limits the range of spatial frequencies (in retinal space) that can be resolved, while retinal sampling determines the spatial frequency content throughout the retina.
Collapse
Affiliation(s)
- Danny da Costa
- Department of Cognitive Neuroscience, Faculty of Psychology and Neuroscience, Maastricht University, Oxfordlaan 55, 6229 EV, Maastricht, The Netherlands.
- Maastricht Brain Imaging Centre, Maastricht University, Oxfordlaan 55, 6229 EV, Maastricht, The Netherlands.
| | - Lukas Kornemann
- Department of Cognitive Neuroscience, Faculty of Psychology and Neuroscience, Maastricht University, Oxfordlaan 55, 6229 EV, Maastricht, The Netherlands
- University of Bonn, Regina-Pacis-Weg 3, 53113, Bonn, Germany
| | - Rainer Goebel
- Department of Cognitive Neuroscience, Faculty of Psychology and Neuroscience, Maastricht University, Oxfordlaan 55, 6229 EV, Maastricht, The Netherlands
- Maastricht Brain Imaging Centre, Maastricht University, Oxfordlaan 55, 6229 EV, Maastricht, The Netherlands
| | - Mario Senden
- Department of Cognitive Neuroscience, Faculty of Psychology and Neuroscience, Maastricht University, Oxfordlaan 55, 6229 EV, Maastricht, The Netherlands.
- Maastricht Brain Imaging Centre, Maastricht University, Oxfordlaan 55, 6229 EV, Maastricht, The Netherlands.
| |
Collapse
|
11
|
Zhu J, Bolsterlee B, Chow BVY, Song Y, Meijering E. Hybrid dual mean-teacher network with double-uncertainty guidance for semi-supervised segmentation of magnetic resonance images. Comput Med Imaging Graph 2024; 115:102383. [PMID: 38643551 DOI: 10.1016/j.compmedimag.2024.102383] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/02/2023] [Revised: 03/26/2024] [Accepted: 04/14/2024] [Indexed: 04/23/2024]
Abstract
Semi-supervised learning has made significant progress in medical image segmentation. However, existing methods primarily utilize information from a single dimensionality, resulting in sub-optimal performance on challenging magnetic resonance imaging (MRI) data with multiple segmentation objects and anisotropic resolution. To address this issue, we present a Hybrid Dual Mean-Teacher (HD-Teacher) model with hybrid, semi-supervised, and multi-task learning to achieve effective semi-supervised segmentation. HD-Teacher employs a 2D and a 3D mean-teacher network to produce segmentation labels and signed distance fields from the hybrid information captured in both dimensionalities. This hybrid mechanism allows HD-Teacher to utilize features from 2D, 3D, or both dimensions as needed. Outputs from 2D and 3D teacher models are dynamically combined based on confidence scores, forming a single hybrid prediction with estimated uncertainty. We propose a hybrid regularization module to encourage both student models to produce results close to the uncertainty-weighted hybrid prediction to further improve their feature extraction capability. Extensive experiments of binary and multi-class segmentation conducted on three MRI datasets demonstrated that the proposed framework could (1) significantly outperform state-of-the-art semi-supervised methods (2) surpass a fully-supervised VNet trained on substantially more annotated data, and (3) perform on par with human raters on muscle and bone segmentation task. Code will be available at https://github.com/ThisGame42/Hybrid-Teacher.
Collapse
Affiliation(s)
- Jiayi Zhu
- School of Computer Science and Engineering, University of New South Wales, Sydney, NSW 2052, Australia; Neuroscience Research Australia (NeuRA), Randwick, NSW 2031, Australia.
| | - Bart Bolsterlee
- Neuroscience Research Australia (NeuRA), Randwick, NSW 2031, Australia; Graduate School of Biomedical Engineering, University of New South Wales, Sydney, NSW 2052, Australia
| | - Brian V Y Chow
- Neuroscience Research Australia (NeuRA), Randwick, NSW 2031, Australia; School of Biomedical Sciences, University of New South Wales, Sydney, NSW 2052, Australia
| | - Yang Song
- School of Computer Science and Engineering, University of New South Wales, Sydney, NSW 2052, Australia
| | - Erik Meijering
- School of Computer Science and Engineering, University of New South Wales, Sydney, NSW 2052, Australia
| |
Collapse
|
12
|
Marcos L, Babyn P, Alirezaie J. Pure Vision Transformer (CT-ViT) with Noise2Neighbors Interpolation for Low-Dose CT Image Denoising. J Imaging Inform Med 2024:10.1007/s10278-024-01108-8. [PMID: 38622385 DOI: 10.1007/s10278-024-01108-8] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 01/05/2024] [Revised: 03/26/2024] [Accepted: 03/26/2024] [Indexed: 04/17/2024]
Abstract
Convolutional neural networks (CNN) have been used for a wide variety of deep learning applications, especially in computer vision. For medical image processing, researchers have identified certain challenges associated with CNNs. These challenges encompass the generation of less informative features, limitations in capturing both high and low-frequency information within feature maps, and the computational cost incurred when enhancing receptive fields by deepening the network. Transformers have emerged as an approach aiming to address and overcome these specific limitations of CNNs in the context of medical image analysis. Preservation of all spatial details of medical images is necessary to ensure accurate patient diagnosis. Hence, this research introduced the use of a pure Vision Transformer (ViT) for a denoising artificial neural network for medical image processing specifically for low-dose computed tomography (LDCT) image denoising. The proposed model follows a U-Net framework that contains ViT modules with the integration of Noise2Neighbor (N2N) interpolation operation. Five different datasets containing LDCT and normal-dose CT (NDCT) image pairs were used to carry out this experiment. To test the efficacy of the proposed model, this experiment includes comparisons between the quantitative and visual results among CNN-based (BM3D, RED-CNN, DRL-E-MP), hybrid CNN-ViT-based (TED-Net), and the proposed pure ViT-based denoising model. The findings of this study showed that there is about 15-20% increase in SSIM and PSNR when using self-attention transformers than using the typical pure CNN. Visual results also showed improvements especially when it comes to showing fine structural details of CT images.
Collapse
Affiliation(s)
- Luella Marcos
- Department of Electrical, Biomedical and Computer Engineering, Toronto Metropolitan University (formerly Ryerson University), 350 Victoria Street, Toronto, M5B 2K3, Ontario, Canada
| | - Paul Babyn
- Department of Medical Imaging, University of Saskatchewan, 105 Administration Pl, Saskatoon, SK S7N0W8, Saskatchewan, Canada
| | - Javad Alirezaie
- Department of Electrical, Biomedical and Computer Engineering, Toronto Metropolitan University (formerly Ryerson University), 350 Victoria Street, Toronto, M5B 2K3, Ontario, Canada.
| |
Collapse
|
13
|
Pérez-Cano J, Sansano Valero I, Anglada-Rotger D, Pina O, Salembier P, Marques F. Combining graph neural networks and computer vision methods for cell nuclei classification in lung tissue. Heliyon 2024; 10:e28463. [PMID: 38590866 PMCID: PMC10999915 DOI: 10.1016/j.heliyon.2024.e28463] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/13/2024] [Accepted: 03/19/2024] [Indexed: 04/10/2024] Open
Abstract
The detection of tumoural cells from whole slide images is an essential task in medical diagnosis and research. In this article, we propose and analyse a novel approach that combines computer vision-based models with graph neural networks to improve the accuracy of automated tumoural cell detection in lung tissue. Our proposal leverages the inherent structure and relationships between cells in the tissue. Experimental results on our own curated dataset show that modelling the problem with graphs gives the model a clear advantage over just working at pixel level. This change in perspective provides extra information that makes it possible to improve the performance. The reduction of dimensionality that comes from working with the graph also allows us to increase the field of view with low computational requirements. Code is available at https://github.com/Jerry-Master/lung-tumour-study, models are uploaded to https://huggingface.co/Jerry-Master/Hovernet-plus-Graphs, and the dataset is published on Zenodo https://zenodo.org/doi/10.5281/zenodo.8368122.
Collapse
Affiliation(s)
- Jose Pérez-Cano
- Department of Signal Theory and Communications, Universitat Politècnica de Catalunya, Barcelona, Spain
| | | | - David Anglada-Rotger
- Department of Signal Theory and Communications, Universitat Politècnica de Catalunya, Barcelona, Spain
| | - Oscar Pina
- Department of Signal Theory and Communications, Universitat Politècnica de Catalunya, Barcelona, Spain
| | - Philippe Salembier
- Department of Signal Theory and Communications, Universitat Politècnica de Catalunya, Barcelona, Spain
| | - Ferran Marques
- Department of Signal Theory and Communications, Universitat Politècnica de Catalunya, Barcelona, Spain
| |
Collapse
|
14
|
Katayama A, Aoki Y, Watanabe Y, Horiguchi J, Rakha EA, Oyama T. Current status and prospects of artificial intelligence in breast cancer pathology: convolutional neural networks to prospective Vision Transformers. Int J Clin Oncol 2024:10.1007/s10147-024-02513-3. [PMID: 38619651 DOI: 10.1007/s10147-024-02513-3] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/16/2024] [Accepted: 03/12/2024] [Indexed: 04/16/2024]
Abstract
Breast cancer is the most prevalent cancer among women, and its diagnosis requires the accurate identification and classification of histological features for effective patient management. Artificial intelligence, particularly through deep learning, represents the next frontier in cancer diagnosis and management. Notably, the use of convolutional neural networks and emerging Vision Transformers (ViT) has been reported to automate pathologists' tasks, including tumor detection and classification, in addition to improving the efficiency of pathology services. Deep learning applications have also been extended to the prediction of protein expression, molecular subtype, mutation status, therapeutic efficacy, and outcome prediction directly from hematoxylin and eosin-stained slides, bypassing the need for immunohistochemistry or genetic testing. This review explores the current status and prospects of deep learning in breast cancer diagnosis with a focus on whole-slide image analysis. Artificial intelligence applications are increasingly applied to many tasks in breast pathology ranging from disease diagnosis to outcome prediction, thus serving as valuable tools for assisting pathologists and supporting breast cancer management.
Collapse
Affiliation(s)
- Ayaka Katayama
- Diagnostic Pathology, Gunma University Graduate School of Medicine, 3-39-22 Showamachi, Maebashi, Gunma, 371-8511, Japan.
| | - Yuki Aoki
- Center for Mathematics and Data Science, Gunma University, Maebashi, Japan
| | - Yukako Watanabe
- Clinical Training Center, Gunma University Hospital, Maebashi, Japan
| | - Jun Horiguchi
- Department of Breast Surgery, International University of Health and Welfare, Narita, Japan
| | - Emad A Rakha
- Department of Histopathology School of Medicine, University of Nottingham, University Park, Nottingham, UK
- Department of Pathology, Hamad Medical Corporation, Doha, Qatar
| | - Tetsunari Oyama
- Diagnostic Pathology, Gunma University Graduate School of Medicine, 3-39-22 Showamachi, Maebashi, Gunma, 371-8511, Japan
| |
Collapse
|
15
|
liang Zhang D, Jiang Z, Mohammadzadeh F, Hasani Azhdari SM, Abualigah L, Ghazal TM. FUZ-SMO: A fuzzy slime mould optimizer for mitigating false alarm rates in the classification of underwater datasets using deep convolutional neural networks. Heliyon 2024; 10:e28681. [PMID: 38586386 PMCID: PMC10998124 DOI: 10.1016/j.heliyon.2024.e28681] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/31/2023] [Revised: 03/21/2024] [Accepted: 03/22/2024] [Indexed: 04/09/2024] Open
Abstract
Sonar sound datasets are of significant importance in the domains of underwater surveillance and marine research as they enable experts to discern intricate patterns within the depths of the water. Nevertheless, the task of classifying sonar sound datasets continues to pose significant challenges. In this study, we present a novel approach aimed at enhancing the precision and efficacy of sonar sound dataset classification. The integration of deep long-short-term memory (DLSTM) and convolutional neural networks (CNNs) models is employed in order to capitalize on their respective advantages while also utilizing distinctive feature engineering techniques to achieve the most favorable outcomes. Although DLSTM networks have demonstrated effectiveness in tasks involving sequence classification, attaining their optimal performance necessitates careful adjustment of hyperparameters. While traditional methods such as grid and random search are effective, they frequently encounter challenges related to computational inefficiencies. This study aims to investigate the unexplored capabilities of the fuzzy slime mould optimizer (FUZ-SMO) in the context of LSTM hyperparameter tuning, with the objective of addressing the existing research gap in this area. Drawing inspiration from the adaptive behavior exhibited by slime moulds, the FUZ-SMO proposes a novel approach to optimization. The amalgamated model, which combines CNN, LSTM, fuzzy, and SMO, exhibits a notable improvement in classification accuracy, outperforming conventional LSTM architectures by a margin of 2.142%. This model not only demonstrates accelerated convergence milestones but also displays significant resilience against overfitting tendencies.
Collapse
Affiliation(s)
- Dong liang Zhang
- School of Computer Science & Technology, Zhoukou Normal University, Zhoukou, 466001, Henan, China
| | - Zhiyong Jiang
- Engineering Comprehensive Training Center, Guilin University of Aerospace Technology, Guilin, 541004, Guangxi, China
| | - Fallah Mohammadzadeh
- Department of Electrical Engineering, Imam Khomeini Naval Science University, Nowshahr, Iran
| | | | - Laith Abualigah
- Hourani Center for Applied Scientific Research, Al-Ahliyya Amman University, Amman, 19328, Jordan
- Department of Electrical and Computer Engineering, Lebanese American University, Byblos, 13-5053, Lebanon
- MEU Research Unit, Middle East University, Amman, 11831, Jordan
- College of Engineering, Yuan Ze University, Taoyuan, Taiwan
- School of Computer Sciences, Universiti Sains Malaysia, Pulau Pinang, 11800, Malaysia
- School of Engineering and Technology, Sunway University Malaysia, Petaling Jaya, 27500, Malaysia
| | - Taher M. Ghazal
- Center for Cyber Physical Systems, Computer Science Department, Khalifa University, UAE
- Center for Cyber Security, Faculty of Information Science and Technology, Universiti KebangsaanMalaysia (UKM), Bangi, 43600, Malaysia
- Applied Science Research Center, Applied Science Private University, Amman, 11937, Jordan
| |
Collapse
|
16
|
Xu J, Wang Z. Intelligent classification and pollution characteristics analysis of microplastics in urban surface waters using YNet. J Hazard Mater 2024; 467:133694. [PMID: 38330648 DOI: 10.1016/j.jhazmat.2024.133694] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Received: 12/18/2023] [Revised: 01/23/2024] [Accepted: 01/31/2024] [Indexed: 02/10/2024]
Abstract
Microplastics (MPs, ≤ 5 mm in size) are hazardous contaminants that pose threats to ecosystems and human health. YNet was developed to analyze MPs abundance and shape to gain insights into MPs pollution characteristics in urban surface waters. The study found that YNet achieved an accurate identification and intelligent classification performance, with a dice similarity coefficient (DSC) of 90.78%, precision of 94.17%, and recall of 89.14%. Analysis of initial MPs levels in wetlands and reservoirs revealed 127.3 items/L and 56.0 items/L. Additionally, the MPs in effluents were 27.0 items/L and 26.3 items/L, indicating the ability of wetlands and reservoirs to retain MPs. The concentration of MPs in the lower reaches of the river was higher (45.6 items/L) compared to the upper reaches (22.0 items/L). The majority of MPs detected in this study were fragments, accounting for 51.63%, 54.94%, and 74.74% in the river, wetland, and reservoir. Conversely, granules accounted for the smallest proportion of MPs in the river, wetland, and reservoir, representing only 11.43%, 10.38%, and 6.5%. The study proves that the trained YNet accurately identify and intelligently classify MPs. This tool is essential in comprehending the distribution of MPs in urban surface waters and researching their sources and fate.
Collapse
Affiliation(s)
- Jiongji Xu
- School of Civil Engineering and Transportation, State Key Laboratory of Subtropical Building and Urban Science, South China University of Technology, Guangzhou 510641, China.
| | - Zhaoli Wang
- School of Civil Engineering and Transportation, State Key Laboratory of Subtropical Building and Urban Science, South China University of Technology, Guangzhou 510641, China; Pazhou Lab, Guangzhou 510335, China.
| |
Collapse
|
17
|
Grajales D, Le WT, Tran T, David S, Dallaire F, Ember K, Leblond F, Ménard C, Kadoury S. Robot-assisted biopsy sampling for online Raman spectroscopy cancer confirmation in the operating room. Int J Comput Assist Radiol Surg 2024:10.1007/s11548-024-03100-7. [PMID: 38573566 DOI: 10.1007/s11548-024-03100-7] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/22/2024] [Accepted: 03/04/2024] [Indexed: 04/05/2024]
Abstract
PURPOSE Cancer confirmation in the operating room (OR) is crucial to improve local control in cancer therapies. Histopathological analysis remains the gold standard, but there is a lack of real-time in situ cancer confirmation to support margin confirmation or remnant tissue. Raman spectroscopy (RS), as a label-free optical technique, has proven its power in cancer detection and, when integrated into a robotic assistance system, can positively impact the efficiency of procedures and the quality of life of patients, avoiding potential recurrence. METHODS A workflow is proposed where a 6-DOF robotic system (optical camera + MECA500 robotic arm) assists the characterization of fresh tissue samples using RS. Three calibration methods are compared for the robot, and the temporal efficiency is compared with standard hand-held analysis. For healthy/cancerous tissue discrimination, a 1D-convolutional neural network is proposed and tested on three ex vivo datasets (brain, breast, and prostate) containing processed RS and histopathology ground truth. RESULTS The robot achieves a minimum error of 0.20 mm (0.12) on a set of 30 test landmarks and demonstrates significant time reduction in 4 of the 5 proposed tasks. The proposed classification model can identify brain, breast, and prostate cancer with an accuracy of 0.83 (0.02), 0.93 (0.01), and 0.71 (0.01), respectively. CONCLUSION Automated RS analysis with deep learning demonstrates promising classification performance compared to commonly used support vector machines. Robotic assistance in tissue characterization can contribute to highly accurate, rapid, and robust biopsy analysis in the OR. These two elements are an important step toward real-time cancer confirmation using RS and OR integration.
Collapse
Affiliation(s)
- David Grajales
- Polytechnique Montréal, Montréal, QC, Canada.
- Centre de recherche du Centre Hospitalier de l'Université de Montréal, Montréal, QC, Canada.
| | - William T Le
- Polytechnique Montréal, Montréal, QC, Canada
- Centre de recherche du Centre Hospitalier de l'Université de Montréal, Montréal, QC, Canada
| | - Trang Tran
- Polytechnique Montréal, Montréal, QC, Canada
- Centre de recherche du Centre Hospitalier de l'Université de Montréal, Montréal, QC, Canada
| | - Sandryne David
- Polytechnique Montréal, Montréal, QC, Canada
- Centre de recherche du Centre Hospitalier de l'Université de Montréal, Montréal, QC, Canada
| | - Frédérick Dallaire
- Polytechnique Montréal, Montréal, QC, Canada
- Centre de recherche du Centre Hospitalier de l'Université de Montréal, Montréal, QC, Canada
| | - Katherine Ember
- Polytechnique Montréal, Montréal, QC, Canada
- Centre de recherche du Centre Hospitalier de l'Université de Montréal, Montréal, QC, Canada
| | - Frédéric Leblond
- Polytechnique Montréal, Montréal, QC, Canada
- Centre de recherche du Centre Hospitalier de l'Université de Montréal, Montréal, QC, Canada
- Institut du Cancer de Montréal, Montréal, QC, Canada
| | - Cynthia Ménard
- Centre de recherche du Centre Hospitalier de l'Université de Montréal, Montréal, QC, Canada
| | - Samuel Kadoury
- Polytechnique Montréal, Montréal, QC, Canada
- Centre de recherche du Centre Hospitalier de l'Université de Montréal, Montréal, QC, Canada
| |
Collapse
|
18
|
Borra D, Filippini M, Ursino M, Fattori P, Magosso E. Convolutional neural networks reveal properties of reach-to-grasp encoding in posterior parietal cortex. Comput Biol Med 2024; 172:108188. [PMID: 38492454 DOI: 10.1016/j.compbiomed.2024.108188] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/20/2023] [Revised: 01/26/2024] [Accepted: 02/18/2024] [Indexed: 03/18/2024]
Abstract
Deep neural networks (DNNs) are widely adopted to decode motor states from both non-invasively and invasively recorded neural signals, e.g., for realizing brain-computer interfaces. However, the neurophysiological interpretation of how DNNs make the decision based on the input neural activity is limitedly addressed, especially when applied to invasively recorded data. This reduces decoder reliability and transparency, and prevents the exploitation of decoders to better comprehend motor neural encoding. Here, we adopted an explainable artificial intelligence approach - based on a convolutional neural network and an explanation technique - to reveal spatial and temporal neural properties of reach-to-grasping from single-neuron recordings of the posterior parietal area V6A. The network was able to accurately decode 5 different grip types, and the explanation technique automatically identified the cells and temporal samples that most influenced the network prediction. Grip encoding in V6A neurons already started at movement preparation, peaking during movement execution. A difference was found within V6A: dorsal V6A neurons progressively encoded more for increasingly advanced grips, while ventral V6A neurons for increasingly rudimentary grips, with both subareas following a linear trend between the amount of grip encoding and the level of grip skills. By revealing the elements of the neural activity most relevant for each grip with no a priori assumptions, our approach supports and advances current knowledge about reach-to-grasp encoding in V6A, and it may represent a general tool able to investigate neural correlates of motor or cognitive tasks (e.g., attention and memory tasks) from single-neuron recordings.
Collapse
Affiliation(s)
- Davide Borra
- Department of Electrical, Electronic and Information Engineering "Guglielmo Marconi" (DEI), University of Bologna, Cesena Campus, Cesena, 47522, Italy.
| | - Matteo Filippini
- Department of Biomedical and Neuromotor Sciences (DIBINEM), University of Bologna, Bologna, 40126, Italy
| | - Mauro Ursino
- Department of Electrical, Electronic and Information Engineering "Guglielmo Marconi" (DEI), University of Bologna, Cesena Campus, Cesena, 47522, Italy; Alma Mater Research Institute for Human-Centered Artificial Intelligence, University of Bologna, Bologna, 40126, Italy
| | - Patrizia Fattori
- Alma Mater Research Institute for Human-Centered Artificial Intelligence, University of Bologna, Bologna, 40126, Italy; Department of Biomedical and Neuromotor Sciences (DIBINEM), University of Bologna, Bologna, 40126, Italy
| | - Elisa Magosso
- Department of Electrical, Electronic and Information Engineering "Guglielmo Marconi" (DEI), University of Bologna, Cesena Campus, Cesena, 47522, Italy; Alma Mater Research Institute for Human-Centered Artificial Intelligence, University of Bologna, Bologna, 40126, Italy
| |
Collapse
|
19
|
Tung CH, Hsiao YJ, Chen HL, Huang GR, Porcar L, Chang MC, Carrillo JM, Wang Y, Sumpter BG, Shinohara Y, Taylor J, Do C, Chen WR. Unveiling mesoscopic structures in distorted lamellar phases through deep learning-based small angle neutron scattering analysis. J Colloid Interface Sci 2024; 659:739-750. [PMID: 38211491 DOI: 10.1016/j.jcis.2024.01.003] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/08/2023] [Revised: 12/18/2023] [Accepted: 01/02/2024] [Indexed: 01/13/2024]
Abstract
HYPOTHESIS The formation of distorted lamellar phases, distinguished by their arrangement of crumpled, stacked layers, is frequently accompanied by the disruption of long-range order, leading to the formation of interconnected network structures commonly observed in the sponge phase. Nevertheless, traditional scattering functions grounded in deterministic modeling fall short of fully representing these intricate structural characteristics. Our hypothesis posits that a deep learning method, in conjunction with the generalized leveled wave approach used for describing structural features of distorted lamellar phases, can quantitatively unveil the inherent spatial correlations within these phases. EXPERIMENTS AND SIMULATIONS This report outlines a novel strategy that integrates convolutional neural networks and variational autoencoders, supported by stochastically generated density fluctuations, into a regression analysis framework for extracting structural features of distorted lamellar phases from small angle neutron scattering data. To evaluate the efficacy of our proposed approach, we conducted computational accuracy assessments and applied it to the analysis of experimentally measured small angle neutron scattering spectra of AOT surfactant solutions, a frequently studied lamellar system. FINDINGS The findings unambiguously demonstrate that deep learning provides a dependable and quantitative approach for investigating the morphology of wide variations of distorted lamellar phases. It is adaptable for deciphering structures from the lamellar to sponge phase including intermediate structures exhibiting fused topological features. This research highlights the effectiveness of deep learning methods in tackling complex issues in the field of soft matter structural analysis and beyond.
Collapse
Affiliation(s)
- Chi-Huan Tung
- Department of Chemical Engineering, National Tsing Hua University, Hsinchu, 30013, Taiwan
| | - Yu-Jung Hsiao
- Department of Chemical Engineering, National Tsing Hua University, Hsinchu, 30013, Taiwan
| | - Hsin-Lung Chen
- Department of Chemical Engineering, National Tsing Hua University, Hsinchu, 30013, Taiwan
| | - Guan-Rong Huang
- Department of Materials and Optoelectronic Science, National Sun Yat-sen University, Kaohsiung, 80424, Taiwan
| | - Lionel Porcar
- Institut Laue-Langevin, B.P. 156, F-38042 Grenoble Cedex 9, France
| | - Ming-Ching Chang
- Department of Computer Science, University at Albany - State University of New York, Albany, 12222, NY, United States
| | - Jan-Michael Carrillo
- Center for Nanophase Materials Sciences, Oak Ridge National Laboratory, Oak Ridge, 37831, TN, United States
| | - Yangyang Wang
- Center for Nanophase Materials Sciences, Oak Ridge National Laboratory, Oak Ridge, 37831, TN, United States
| | - Bobby G Sumpter
- Center for Nanophase Materials Sciences, Oak Ridge National Laboratory, Oak Ridge, 37831, TN, United States
| | - Yuya Shinohara
- Materials Science and Technology Division, Oak Ridge National Laboratory, Oak Ridge, 37831, TN, United States
| | - Jon Taylor
- Neutron Scattering Division, Oak Ridge National Laboratory, Oak Ridge, 37831, TN, United States
| | - Changwoo Do
- Neutron Scattering Division, Oak Ridge National Laboratory, Oak Ridge, 37831, TN, United States
| | - Wei-Ren Chen
- Neutron Scattering Division, Oak Ridge National Laboratory, Oak Ridge, 37831, TN, United States.
| |
Collapse
|
20
|
Abbasi M, Carvalho FG, Ribeiro B, Arrais JP. Predicting drug activity against cancer through genomic profiles and SMILES. Artif Intell Med 2024; 150:102820. [PMID: 38553160 DOI: 10.1016/j.artmed.2024.102820] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/05/2022] [Revised: 01/09/2024] [Accepted: 02/21/2024] [Indexed: 04/02/2024]
Abstract
Due to the constant increase in cancer rates, the disease has become a leading cause of death worldwide, enhancing the need for its detection and treatment. In the era of personalized medicine, the main goal is to incorporate individual variability in order to choose more precisely which therapy and prevention strategies suit each person. However, predicting the sensitivity of tumors to anticancer treatments remains a challenge. In this work, we propose two deep neural network models to predict the impact of anticancer drugs in tumors through the half-maximal inhibitory concentration (IC50). These models join biological and chemical data to apprehend relevant features of the genetic profile and the drug compounds, respectively. In order to predict the drug response in cancer cell lines, this study employed different DL methods, resorting to Recurrent Neural Networks (RNNs) and Convolutional Neural Networks (CNNs). In the first stage, two autoencoders were pre-trained with high-dimensional gene expression and mutation data of tumors. Afterward, this genetic background is transferred to the prediction models that return the IC50 value that portrays the potency of a substance in inhibiting a cancer cell line. When comparing RSEM Expected counts and TPM as methods for displaying gene expression data, RSEM has been shown to perform better in deep models and CNNs model can obtain better insight in these types of data. Moreover, the obtained results reflect the effectiveness of the extracted deep representations in the prediction of the IC50 value that portrays the potency of a substance in inhibiting a tumor, achieving a performance of a mean squared error of 1.06 and surpassing previous state-of-the-art models.
Collapse
Affiliation(s)
- Maryam Abbasi
- Centre for Informatics and Systems of the University of Coimbra, Department of Informatics Engineering, University of Coimbra, Coimbra, Portugal; Polytechnic Institute of Coimbra, Applied Research Institute, Coimbra, Portugal; Research Centre for Natural Resources Environment and Society (CERNAS), Polytechnic Institute of Coimbra, Coimbra, Portugal.
| | - Filipa G Carvalho
- Centre for Informatics and Systems of the University of Coimbra, Department of Informatics Engineering, University of Coimbra, Coimbra, Portugal
| | - Bernardete Ribeiro
- Centre for Informatics and Systems of the University of Coimbra, Department of Informatics Engineering, University of Coimbra, Coimbra, Portugal
| | - Joel P Arrais
- Centre for Informatics and Systems of the University of Coimbra, Department of Informatics Engineering, University of Coimbra, Coimbra, Portugal
| |
Collapse
|
21
|
Russo C, Bria A, Marrocco C. GravityNet for end-to-end small lesion detection. Artif Intell Med 2024; 150:102842. [PMID: 38553147 DOI: 10.1016/j.artmed.2024.102842] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/27/2023] [Revised: 03/01/2024] [Accepted: 03/11/2024] [Indexed: 04/02/2024]
Abstract
This paper introduces a novel one-stage end-to-end detector specifically designed to detect small lesions in medical images. Precise localization of small lesions presents challenges due to their appearance and the diverse contextual backgrounds in which they are found. To address this, our approach introduces a new type of pixel-based anchor that dynamically moves towards the targeted lesion for detection. We refer to this new architecture as GravityNet, and the novel anchors as gravity points since they appear to be "attracted" by the lesions. We conducted experiments on two well-established medical problems involving small lesions to evaluate the performance of the proposed approach: microcalcifications detection in digital mammograms and microaneurysms detection in digital fundus images. Our method demonstrates promising results in effectively detecting small lesions in these medical imaging tasks.
Collapse
Affiliation(s)
- Ciro Russo
- Department of Electrical and Information Engineering, University of Cassino and L.M., Via G. Di Biasio 43, 03043 Cassino (FR), Italy.
| | - Alessandro Bria
- Department of Electrical and Information Engineering, University of Cassino and L.M., Via G. Di Biasio 43, 03043 Cassino (FR), Italy.
| | - Claudio Marrocco
- Department of Electrical and Information Engineering, University of Cassino and L.M., Via G. Di Biasio 43, 03043 Cassino (FR), Italy.
| |
Collapse
|
22
|
Yüksek M, Yokuş A, Arslan H, Canayaz M, Akdemir Z. The Success of Deep Learning Modalities in Evaluating Modic Changes. World Neurosurg 2024; 184:e354-e359. [PMID: 38296043 DOI: 10.1016/j.wneu.2024.01.129] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/24/2023] [Revised: 01/22/2024] [Accepted: 01/23/2024] [Indexed: 03/01/2024]
Abstract
BACKGROUND Modic changes are pathologies that are common in the population and cause low back pain. The aim of the study is to analyze the modic changes detected in magnetic resonance imaging (MRI) using deep learning modalities. METHODS The sagittal T1, sagittal and axial T2-weighted lumbar MRI images of 307 patients, of which 125 were female and 182 were male, aged 19-86 years, who underwent MRI examination between 2016-2021 were analyzed. Modic changes (MC) were categorized and marked according to signal changes. Our study consists of 2 independent stages: classification and segmentation. The categorized data were first classified using convolutional neural network (CNN) architectures such as DenseNet-121, DenseNet-169, and VGG-19. In the next stage, masks were removed by segmentation using U-Net, which is the CNN architecture, with image processing programs on the marked images. RESULTS During the classification stage, the success rates for modic type 1, type 2, and type 3 changes were 98%, 96%, 100% in DenseNet-121, 100%, 94%, 100% in DenseNet-169, and 98%, 92%, 97% in VGG-19, respectively. At the segmentation phase, the success rate was 71% with the U-Net architecture. CONCLUSIONS Evaluation of MRI findings of MC in the etiology of lower back pain with deep learning architectures can significantly reduce the workload of the radiologist by providing ease of diagnosis.
Collapse
Affiliation(s)
- Mehmet Yüksek
- Department of Radiology, Van Training and Research Hospital, Van, Turkey
| | - Adem Yokuş
- Department of Radiology, Faculty of Medicine, Van Yüzüncü Yıl University, Van, Turkey.
| | - Harun Arslan
- Department of Radiology, Faculty of Medicine, Van Yüzüncü Yıl University, Van, Turkey
| | - Murat Canayaz
- Department of Computer Engineering, Faculty of Engineering, Van Yüzüncü Yıl University, Van, Turkey
| | - Zülküf Akdemir
- Department of Radiology, Faculty of Medicine, Van Yüzüncü Yıl University, Van, Turkey
| |
Collapse
|
23
|
Brady TF, Störmer VS. Comparing memory capacity across stimuli requires maximally dissimilar foils: Using deep convolutional neural networks to understand visual working memory capacity for real-world objects. Mem Cognit 2024; 52:595-609. [PMID: 37973770 DOI: 10.3758/s13421-023-01485-5] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 10/17/2023] [Indexed: 11/19/2023]
Abstract
The capacity of visual working and visual long-term memory plays a critical role in theories of cognitive architecture and the relationship between memory and other cognitive systems. Here, we argue that before asking the question of how capacity varies across different stimuli or what the upper bound of capacity is for a given memory system, it is necessary to establish a methodology that allows a fair comparison between distinct stimulus sets and conditions. One of the most important factors determining performance in a memory task is target/foil dissimilarity. We argue that only by maximizing the dissimilarity of the target and foil in each stimulus set can we provide a fair basis for memory comparisons between stimuli. In the current work we focus on a way to pick such foils objectively for complex, meaningful real-world objects by using deep convolutional neural networks, and we validate this using both memory tests and similarity metrics. Using this method, we then provide evidence that there is a greater capacity for real-world objects relative to simple colors in visual working memory; critically, we also show that this difference can be reduced or eliminated when non-comparable foils are used, potentially explaining why previous work has not always found such a difference. Our study thus demonstrates that working memory capacity depends on the type of information that is remembered and that assessing capacity depends critically on foil dissimilarity, especially when comparing memory performance and other cognitive systems across different stimulus sets.
Collapse
Affiliation(s)
- Timothy F Brady
- Department of Psychology, University of California San Diego, La Jolla, CA, 92093, USA.
| | - Viola S Störmer
- Department of Psychological and Brain Sciences, Dartmouth College, Hanover, NH, USA
| |
Collapse
|
24
|
Gu H, Wang S, Hu S, Wu X, Li Q, Zhang R, Zhang J, Zhang W, Peng Y. Identification of Panax notoginseng origin using terahertz precision spectroscopy and neural network algorithm. Talanta 2024; 274:125968. [PMID: 38581849 DOI: 10.1016/j.talanta.2024.125968] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/09/2024] [Revised: 03/11/2024] [Accepted: 03/20/2024] [Indexed: 04/08/2024]
Abstract
Panax notoginseng (P. notoginseng), a Chinese herb containing various saponins, benefits immune system in medicines development, which from Wenshan (authentic cultivation) is often counterfeited by others for large demand and limited supply. Here, we proposed a method for identifying P. notoginseng origin combining terahertz (THz) precision spectroscopy and neural network. Based on the comparative analysis of four qualitative identification methods, we chose high-performance liquid chromatography (HPLC) and THz spectroscopy to detect 252 samples from five origins. After classifications using Convolutional Neural Networks (CNNs) model, we found that the performance of THz spectra was superior to that of HPLC. The underlying mechanism is that there are clear nonlinear relations among the THz spectra and the origins due to the wide spectra and multi-parameter characteristics, which makes the accuracy of five-classification origin identification up to 97.62%. This study realizes the rapid, non-destructive and accurate identification of P. notoginseng origin, providing a practical reference for herbal medicine.
Collapse
Affiliation(s)
- Hongyu Gu
- University of Shanghai for Science and Technology, Terahertz Technology Innovation Research Institute, Shanghai Key Lab of Modern Optical System, Shanghai Institute of Intelligent Science and Technology, Shanghai, 200093, China
| | - Shengfeng Wang
- University of Shanghai for Science and Technology, Terahertz Technology Innovation Research Institute, Shanghai Key Lab of Modern Optical System, Shanghai Institute of Intelligent Science and Technology, Shanghai, 200093, China
| | - Songyan Hu
- University of Shanghai for Science and Technology, Terahertz Technology Innovation Research Institute, Shanghai Key Lab of Modern Optical System, Shanghai Institute of Intelligent Science and Technology, Shanghai, 200093, China
| | - Xu Wu
- University of Shanghai for Science and Technology, Terahertz Technology Innovation Research Institute, Shanghai Key Lab of Modern Optical System, Shanghai Institute of Intelligent Science and Technology, Shanghai, 200093, China
| | - Qiuye Li
- Wenshan Institute for Food and Drug Control, Yunnan, 663099, China
| | - Rongrong Zhang
- Wenshan Institute for Food and Drug Control, Yunnan, 663099, China
| | - Juan Zhang
- Wenshan Institute for Food and Drug Control, Yunnan, 663099, China
| | - Wenbin Zhang
- Wenshan Sanqi Institute of Science and Technology, China
| | - Yan Peng
- University of Shanghai for Science and Technology, Terahertz Technology Innovation Research Institute, Shanghai Key Lab of Modern Optical System, Shanghai Institute of Intelligent Science and Technology, Shanghai, 200093, China.
| |
Collapse
|
25
|
Lancia G, Varkila MRJ, Cremer OL, Spitoni C. Two-step interpretable modeling of ICU-AIs. Artif Intell Med 2024; 151:102862. [PMID: 38579437 DOI: 10.1016/j.artmed.2024.102862] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/02/2023] [Revised: 03/25/2024] [Accepted: 03/25/2024] [Indexed: 04/07/2024]
Abstract
We present a novel methodology for integrating high resolution longitudinal data with the dynamic prediction capabilities of survival models. The aim is two-fold: to improve the predictive power while maintaining the interpretability of the models. To go beyond the black box paradigm of artificial neural networks, we propose a parsimonious and robust semi-parametric approach (i.e., a landmarking competing risks model) that combines routinely collected low-resolution data with predictive features extracted from a convolutional neural network, that was trained on high resolution time-dependent information. We then use saliency maps to analyze and explain the extra predictive power of this model. To illustrate our methodology, we focus on healthcare-associated infections in patients admitted to an intensive care unit.
Collapse
Affiliation(s)
- G Lancia
- Mathematics Department, Utrecht University, Budapestlaan, 6, Utrecht, 3584CD, The Netherlands.
| | - M R J Varkila
- Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Universiteitsweg 100, Utrecht, 3584 CG, The Netherlands
| | - O L Cremer
- Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Universiteitsweg 100, Utrecht, 3584 CG, The Netherlands
| | - C Spitoni
- Mathematics Department, Utrecht University, Budapestlaan, 6, Utrecht, 3584CD, The Netherlands
| |
Collapse
|
26
|
Barash Y, Livne A, Klang E, Sorin V, Cohen I, Khaitovich B, Raskin D. Artificial Intelligence for Identification of Images with Active Bleeding in Mesenteric and Celiac Arteries Angiography. Cardiovasc Intervent Radiol 2024:10.1007/s00270-024-03689-x. [PMID: 38530394 DOI: 10.1007/s00270-024-03689-x] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 09/01/2023] [Accepted: 02/20/2024] [Indexed: 03/28/2024]
Abstract
PURPOSE The purpose of this study is to evaluate the efficacy of an artificial intelligence (AI) model designed to identify active bleeding in digital subtraction angiography images for upper gastrointestinal bleeding. METHODS Angiographic images were retrospectively collected from mesenteric and celiac artery embolization procedures performed between 2018 and 2022. This dataset included images showing both active bleeding and non-bleeding phases from the same patients. The images were labeled as normal versus images that contain active bleeding. A convolutional neural network was trained and validated to automatically classify the images. Algorithm performance was tested in terms of area under the curve, accuracy, sensitivity, specificity, F1 score, positive and negative predictive value. RESULTS The dataset included 587 pre-labeled images from 142 patients. Of these, 302 were labeled as normal angiogram and 285 as containing active bleeding. The model's performance on the validation cohort was area under the curve 85.0 ± 10.9% (standard deviation) and average classification accuracy 77.43 ± 4.9%. For Youden's index cutoff, sensitivity and specificity were 85.4 ± 9.4% and 81.2 ± 8.6%, respectively. CONCLUSION In this study, we explored the application of AI in mesenteric and celiac artery angiography for detecting active bleeding. The results of this study show the potential of an AI-based algorithm to accurately classify images with active bleeding. Further studies using a larger dataset are needed to improve accuracy and allow segmentation of the bleeding.
Collapse
Affiliation(s)
- Yiftach Barash
- Department of Diagnostic Imaging, Chaim Sheba Medical Center, Emek Haela St. 1, 52621, Ramat Gan, Israel.
- The Faculty of Medicine, Tel-Aviv University, Tel Aviv, Israel.
- DeepVision Lab, Chaim Sheba Medical Center, Emek Haela St. 1, 52621, Ramat Gan, Israel.
| | - Adva Livne
- Department of Diagnostic Imaging, Chaim Sheba Medical Center, Emek Haela St. 1, 52621, Ramat Gan, Israel
- DeepVision Lab, Chaim Sheba Medical Center, Emek Haela St. 1, 52621, Ramat Gan, Israel
| | - Eyal Klang
- Department of Diagnostic Imaging, Chaim Sheba Medical Center, Emek Haela St. 1, 52621, Ramat Gan, Israel
- The Faculty of Medicine, Tel-Aviv University, Tel Aviv, Israel
- DeepVision Lab, Chaim Sheba Medical Center, Emek Haela St. 1, 52621, Ramat Gan, Israel
- Sami Sagol AI Hub, ARC, Chaim Sheba Medical Center, Emek Haela St. 1, 52621, Ramat Gan, Israel
| | - Vera Sorin
- Department of Diagnostic Imaging, Chaim Sheba Medical Center, Emek Haela St. 1, 52621, Ramat Gan, Israel
- The Faculty of Medicine, Tel-Aviv University, Tel Aviv, Israel
- DeepVision Lab, Chaim Sheba Medical Center, Emek Haela St. 1, 52621, Ramat Gan, Israel
| | - Israel Cohen
- Department of Diagnostic Imaging, Chaim Sheba Medical Center, Emek Haela St. 1, 52621, Ramat Gan, Israel
- The Faculty of Medicine, Tel-Aviv University, Tel Aviv, Israel
| | - Boris Khaitovich
- Department of Diagnostic Imaging, Chaim Sheba Medical Center, Emek Haela St. 1, 52621, Ramat Gan, Israel
- The Faculty of Medicine, Tel-Aviv University, Tel Aviv, Israel
| | - Daniel Raskin
- Department of Diagnostic Imaging, Chaim Sheba Medical Center, Emek Haela St. 1, 52621, Ramat Gan, Israel
- The Faculty of Medicine, Tel-Aviv University, Tel Aviv, Israel
| |
Collapse
|
27
|
Patel Y, Shah T, Dhar MK, Zhang T, Niezgoda J, Gopalakrishnan S, Yu Z. Integrated image and location analysis for wound classification: a deep learning approach. Sci Rep 2024; 14:7043. [PMID: 38528003 DOI: 10.1038/s41598-024-56626-w] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/01/2023] [Accepted: 03/08/2024] [Indexed: 03/27/2024] Open
Abstract
The global burden of acute and chronic wounds presents a compelling case for enhancing wound classification methods, a vital step in diagnosing and determining optimal treatments. Recognizing this need, we introduce an innovative multi-modal network based on a deep convolutional neural network for categorizing wounds into four categories: diabetic, pressure, surgical, and venous ulcers. Our multi-modal network uses wound images and their corresponding body locations for more precise classification. A unique aspect of our methodology is incorporating a body map system that facilitates accurate wound location tagging, improving upon traditional wound image classification techniques. A distinctive feature of our approach is the integration of models such as VGG16, ResNet152, and EfficientNet within a novel architecture. This architecture includes elements like spatial and channel-wise Squeeze-and-Excitation modules, Axial Attention, and an Adaptive Gated Multi-Layer Perceptron, providing a robust foundation for classification. Our multi-modal network was trained and evaluated on two distinct datasets comprising relevant images and corresponding location information. Notably, our proposed network outperformed traditional methods, reaching an accuracy range of 74.79-100% for Region of Interest (ROI) without location classifications, 73.98-100% for ROI with location classifications, and 78.10-100% for whole image classifications. This marks a significant enhancement over previously reported performance metrics in the literature. Our results indicate the potential of our multi-modal network as an effective decision-support tool for wound image classification, paving the way for its application in various clinical contexts.
Collapse
Affiliation(s)
- Yash Patel
- Department of Computer Science, University of Wisconsin-Milwaukee, Milwaukee, WI, USA
| | - Tirth Shah
- Department of Computer Science, University of Wisconsin-Milwaukee, Milwaukee, WI, USA
| | - Mrinal Kanti Dhar
- Department of Computer Science, University of Wisconsin-Milwaukee, Milwaukee, WI, USA
| | - Taiyu Zhang
- Department of Computer Science, University of Wisconsin-Milwaukee, Milwaukee, WI, USA
| | - Jeffrey Niezgoda
- Advancing the Zenith of Healthcare (AZH) Wound and Vascular Center, Milwaukee, WI, USA
| | | | - Zeyun Yu
- Department of Computer Science, University of Wisconsin-Milwaukee, Milwaukee, WI, USA.
- Department of Biomedical Engineering, University of Wisconsin-Milwaukee, Milwaukee, WI, USA.
| |
Collapse
|
28
|
Iwata H, Hayashi Y, Koyama T, Hasegawa A, Ohgi K, Kobayashi I, Okuno Y. Feature extraction of particle morphologies of pharmaceutical excipients from scanning electron microscope images using convolutional neural networks. Int J Pharm 2024; 653:123873. [PMID: 38336179 DOI: 10.1016/j.ijpharm.2024.123873] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/23/2023] [Revised: 01/08/2024] [Accepted: 01/30/2024] [Indexed: 02/12/2024]
Abstract
Scanning electron microscopy (SEM) images are the most widely used tool for evaluating particle morphology; however, quantitative evaluation using SEM images is time-consuming and often neglected. In this study, we aimed to extract features related to particle morphology of pharmaceutical excipients from SEM images using a convolutional neural network (CNN). SEM images of 67 excipients were acquired and used as models. A classification CNN model of the excipients was constructed based on the SEM images. Further, features were extracted from the middle layer of this CNN model, and the data was compressed to two dimensions using uniform manifold approximation and projection. Lastly, hierarchical clustering analysis (HCA) was performed to categorize the excipients into several clusters and identify similarities among the samples. The classification CNN model showed high accuracy, allowing each excipient to be identified with a high degree of accuracy. HCA revealed that the 67 excipients were classified into seven clusters. Additionally, the particle morphologies of excipients belonging to the same cluster were found to be very similar. These results suggest that CNN models are useful tools for extracting information and identifying similarities among the particle morphologies of excipients.
Collapse
Affiliation(s)
- Hiroaki Iwata
- Graduate School of Medicine, Kyoto University, 53 Shogoin-kawaharacho, Sakyo-ku, Kyoto 606-8507, Japan.
| | - Yoshihiro Hayashi
- Graduate School of Medicine, Kyoto University, 53 Shogoin-kawaharacho, Sakyo-ku, Kyoto 606-8507, Japan; Pharmaceutical Technology Management Department, Production Division, Nichi-Iko Pharmaceutical Co., Ltd., 205-1, Shimoumezawa Namerikawa-shi, Toyama 936-0857, Japan.
| | - Takuto Koyama
- Graduate School of Medicine, Kyoto University, 53 Shogoin-kawaharacho, Sakyo-ku, Kyoto 606-8507, Japan
| | - Aki Hasegawa
- Graduate School of Medicine, Kyoto University, 53 Shogoin-kawaharacho, Sakyo-ku, Kyoto 606-8507, Japan
| | - Kosuke Ohgi
- Formulation Development Department, Development & Planning Division, Nichi-Iko Pharmaceutical Co., Ltd., 205-1, Shimoumezawa Namerikawa-shi, Toyama 936-0857, Japan
| | - Ippei Kobayashi
- Formulation Development Department, Development & Planning Division, Nichi-Iko Pharmaceutical Co., Ltd., 205-1, Shimoumezawa Namerikawa-shi, Toyama 936-0857, Japan
| | - Yasushi Okuno
- Graduate School of Medicine, Kyoto University, 53 Shogoin-kawaharacho, Sakyo-ku, Kyoto 606-8507, Japan; RIKEN Center for Computational Science, Kobe 650-0047, Japan
| |
Collapse
|
29
|
Le VT, Malik MS, Tseng YH, Lee YC, Huang CI, Ou YY. DeepPLM_mCNN: An approach for enhancing ion channel and ion transporter recognition by multi-window CNN based on features from pre-trained language models. Comput Biol Chem 2024; 110:108055. [PMID: 38555810 DOI: 10.1016/j.compbiolchem.2024.108055] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/24/2023] [Revised: 02/28/2024] [Accepted: 03/19/2024] [Indexed: 04/02/2024]
Abstract
Accurate classification of membrane proteins like ion channels and transporters is critical for elucidating cellular processes and drug development. We present DeepPLM_mCNN, a novel framework combining Pretrained Language Models (PLMs) and multi-window convolutional neural networks (mCNNs) for effective classification of membrane proteins into ion channels and ion transporters. Our approach extracts informative features from protein sequences by utilizing various PLMs, including TAPE, ProtT5_XL_U50, ESM-1b, ESM-2_480, and ESM-2_1280. These PLM-derived features are then input into a mCNN architecture to learn conserved motifs important for classification. When evaluated on ion transporters, our best performing model utilizing ProtT5 achieved 90% sensitivity, 95.8% specificity, and 95.4% overall accuracy. For ion channels, we obtained 88.3% sensitivity, 95.7% specificity, and 95.2% overall accuracy using ESM-1b features. Our proposed DeepPLM_mCNN framework demonstrates significant improvements over previous methods on unseen test data. This study illustrates the potential of combining PLMs and deep learning for accurate computational identification of membrane proteins from sequence data alone. Our findings have important implications for membrane protein research and drug development targeting ion channels and transporters. The data and source codes in this study are publicly available at the following link: https://github.com/s1129108/DeepPLM_mCNN.
Collapse
Affiliation(s)
- Van-The Le
- Department of Computer Science and Engineering, Yuan Ze University, Chung-Li, 32003, Taiwan
| | - Muhammad-Shahid Malik
- Department of Computer Science and Engineering, Yuan Ze University, Chung-Li, 32003, Taiwan; Department of Computer Science and Engineering, Karakoram International University, Pakistan
| | - Yi-Hsuan Tseng
- Department of Computer Science and Engineering, Yuan Ze University, Chung-Li, 32003, Taiwan
| | - Yu-Cheng Lee
- Department of Computer Science and Engineering, Yuan Ze University, Chung-Li, 32003, Taiwan
| | - Cheng-I Huang
- Department of Computer Science and Engineering, Yuan Ze University, Chung-Li, 32003, Taiwan
| | - Yu-Yen Ou
- Department of Computer Science and Engineering, Yuan Ze University, Chung-Li, 32003, Taiwan; Graduate Program in Biomedical Informatics, Yuan Ze University, Chung-Li, 32003, Taiwan.
| |
Collapse
|
30
|
Gu Z, Dai W, Chen J, Jiang Q, Lin W, Wang Q, Chen J, Gu C, Li J, Ying G, Zhu Y. Convolutional neural network-based magnetic resonance image differentiation of filum terminale ependymomas from schwannomas. BMC Cancer 2024; 24:350. [PMID: 38504164 PMCID: PMC10949807 DOI: 10.1186/s12885-024-12023-0] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/18/2023] [Accepted: 02/20/2024] [Indexed: 03/21/2024] Open
Abstract
PURPOSE Preoperative diagnosis of filum terminale ependymomas (FTEs) versus schwannomas is difficult but essential for surgical planning and prognostic assessment. With the advancement of deep-learning approaches based on convolutional neural networks (CNNs), the aim of this study was to determine whether CNN-based interpretation of magnetic resonance (MR) images of these two tumours could be achieved. METHODS Contrast-enhanced MRI data from 50 patients with primary FTE and 50 schwannomas in the lumbosacral spinal canal were retrospectively collected and used as training and internal validation datasets. The diagnostic accuracy of MRI was determined by consistency with postoperative histopathological examination. T1-weighted (T1-WI), T2-weighted (T2-WI) and contrast-enhanced T1-weighted (CE-T1) MR images of the sagittal plane containing the tumour mass were selected for analysis. For each sequence, patient MRI data were randomly allocated to 5 groups that further underwent fivefold cross-validation to evaluate the diagnostic efficacy of the CNN models. An additional 34 pairs of cases were used as an external test dataset to validate the CNN classifiers. RESULTS After comparing multiple backbone CNN models, we developed a diagnostic system using Inception-v3. In the external test dataset, the per-examination combined sensitivities were 0.78 (0.71-0.84, 95% CI) based on T1-weighted images, 0.79 (0.72-0.84, 95% CI) for T2-weighted images, 0.88 (0.83-0.92, 95% CI) for CE-T1 images, and 0.88 (0.83-0.92, 95% CI) for all weighted images. The combined specificities were 0.72 based on T1-WI (0.66-0.78, 95% CI), 0.84 (0.78-0.89, 95% CI) based on T2-WI, 0.74 (0.67-0.80, 95% CI) for CE-T1, and 0.81 (0.76-0.86, 95% CI) for all weighted images. After all three MRI modalities were merged, the receiver operating characteristic (ROC) curve was calculated, and the area under the curve (AUC) was 0.93, with an accuracy of 0.87. CONCLUSIONS CNN based MRI analysis has the potential to accurately differentiate ependymomas from schwannomas in the lumbar segment.
Collapse
Affiliation(s)
- Zhaowen Gu
- Department of Neurosurgery, The Second Affiliated Hospital, School of Medicine, Zhejiang University, 88, Jiefang Road, Hangzhou, China
| | - Wenli Dai
- Zhejiang University School of Mathematical Sciences, Hangzhou, Zhejiang, China
| | - Jiarui Chen
- Department of Neurosurgery, The Second Affiliated Hospital, School of Medicine, Zhejiang University, 88, Jiefang Road, Hangzhou, China
| | - Qixuan Jiang
- Department of Neurosurgery, The Second Affiliated Hospital, School of Medicine, Zhejiang University, 88, Jiefang Road, Hangzhou, China
| | - Weiwei Lin
- Department of Neurosurgery, The Second Affiliated Hospital, School of Medicine, Zhejiang University, 88, Jiefang Road, Hangzhou, China
| | - Qiangwei Wang
- Department of Neurosurgery, The Second Affiliated Hospital, School of Medicine, Zhejiang University, 88, Jiefang Road, Hangzhou, China
| | - Jingyin Chen
- Department of Neurosurgery, The Second Affiliated Hospital, School of Medicine, Zhejiang University, 88, Jiefang Road, Hangzhou, China
| | - Chi Gu
- Department of Neurosurgery, The Second Affiliated Hospital, School of Medicine, Zhejiang University, 88, Jiefang Road, Hangzhou, China
| | - Jia Li
- Ningbo Medical Center Lihuili Hospital, Department of Neurosurgery, Ningbo University, 1111, Jiangnan Road, Ningbo, Zhejiang, China.
| | - Guangyu Ying
- Department of Neurosurgery, The Second Affiliated Hospital, School of Medicine, Zhejiang University, 88, Jiefang Road, Hangzhou, China.
| | - Yongjian Zhu
- Department of Neurosurgery, The Second Affiliated Hospital, School of Medicine, Zhejiang University, 88, Jiefang Road, Hangzhou, China.
- Clinical Research Center for Neurological Diseases of Zhejiang Province, Hangzhou, China.
| |
Collapse
|
31
|
Bhardwaj N, Sood M, Gill SS. Design and Development of Hypertuned Deep learning Frameworks for Detection and Severity Grading of Brain Tumor using Medical Brain MR images. Curr Med Imaging 2024; 20:CMIR-EPUB-139196. [PMID: 38494939 DOI: 10.2174/0115734056288248240309044616] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/06/2023] [Revised: 02/22/2024] [Accepted: 02/27/2024] [Indexed: 03/19/2024]
Abstract
BACKGROUND Brain tumor is a grave illness causing worldwide fatalities. The current detection methods for brain tumors are manual, invasive, and rely on histopathological analysis. Determining the type of brain tumor after its detection relies on biopsy measures and involves human subjectivity. The use of automated CAD techniques for brain tumor detection and classification can overcome these drawbacks. OBJECTIVE The paper aims to create two deep learning-based CAD frameworks for automatic detection and severity grading of brain tumors - the first model for brain tumor detection in brain MR images and model 2 for the classification of tumors into three types: Glioma, Meningioma, and Pituitary based on severity grading. METHODS The novelty of the research work includes the architectural design of deep learning frameworks for detection and classification of brain tumor using brain MR images. The hyperparameter tuning of the proposed models is done to achieve the optimal parameters that result in maximizing the models' performance and minimizing losses. RESULTS The proposed CNN models outperform the existing state of the art models in terms of accuracy and complexity of the models. The proposed model developed for detection of brain tumors achieved an accuracy of 98.56% and CNN Model developed for severity grading of brain tumor achieved an accuracy of 92.36% on BraTs dataset. CONCLUSION The proposed models have an edge over the existing CNN models in terms of less complexity of the structure and appreciable accuracy with low training and test errors. The proposed CNN Models can be employed for clinical diagnostic purposes to aid the medical fraternity in validating their initial screening for brain tumor detection and its multi-classification.
Collapse
Affiliation(s)
- Neha Bhardwaj
- National Institute of Technical Teachers Training and Research, Electronics & Communication Engineering, India
| | - Meenakshi Sood
- National Institute of Technical Teachers Training and Research, CDC Department, India
| | - Sandeep Singh Gill
- National Institute of Technical Teachers Training and Research, IMEE Department, India
| |
Collapse
|
32
|
Dang TH, Vu TA. xCAPT5: protein-protein interaction prediction using deep and wide multi-kernel pooling convolutional neural networks with protein language model. BMC Bioinformatics 2024; 25:106. [PMID: 38461247 PMCID: PMC10924985 DOI: 10.1186/s12859-024-05725-6] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/05/2024] [Accepted: 02/28/2024] [Indexed: 03/11/2024] Open
Abstract
BACKGROUND Predicting protein-protein interactions (PPIs) from sequence data is a key challenge in computational biology. While various computational methods have been proposed, the utilization of sequence embeddings from protein language models, which contain diverse information, including structural, evolutionary, and functional aspects, has not been fully exploited. Additionally, there is a significant need for a comprehensive neural network capable of efficiently extracting these multifaceted representations. RESULTS Addressing this gap, we propose xCAPT5, a novel hybrid classifier that uniquely leverages the T5-XL-UniRef50 protein large language model for generating rich amino acid embeddings from protein sequences. The core of xCAPT5 is a multi-kernel deep convolutional siamese neural network, which effectively captures intricate interaction features at both micro and macro levels, integrated with the XGBoost algorithm, enhancing PPIs classification performance. By concatenating max and average pooling features in a depth-wise manner, xCAPT5 effectively learns crucial features with low computational cost. CONCLUSION This study represents one of the initial efforts to extract informative amino acid embeddings from a large protein language model using a deep and wide convolutional network. Experimental results show that xCAPT5 outperforms recent state-of-the-art methods in binary PPI prediction, excelling in cross-validation on several benchmark datasets and demonstrating robust generalization across intra-species, cross-species, inter-species, and stringent similarity contexts.
Collapse
Affiliation(s)
- Thanh Hai Dang
- Faculty of Information Technology, VNU University of Engineering and Technology, 144 Xuan Thuy, Hanoi, 10000, Vietnam.
| | - Tien Anh Vu
- Faculty of Biology, VNU University of Science, 334 Nguyen Trai, Hanoi, 10000, Vietnam
| |
Collapse
|
33
|
Morita K, Karashima S, Terao T, Yoshida K, Yamashita T, Yoroidaka T, Tanabe M, Imi T, Zaimoku Y, Yoshida A, Maruyama H, Iwaki N, Aoki G, Kotani T, Murata R, Miyamoto T, Machida Y, Matsue K, Nambo H, Takamatsu H. 3D CNN-based Deep Learning Model-based Explanatory Prognostication in Patients with Multiple Myeloma using Whole-body MRI. J Med Syst 2024; 48:30. [PMID: 38456950 DOI: 10.1007/s10916-024-02040-8] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/25/2023] [Accepted: 02/05/2024] [Indexed: 03/09/2024]
Abstract
Although magnetic resonance imaging (MRI) data of patients with multiple myeloma (MM) are used to predict prognosis, few reports have applied artificial intelligence (AI) techniques for this purpose. We aimed to analyze whole-body diffusion-weighted MRI data using three-dimensional (3D) convolutional neural networks (CNNs) and Gradient-weighted Class Activation Mapping (Grad-CAM), an explainable AI, to predict prognosis and explore the factors involved in prediction. We retrospectively analyzed the MRI data of a total of 142 patients with MM obtained from two medical centers. We defined the occurrence of progressive disease after MRI evaluation within 12 months as a poor prognosis and constructed a 3D CNN-based deep learning model to predict prognosis. Images from 111 cases were used as the training and internal validation data; images from 31 cases were used as the external validation data. Internal validation of the AI model with stratified 5-fold cross-validation resulted in a significant difference in progression-free survival (PFS) between good and poor prognostic cases (2-year PFS, 91.2% versus [vs.] 61.1%, P = 0.0002). The AI model clearly stratified good and poor prognostic cases in the external validation cohort (2-year PFS, 92.9% vs. 55.6%, P = 0.004), with an area under the receiver operating characteristic curve of 0.804. According to Grad-CAM, the MRI signals of the spleen and bones of the vertebrae and pelvis contributed to prognosis prediction. This study is the first to show that image analysis of whole-body MRI using a 3D CNN without any other clinical data is effective in predicting the prognosis of patients with MM.
Collapse
Affiliation(s)
- Kento Morita
- School of Electrical, Information and Communication Engineering, Kanazawa University, Kakumamachi, Kanazawa, Ishikawa, 920-1192, Japan
| | | | - Toshiki Terao
- Department of Hematology/Oncology, Kameda Medical Center, Kamogawa, Japan
- Department of Hematology and Oncology, Okayama University Hospital, Okayama, Japan
| | - Kotaro Yoshida
- Department of Radiology, Kanazawa University, Kanazawa, Japan
| | - Takeshi Yamashita
- Division of Internal Medicine, Keiju Kanazawa Hospital, Kanazawa, Japan
| | - Takeshi Yoroidaka
- Department of Hematology, Ishikawa Central Prefectural Hospital, Kanazawa, Japan
- Department of Hematology, Faculty of Medicine, Institute of Medical, Pharmaceutical, and Health Sciences, Kanazawa University, Kanazawa, Japan
| | - Mikoto Tanabe
- Department of Hematology, Ishikawa Central Prefectural Hospital, Kanazawa, Japan
| | - Tatsuya Imi
- Department of Hematology, Faculty of Medicine, Institute of Medical, Pharmaceutical, and Health Sciences, Kanazawa University, Kanazawa, Japan
| | - Yoshitaka Zaimoku
- Department of Hematology, Faculty of Medicine, Institute of Medical, Pharmaceutical, and Health Sciences, Kanazawa University, Kanazawa, Japan
| | - Akiyo Yoshida
- Department of Hematology, Faculty of Medicine, Institute of Medical, Pharmaceutical, and Health Sciences, Kanazawa University, Kanazawa, Japan
| | - Hiroyuki Maruyama
- Department of Hematology, Faculty of Medicine, Institute of Medical, Pharmaceutical, and Health Sciences, Kanazawa University, Kanazawa, Japan
| | - Noriko Iwaki
- Department of Hematology, Faculty of Medicine, Institute of Medical, Pharmaceutical, and Health Sciences, Kanazawa University, Kanazawa, Japan
| | - Go Aoki
- Department of Hematology, Faculty of Medicine, Institute of Medical, Pharmaceutical, and Health Sciences, Kanazawa University, Kanazawa, Japan
| | - Takeharu Kotani
- Department of Hematology, Ishikawa Central Prefectural Hospital, Kanazawa, Japan
| | - Ryoichi Murata
- Division of Internal Medicine, Keiju Kanazawa Hospital, Kanazawa, Japan
| | - Toshihiro Miyamoto
- Department of Hematology, Faculty of Medicine, Institute of Medical, Pharmaceutical, and Health Sciences, Kanazawa University, Kanazawa, Japan
| | - Youichi Machida
- Department of Radiology, Kameda Medical Center, Kamogawa, Japan
| | - Kosei Matsue
- Department of Hematology/Oncology, Kameda Medical Center, Kamogawa, Japan
| | - Hidetaka Nambo
- Faculty of Transdisciplinary Sciences for Innovation, Institute of Transdisciplinary Sciences for Innovation, Kanazawa University, Kakumamachi, Kanazawa, Ishikawa, 920-1192, Japan.
| | - Hiroyuki Takamatsu
- Department of Hematology, Faculty of Medicine, Institute of Medical, Pharmaceutical, and Health Sciences, Kanazawa University, Kanazawa, Japan.
- Faculty of Transdisciplinary Sciences for Innovation, Institute of Transdisciplinary Sciences for Innovation, Kanazawa University, Kakumamachi, Kanazawa, Ishikawa, 920-1192, Japan.
| |
Collapse
|
34
|
Siami M, Barszcz T, Wodecki J, Zimroz R. Semantic segmentation of thermal defects in belt conveyor idlers using thermal image augmentation and U-Net-based convolutional neural networks. Sci Rep 2024; 14:5748. [PMID: 38459162 PMCID: PMC10923815 DOI: 10.1038/s41598-024-55864-2] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/12/2023] [Accepted: 02/28/2024] [Indexed: 03/10/2024] Open
Abstract
The belt conveyor (BC) is the main means of horizontal transportation of bulk materials at mining sites. The sudden fault in BC modules may cause unexpected stops in production lines. With the increasing number of applications of inspection mobile robots in condition monitoring (CM) of industrial infrastructure in hazardous environments, in this article we introduce an image processing pipeline for automatic segmentation of thermal defects in thermal images captured from BC idlers using a mobile robot. This study follows the fact that CM of idler temperature is an important task for preventing sudden breakdowns in BC system networks. We compared the performance of three different types of U-Net-based convolutional neural network architectures for the identification of thermal anomalies using a small number of hand-labeled thermal images. Experiments on the test data set showed that the attention residual U-Net with binary cross entropy as the loss function handled the semantic segmentation problem better than our previous research and other studied U-Net variations.
Collapse
Affiliation(s)
- Mohammad Siami
- AMC Vibro Sp. z o.o., Pilotow 2e, 31-462, Kraków, Poland.
| | - Tomasz Barszcz
- Faculty of Mechanical Engineering and Robotics, AGH University, Al. Mickiewicza 30, 30-059, Kraków, Poland
| | - Jacek Wodecki
- Faculty of Geoengineering, Mining and Geology, Wroclaw University of Science and Technology, Na Grobli 15, 50-421, Wroclaw, Poland
| | - Radoslaw Zimroz
- Faculty of Geoengineering, Mining and Geology, Wroclaw University of Science and Technology, Na Grobli 15, 50-421, Wroclaw, Poland
| |
Collapse
|
35
|
Pundhir A, Sagar S, Singh P, Raman B. Echoes of images: multi-loss network for image retrieval in vision transformers. Med Biol Eng Comput 2024:10.1007/s11517-024-03055-6. [PMID: 38436836 DOI: 10.1007/s11517-024-03055-6] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/06/2023] [Accepted: 02/16/2024] [Indexed: 03/05/2024]
Abstract
This paper introduces a novel approach to enhance content-based image retrieval, validated on two benchmark datasets: ISIC-2017 and ISIC-2018. These datasets comprise skin lesion images that are crucial for innovations in skin cancer diagnosis and treatment. We advocate the use of pre-trained Vision Transformer (ViT), a relatively uncharted concept in the realm of image retrieval, particularly in medical scenarios. In contrast to the traditionally employed Convolutional Neural Networks (CNNs), our findings suggest that ViT offers a more comprehensive understanding of the image context, essential in medical imaging. We further incorporate a weighted multi-loss function, delving into various losses such as triplet loss, distillation loss, contrastive loss, and cross-entropy loss. Our exploration investigates the most resilient combination of these losses to create a robust multi-loss function, thus enhancing the robustness of the learned feature space and ameliorating the precision and recall in the retrieval process. Instead of using all the loss functions, the proposed multi-loss function utilizes the combination of only cross-entropy loss, triplet loss, and distillation loss and gains improvement of 6.52% and 3.45% for mean average precision over ISIC-2017 and ISIC-2018. Another innovation in our methodology is a two-branch network strategy, which concurrently boosts image retrieval and classification. Through our experiments, we underscore the effectiveness and the pitfalls of diverse loss configurations in image retrieval. Furthermore, our approach underlines the advantages of retrieval-based classification through majority voting rather than relying solely on the classification head, leading to enhanced prediction for melanoma - the most lethal type of skin cancer. Our results surpass existing state-of-the-art techniques on the ISIC-2017 and ISIC-2018 datasets by improving mean average precision by 1.01% and 4.36% respectively, emphasizing the efficacy and promise of Vision Transformers paired with our tailor-made weighted loss function, especially in medical contexts. The proposed approach's effectiveness is substantiated through thorough ablation studies and an array of quantitative and qualitative outcomes. To promote reproducibility and support forthcoming research, our source code will be accessible on GitHub.
Collapse
Affiliation(s)
- Anshul Pundhir
- Department of Computer Science and Engineering, Indian Institute of Technology, Roorkee, 247667, Uttarakhand, India.
| | - Shivam Sagar
- Department of Electrical Engineering, Indian Institute of Technology, Roorkee, 247667, Uttarakhand, India
| | - Pradeep Singh
- Department of Computer Science and Engineering, Indian Institute of Technology, Roorkee, 247667, Uttarakhand, India
| | - Balasubramanian Raman
- Department of Computer Science and Engineering, Indian Institute of Technology, Roorkee, 247667, Uttarakhand, India
| |
Collapse
|
36
|
Hu F, Chen Z, Wu F. A novel difficult-to-segment samples focusing network for oral CBCT image segmentation. Sci Rep 2024; 14:5068. [PMID: 38429362 PMCID: PMC10907706 DOI: 10.1038/s41598-024-55522-7] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/13/2023] [Accepted: 02/24/2024] [Indexed: 03/03/2024] Open
Abstract
Using deep learning technology to segment oral CBCT images for clinical diagnosis and treatment is one of the important research directions in the field of clinical dentistry. However, the blurred contour and the scale difference limit the segmentation accuracy of the crown edge and the root part of the current methods, making these regions become difficult-to-segment samples in the oral CBCT segmentation task. Aiming at the above problems, this work proposed a Difficult-to-Segment Focus Network (DSFNet) for segmenting oral CBCT images. The network utilizes a Feature Capturing Module (FCM) to efficiently capture local and long-range features, enhancing the feature extraction performance. Additionally, a Multi-Scale Feature Fusion Module (MFFM) is employed to merge multiscale feature information. To further improve the loss ratio for difficult-to-segment samples, a hybrid loss function is proposed, combining Focal Loss and Dice Loss. By utilizing the hybrid loss function, DSFNet achieves 91.85% Dice Similarity Coefficient (DSC) and 0.216 mm Average Symmetric Surface Distance (ASSD) performance in oral CBCT segmentation tasks. Experimental results show that the proposed method is superior to current dental CBCT image segmentation techniques and has real-world applicability.
Collapse
Affiliation(s)
- Fengjun Hu
- College of Information Science and Technology, Zhejiang Shuren University, Hangzhou, 310015, China
- Zhejiang-Netherlands Joint Laboratory for Digital Diagnosis and Treatment of Oral Diseases, Zhejiang Shuren University, Hangzhou, 310015, China
| | - Zeyu Chen
- Zhejiang-Netherlands Joint Laboratory for Digital Diagnosis and Treatment of Oral Diseases, Zhejiang Shuren University, Hangzhou, 310015, China
| | - Fan Wu
- College of Information Science and Technology, Zhejiang Shuren University, Hangzhou, 310015, China.
- Zhejiang-Netherlands Joint Laboratory for Digital Diagnosis and Treatment of Oral Diseases, Zhejiang Shuren University, Hangzhou, 310015, China.
| |
Collapse
|
37
|
Wu H, Liu X, Fang Y, Yang Y, Huang Y, Pan X, Shen HB. Decoding protein binding landscape on circular RNAs with base-resolution transformer models. Comput Biol Med 2024; 171:108175. [PMID: 38402841 DOI: 10.1016/j.compbiomed.2024.108175] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/02/2023] [Revised: 01/16/2024] [Accepted: 02/18/2024] [Indexed: 02/27/2024]
Abstract
Circular RNAs (circRNAs), a class of endogenous RNA with a covalent loop structure, can regulate gene expression by serving as sponges for microRNAs and RNA-binding proteins (RBPs). To date, most computational methods for predicting RBP binding sites on circRNAs focus on circRNA fragments instead of circRNAs. These methods detect whether a circRNA fragment contains binding sites, but cannot determine where are the binding sites and how many binding sites are on the circRNA transcript. We report a hybrid deep learning-based tool, CircSite, to predict RBP binding sites at single-nucleotide resolution and detect key contributed nucleotides on circRNA transcripts. CircSite takes advantage of convolutional neural networks (CNNs) and Transformer for learning local and global representations of circRNAs binding to RBPs, respectively. We construct 37 datasets of circRNAs interacting with proteins for benchmarking and the experimental results show that CircSite offers accurate predictions of RBP binding nucleotides and detects key subsequences aligning well with known binding motifs. CircSite is an easy-to-use online webserver for predicting RBP binding sites on circRNA transcripts and freely available at http://www.csbio.sjtu.edu.cn/bioinf/CircSite/.
Collapse
Affiliation(s)
- Hehe Wu
- Institute of Image Processing and Pattern Recognition, Shanghai Jiao Tong University, And Key Laboratory of System Control and Information Processing, Ministry of Education of China, Shanghai 200240, China
| | - Xiaojian Liu
- Institute of Image Processing and Pattern Recognition, Shanghai Jiao Tong University, And Key Laboratory of System Control and Information Processing, Ministry of Education of China, Shanghai 200240, China
| | - Yi Fang
- Institute of Image Processing and Pattern Recognition, Shanghai Jiao Tong University, And Key Laboratory of System Control and Information Processing, Ministry of Education of China, Shanghai 200240, China
| | - Yang Yang
- Center for Brain-Like Computing and Machine Intelligence, Department of Computer Science and Engineering, Shanghai Jiao Tong University, Shanghai 200240, China
| | - Yan Huang
- State Key Laboratory of Infrared Physics, Shanghai Institute of Technical Physics Chinese Academy of Sciences, 500 Yutian Road, Shanghai, 200083, China
| | - Xiaoyong Pan
- Institute of Image Processing and Pattern Recognition, Shanghai Jiao Tong University, And Key Laboratory of System Control and Information Processing, Ministry of Education of China, Shanghai 200240, China.
| | - Hong-Bin Shen
- Institute of Image Processing and Pattern Recognition, Shanghai Jiao Tong University, And Key Laboratory of System Control and Information Processing, Ministry of Education of China, Shanghai 200240, China.
| |
Collapse
|
38
|
Ilyas T, Ahmad K, Arsa DMS, Jeong YC, Kim H. Enhancing medical image analysis with unsupervised domain adaptation approach across microscopes and magnifications. Comput Biol Med 2024; 170:108055. [PMID: 38295480 DOI: 10.1016/j.compbiomed.2024.108055] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/06/2023] [Revised: 01/05/2024] [Accepted: 01/26/2024] [Indexed: 02/02/2024]
Abstract
In the domain of medical image analysis, deep learning models are heralding a revolution, especially in detecting complex and nuanced features characteristic of diseases like tumors and cancers. However, the robustness and adaptability of these models across varied imaging conditions and magnifications remain a formidable challenge. This paper introduces the Fourier Adaptive Recognition System (FARS), a pioneering model primarily engineered to address adaptability in malarial parasite recognition. Yet, the foundational principles guiding FARS lend themselves seamlessly to broader applications, including tumor and cancer diagnostics. FARS capitalizes on the untapped potential of transitioning from bounding box labels to richer semantic segmentation labels, enabling a more refined examination of microscopy slides. With the integration of adversarial training and the Color Domain Aware Fourier Domain Adaptation (F2DA), the model ensures consistent feature extraction across diverse microscopy configurations. The further inclusion of category-dependent context attention amplifies FARS's cross-domain versatility. Evidenced by a substantial elevation in cross-magnification performance from 31.3% mAP to 55.19% mAP and a 15.68% boost in cross-domain adaptability, FARS positions itself as a significant advancement in malarial parasite recognition. Furthermore, the core methodologies of FARS can serve as a blueprint for enhancing precision in other realms of medical image analysis, especially in the complex terrains of tumor and cancer imaging. The code is available at; https://github.com/Mr-TalhaIlyas/FARS.
Collapse
Affiliation(s)
- Talha Ilyas
- Division of Electronics and Information Engineering, Jeonbuk National University, Jeonju, 54896, Republic of Korea; Core Research Institute of Intelligent Robots, Jeonbuk National University, Jeonju, 54896, Republic of Korea.
| | - Khubaib Ahmad
- Division of Electronics and Information Engineering, Jeonbuk National University, Jeonju, 54896, Republic of Korea; Core Research Institute of Intelligent Robots, Jeonbuk National University, Jeonju, 54896, Republic of Korea
| | - Dewa Made Sri Arsa
- Division of Electronics and Information Engineering, Jeonbuk National University, Jeonju, 54896, Republic of Korea; Core Research Institute of Intelligent Robots, Jeonbuk National University, Jeonju, 54896, Republic of Korea; Department of Information Technology, Universitas Udayana, Bali, 80361, Indonesia
| | - Yong Chae Jeong
- Core Research Institute of Intelligent Robots, Jeonbuk National University, Jeonju, 54896, Republic of Korea; Division of Electronics Engineering, Jeonbuk National University, Jeonju, 54896, Republic of Korea
| | - Hyongsuk Kim
- Division of Electronics and Information Engineering, Jeonbuk National University, Jeonju, 54896, Republic of Korea; Core Research Institute of Intelligent Robots, Jeonbuk National University, Jeonju, 54896, Republic of Korea.
| |
Collapse
|
39
|
Ao Y, Shi W, Ji B, Miao Y, He W, Jiang Z. MS-TCNet: An effective Transformer-CNN combined network using multi-scale feature learning for 3D medical image segmentation. Comput Biol Med 2024; 170:108057. [PMID: 38301516 DOI: 10.1016/j.compbiomed.2024.108057] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/29/2023] [Revised: 12/31/2023] [Accepted: 01/26/2024] [Indexed: 02/03/2024]
Abstract
Medical image segmentation is a fundamental research problem in the field of medical image processing. Recently, the Transformer have achieved highly competitive performance in computer vision. Therefore, many methods combining Transformer with convolutional neural networks (CNNs) have emerged for segmenting medical images. However, these methods cannot effectively capture the multi-scale features in medical images, even though texture and contextual information embedded in the multi-scale features are extremely beneficial for segmentation. To alleviate this limitation, we propose a novel Transformer-CNN combined network using multi-scale feature learning for three-dimensional (3D) medical image segmentation, which is called MS-TCNet. The proposed model utilizes a shunted Transformer and CNN to construct an encoder and pyramid decoder, allowing six different scale levels of feature learning. It captures multi-scale features with refinement at each scale level. Additionally, we propose a novel lightweight multi-scale feature fusion (MSFF) module that can fully fuse the different-scale semantic features generated by the pyramid decoder for each segmentation class, resulting in a more accurate segmentation output. We conducted experiments on three widely used 3D medical image segmentation datasets. The experimental results indicated that our method outperformed state-of-the-art medical image segmentation methods, suggesting its effectiveness, robustness, and superiority. Meanwhile, our model has a smaller number of parameters and lower computational complexity than conventional 3D segmentation networks. The results confirmed that the model is capable of effective multi-scale feature learning and that the learned multi-scale features are useful for improving segmentation performance. We open-sourced our code, which can be found at https://github.com/AustinYuAo/MS-TCNet.
Collapse
Affiliation(s)
- Yu Ao
- School of Computer Science and Technology, Changchun University of Science and Technology, Changchun, 130022, China
| | - Weili Shi
- School of Computer Science and Technology, Changchun University of Science and Technology, Changchun, 130022, China; Zhongshan Institute of Changchun University of Science and Technology, Zhongshan, 528437, China
| | - Bai Ji
- Department of Hepatobiliary and Pancreatic Surgery, The First Hospital of Jilin University, Changchun, 130061, China
| | - Yu Miao
- School of Computer Science and Technology, Changchun University of Science and Technology, Changchun, 130022, China; Zhongshan Institute of Changchun University of Science and Technology, Zhongshan, 528437, China
| | - Wei He
- School of Computer Science and Technology, Changchun University of Science and Technology, Changchun, 130022, China; Zhongshan Institute of Changchun University of Science and Technology, Zhongshan, 528437, China
| | - Zhengang Jiang
- School of Computer Science and Technology, Changchun University of Science and Technology, Changchun, 130022, China; Zhongshan Institute of Changchun University of Science and Technology, Zhongshan, 528437, China.
| |
Collapse
|
40
|
Bennis FC, Aussems C, Korevaar JC, Hoogendoorn M. The added value of temporal data and the best way to handle it: A use-case for atrial fibrillation using general practitioner data. Comput Biol Med 2024; 171:108097. [PMID: 38412689 DOI: 10.1016/j.compbiomed.2024.108097] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/25/2023] [Revised: 01/29/2024] [Accepted: 02/01/2024] [Indexed: 02/29/2024]
Abstract
INTRODUCTION Temporal data has numerous challenges for deep learning such as irregularity of sampling. New algorithms are being developed that can handle these temporal challenges better. However, it is unclear how the performance ranges from classical non-temporal models to newly developed algorithms. Therefore, this study compares different non-temporal and temporal algorithms for a relevant use case, the prediction of atrial fibrillation (AF) using general practitioner (GP) data. METHODS Three datasets with a 365-day observation window and prediction windows of 14, 180 and 360 days were used. Data consisted of medication, lab, symptom, and chronic diseases codings registered by the GP. The benchmark discarded temporality and used logistic regression, XGBoost models and neural networks on the presence of codings over the whole year. Pattern data extracted common patterns of GP codings and tested using the same algorithms. LSTM and CKConv models were trained as models incorporating temporality. RESULTS Algorithms which incorporated temporality (LSTM and CKConv, (max AUC 0.734 at 360 days prediction window) outperformed both benchmark and pattern algorithms (max AUC 0.723, with a significant improvement using the 360 days prediction window (p = 0.04). The difference between the benchmark and the LSTM or CKConv algorithm decreased with smaller prediction windows, indicating temporal importance for longer prediction windows. The CKConv and LSTM algorithm performed similarly, possibly due to limited sequence length. CONCLUSION Temporal models outperformed non-temporal models for the prediction of AF. For temporal models, CKConv is a promising algorithm to handle temporal data using GP data as it can handle irregular data.
Collapse
Affiliation(s)
- Frank C Bennis
- Quantitative Data Analytics Group, Department of Computer Science, VU Amsterdam, Amsterdam, the Netherlands; Nivel, Netherlands Institute for Health Services Research, Utrecht, the Netherlands.
| | - Claire Aussems
- Nivel, Netherlands Institute for Health Services Research, Utrecht, the Netherlands
| | - Joke C Korevaar
- Nivel, Netherlands Institute for Health Services Research, Utrecht, the Netherlands
| | - Mark Hoogendoorn
- Quantitative Data Analytics Group, Department of Computer Science, VU Amsterdam, Amsterdam, the Netherlands
| |
Collapse
|
41
|
Gravina M, García-Pedrero A, Gonzalo-Martín C, Sansone C, Soda P. Multi input-Multi output 3D CNN for dementia severity assessment with incomplete multimodal data. Artif Intell Med 2024; 149:102774. [PMID: 38462278 DOI: 10.1016/j.artmed.2024.102774] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/05/2022] [Revised: 12/08/2023] [Accepted: 01/14/2024] [Indexed: 03/12/2024]
Abstract
Alzheimer's Disease is the most common cause of dementia, whose progression spans in different stages, from very mild cognitive impairment to mild and severe conditions. In clinical trials, Magnetic Resonance Imaging (MRI) and Positron Emission Tomography (PET) are mostly used for the early diagnosis of neurodegenerative disorders since they provide volumetric and metabolic function information of the brain, respectively. In recent years, Deep Learning (DL) has been employed in medical imaging with promising results. Moreover, the use of the deep neural networks, especially Convolutional Neural Networks (CNNs), has also enabled the development of DL-based solutions in domains characterized by the need of leveraging information coming from multiple data sources, raising the Multimodal Deep Learning (MDL). In this paper, we conduct a systematic analysis of MDL approaches for dementia severity assessment exploiting MRI and PET scans. We propose a Multi Input-Multi Output 3D CNN whose training iterations change according to the characteristic of the input as it is able to handle incomplete acquisitions, in which one image modality is missed. Experiments performed on OASIS-3 dataset show the satisfactory results of the implemented network, which outperforms approaches exploiting both single image modality and different MDL fusion techniques.
Collapse
Affiliation(s)
- Michela Gravina
- Department of Electrical Engineering and Information Technology, University of Naples Federico II, Napoli, 80125, Italy
| | - Angel García-Pedrero
- Department of Computer Architecture and Technology, Universidad Politécnica de Madrid, Boadilla del Monte, 28660, Madrid, Spain; Center for Biomedical Technology, Campus de Montegancedo, Universidad Politécnica de Madrid, Pozuelo de Alarcón, 28233, Madrid, Spain
| | - Consuelo Gonzalo-Martín
- Department of Computer Architecture and Technology, Universidad Politécnica de Madrid, Boadilla del Monte, 28660, Madrid, Spain; Center for Biomedical Technology, Campus de Montegancedo, Universidad Politécnica de Madrid, Pozuelo de Alarcón, 28233, Madrid, Spain.
| | - Carlo Sansone
- Department of Electrical Engineering and Information Technology, University of Naples Federico II, Napoli, 80125, Italy
| | - Paolo Soda
- Department of Engineering, Unit of Computer Systems and Bioinformatics, University of Rome Campus Bio-Medico, Roma, 00128, Italy; Department of Diagnostics and Intervention, Radiation Physics, Biomedical Engineering, Umeå University, 90187, Umeå, Sweden
| |
Collapse
|
42
|
Mozaffari J, Amirkhani A, Shokouhi SB. ColonGen: an efficient polyp segmentation system for generalization improvement using a new comprehensive dataset. Phys Eng Sci Med 2024; 47:309-325. [PMID: 38224384 DOI: 10.1007/s13246-023-01368-8] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/09/2023] [Accepted: 12/06/2023] [Indexed: 01/16/2024]
Abstract
Colorectal cancer (CRC) is one of the most common causes of cancer-related deaths. While polyp detection is important for diagnosing CRC, high miss rates for polyps have been reported during colonoscopy. Most deep learning methods extract features from images using convolutional neural networks (CNNs). In recent years, vision transformer (ViT) models have been employed for image processing and have been successful in image segmentation. It is possible to improve image processing by using transformer models that can extract spatial location information, and CNNs that are capable of aggregating local information. Despite this, recent research shows limited effectiveness in increasing data diversity and generalization accuracy. This paper investigates the generalization proficiency of polyp image segmentation based on transformer architecture and proposes a novel approach using two different ViT architectures. This allows the model to learn representations from different perspectives, which can then be combined to create a richer feature representation. Additionally, a more universal and comprehensive dataset has been derived from the datasets presented in the related research, which can be used for improving generalizations. We first evaluated the generalization of our proposed model using three distinct training-testing scenarios. Our experimental results demonstrate that our ColonGen-V1 outperforms other state-of-the-art methods in all scenarios. As a next step, we used the comprehensive dataset for improving the performance of the model against in- and out-of-domain data. The results show that our ColonGen-V2 outperforms state-of-the-art studies by 5.1%, 1.3%, and 1.1% in ETIS-Larib, Kvasir-Seg, and CVC-ColonDB datasets, respectively. The inclusive dataset and the model introduced in this paper are available to the public through this link: https://github.com/javadmozaffari/Polyp_segmentation .
Collapse
Affiliation(s)
- Javad Mozaffari
- School of Electrical Engineering, Iran University of Science and Technology, Tehran, 16846-13114, Iran
| | - Abdollah Amirkhani
- School of Automotive Engineering, Iran University of Science and Technology, Tehran, 16846-13114, Iran.
| | - Shahriar B Shokouhi
- School of Electrical Engineering, Iran University of Science and Technology, Tehran, 16846-13114, Iran
| |
Collapse
|
43
|
Hashimoto F, Onishi Y, Ote K, Tashima H, Reader AJ, Yamaya T. Deep learning-based PET image denoising and reconstruction: a review. Radiol Phys Technol 2024; 17:24-46. [PMID: 38319563 PMCID: PMC10902118 DOI: 10.1007/s12194-024-00780-3] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/29/2023] [Revised: 01/03/2024] [Accepted: 01/04/2024] [Indexed: 02/07/2024]
Abstract
This review focuses on positron emission tomography (PET) imaging algorithms and traces the evolution of PET image reconstruction methods. First, we provide an overview of conventional PET image reconstruction methods from filtered backprojection through to recent iterative PET image reconstruction algorithms, and then review deep learning methods for PET data up to the latest innovations within three main categories. The first category involves post-processing methods for PET image denoising. The second category comprises direct image reconstruction methods that learn mappings from sinograms to the reconstructed images in an end-to-end manner. The third category comprises iterative reconstruction methods that combine conventional iterative image reconstruction with neural-network enhancement. We discuss future perspectives on PET imaging and deep learning technology.
Collapse
Affiliation(s)
- Fumio Hashimoto
- Central Research Laboratory, Hamamatsu Photonics K. K, 5000 Hirakuchi, Hamana-Ku, Hamamatsu, 434-8601, Japan.
- Graduate School of Science and Engineering, Chiba University, 1-33, Yayoicho, Inage-Ku, Chiba, 263-8522, Japan.
- National Institutes for Quantum Science and Technology, 4-9-1, Anagawa, Inage-Ku, Chiba, 263-8555, Japan.
| | - Yuya Onishi
- Central Research Laboratory, Hamamatsu Photonics K. K, 5000 Hirakuchi, Hamana-Ku, Hamamatsu, 434-8601, Japan
| | - Kibo Ote
- Central Research Laboratory, Hamamatsu Photonics K. K, 5000 Hirakuchi, Hamana-Ku, Hamamatsu, 434-8601, Japan
| | - Hideaki Tashima
- National Institutes for Quantum Science and Technology, 4-9-1, Anagawa, Inage-Ku, Chiba, 263-8555, Japan
| | - Andrew J Reader
- School of Biomedical Engineering and Imaging Sciences, King's College London, London, SE1 7EH, UK
| | - Taiga Yamaya
- Graduate School of Science and Engineering, Chiba University, 1-33, Yayoicho, Inage-Ku, Chiba, 263-8522, Japan
- National Institutes for Quantum Science and Technology, 4-9-1, Anagawa, Inage-Ku, Chiba, 263-8555, Japan
| |
Collapse
|
44
|
Pan X, Mu Y, Ma C, He Q. TFCNet: A texture-aware and fine-grained feature compensated polyp detection network. Comput Biol Med 2024; 171:108144. [PMID: 38382386 DOI: 10.1016/j.compbiomed.2024.108144] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/12/2023] [Revised: 01/14/2024] [Accepted: 02/12/2024] [Indexed: 02/23/2024]
Abstract
PURPOSE Abnormal tissue detection is a prerequisite for medical image analysis and computer-aided diagnosis and treatment. The use of neural networks (CNN) to achieve accurate detection of intestinal polyps is beneficial to the early diagnosis and treatment of colorectal cancer. Currently, image detection models using multi-scale feature processing perform well in polyp detection. However, these methods do not fully consider the misalignment of information in the process of feature scale change, resulting in the loss of fine-grained features, and eventually cause the missed and false detection of targets. METHOD To solve this problem, a texture-aware and fine-grained feature compensated polyp detection network (TFCNet) is proposed in this paper. Firstly, design Texture Awareness Module (TAM) to excavate the rich texture information from the low-level layers and utilize high-level semantic information for background suppression, thereby capturing purer fine-grained features. Secondly, the Texture Feature Enhancement Module (TFEM) is designed to enhance the low-level texture information in TAM, and the enhanced texture features were fused with the high-level features. By making full use of the low-level texture features and multi-scale context information, the semantic consistency and integrity of the features were ensured. Finally, the Residual Pyramid Splittable Attention Module (RPSA) is designed to balance the loss of channel information caused by skip connections, and further improve the detection performance of the network. RESULTS Experimental results on 4 datasets demonstrate that the TFCNet network outperforms existing methods. Particularly, on the large dataset PolypSets, the mAP@0.5-0.95 has been improved to 88.9%. On the small datasets CVC-ClinicDB and Kvasir, the mAP@0.5-0.95 is increased by 2% and 1.6%, respectively, compared to the baseline, showcasing a significant superiority over competing methods.
Collapse
Affiliation(s)
- Xiaoying Pan
- Shanxi Key Laboratory of Network Data Analysis and Intelligent Processing, Xi'an, 710121, China; School of Computer Science & Technology, Xi'an University of Post & Telecommunications, Xi'an, 710121, China.
| | - Yaya Mu
- Shanxi Key Laboratory of Network Data Analysis and Intelligent Processing, Xi'an, 710121, China; School of Computer Science & Technology, Xi'an University of Post & Telecommunications, Xi'an, 710121, China
| | - Chenyang Ma
- Shanxi Key Laboratory of Network Data Analysis and Intelligent Processing, Xi'an, 710121, China; School of Computer Science & Technology, Xi'an University of Post & Telecommunications, Xi'an, 710121, China
| | - Qiqi He
- Shanxi Key Laboratory of Network Data Analysis and Intelligent Processing, Xi'an, 710121, China; School of Computer Science & Technology, Xi'an University of Post & Telecommunications, Xi'an, 710121, China
| |
Collapse
|
45
|
Sugino T, Kin T, Saito N, Nakajima Y. Improved segmentation of basal ganglia from MR images using convolutional neural network with crossover-typed skip connection. Int J Comput Assist Radiol Surg 2024; 19:433-442. [PMID: 37982960 DOI: 10.1007/s11548-023-03015-9] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/10/2023] [Accepted: 08/29/2023] [Indexed: 11/21/2023]
Abstract
PURPOSE Accurate and automatic segmentation of basal ganglia from magnetic resonance (MR) images is important for diagnosis and treatment of various brain disorders. However, the basal ganglia segmentation is a challenging task because of the class imbalance and the unclear boundaries among basal ganglia anatomical structures. Thus, we aim to present an encoder-decoder convolutional neural network (CNN)-based method for improved segmentation of basal ganglia by focusing on skip connections that determine the segmentation performance of encoder-decoder CNNs. We also aim to reveal the effect of skip connections on the segmentation of basal ganglia with unclear boundaries. METHODS We used the encoder-decoder CNNs with the following five patterns of skip connections: without skip connection, with full-resolution horizontal skip connection, with horizontal skip connections, with vertical skip connections, and with crossover-typed skip connections (the proposed method). We compared and evaluated the performance of the CNNs in the experiment of basal ganglia segmentation using T1-weighted MR brain images of 79 patients. RESULTS The experimental results showed that the skip connections at each scale level help CNNs to acquire multi-scale image features, the vertical skip connections contribute on acquiring finer image features for segmentation of smaller anatomical structures with more blurred boundaries, and the crossover-typed skip connections, a combination of horizontal and vertical skip connections, provided better segmentation accuracy. CONCLUSION This paper investigated the effect of skip connections on the basal ganglia segmentation and revealed the crossover-typed skip connections might be effective for improving the segmentation of basal ganglia with the class imbalance and the unclear boundaries.
Collapse
Affiliation(s)
- Takaaki Sugino
- Department of Biomedical Informatics, Institute of Biomaterials and Bioengineering, Tokyo Medical and Dental University, Tokyo, Japan.
| | - Taichi Kin
- Department of Neurosurgery, Graduate School of Medicine, The University of Tokyo, Tokyo, Japan
| | - Nobuhito Saito
- Department of Neurosurgery, Graduate School of Medicine, The University of Tokyo, Tokyo, Japan
| | - Yoshikazu Nakajima
- Department of Biomedical Informatics, Institute of Biomaterials and Bioengineering, Tokyo Medical and Dental University, Tokyo, Japan
| |
Collapse
|
46
|
Molina-Moreno M, González-Díaz I, Rivera Gorrín M, Burguera Vion V, Díaz-de-María F. URI-CADS: A Fully Automated Computer-Aided Diagnosis System for Ultrasound Renal Imaging. J Imaging Inform Med 2024:10.1007/s10278-024-01055-4. [PMID: 38413459 DOI: 10.1007/s10278-024-01055-4] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 12/26/2023] [Revised: 02/09/2024] [Accepted: 02/14/2024] [Indexed: 02/29/2024]
Abstract
Ultrasound is a widespread imaging modality, with special application in medical fields such as nephrology. However, automated approaches for ultrasound renal interpretation still pose some challenges: (1) the need for manual supervision by experts at various stages of the system, which prevents its adoption in primary healthcare, and (2) their limited considered taxonomy (e.g., reduced number of pathologies), which makes them unsuitable for training practitioners and providing support to experts. This paper proposes a fully automated computer-aided diagnosis system for ultrasound renal imaging addressing both of these challenges. Our system is based in a multi-task architecture, which is implemented by a three-branched convolutional neural network and is capable of segmenting the kidney and detecting global and local pathologies with no need of human interaction during diagnosis. The integration of different image perspectives at distinct granularities enhanced the proposed diagnosis. We employ a large (1985 images) and demanding ultrasound renal imaging database, publicly released with the system and annotated on the basis of an exhaustive taxonomy of two global and nine local pathologies (including cysts, lithiasis, hydronephrosis, angiomyolipoma), establishing a benchmark for ultrasound renal interpretation. Experiments show that our proposed method outperforms several state-of-the-art methods in both segmentation and diagnosis tasks and leverages the combination of global and local image information to improve the diagnosis. Our results, with a 87.41% of AUC in healthy-pathological diagnosis and 81.90% in multi-pathological diagnosis, support the use of our system as a helpful tool in the healthcare system.
Collapse
Affiliation(s)
- Miguel Molina-Moreno
- Department of Signal Theory and Communications, Universidad Carlos III de Madrid, Avda. de la Universidad, 30, Leganés, 28911, Spain.
| | - Iván González-Díaz
- Department of Signal Theory and Communications, Universidad Carlos III de Madrid, Avda. de la Universidad, 30, Leganés, 28911, Spain
| | - Maite Rivera Gorrín
- Hospital Ramón y Cajal, M-607, 9, 100, Madrid, 28034, Spain
- Instituto Ramón y Cajal de Investigación Sanitaria (IRyCis), Ctra. Colmenar Viejo, Madrid, 28034, Spain
- Universidad de Alcalá, Pl. de San Diego, s/n, Alcalá de Henares, 28801, Spain
| | | | - Fernando Díaz-de-María
- Department of Signal Theory and Communications, Universidad Carlos III de Madrid, Avda. de la Universidad, 30, Leganés, 28911, Spain
| |
Collapse
|
47
|
Shafik W, Tufail A, De Silva Liyanage C, Apong RAAHM. Using transfer learning-based plant disease classification and detection for sustainable agriculture. BMC Plant Biol 2024; 24:136. [PMID: 38408925 PMCID: PMC10895770 DOI: 10.1186/s12870-024-04825-y] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 10/09/2023] [Accepted: 02/15/2024] [Indexed: 02/28/2024]
Abstract
Subsistence farmers and global food security depend on sufficient food production, which aligns with the UN's "Zero Hunger," "Climate Action," and "Responsible Consumption and Production" sustainable development goals. In addition to already available methods for early disease detection and classification facing overfitting and fine feature extraction complexities during the training process, how early signs of green attacks can be identified or classified remains uncertain. Most pests and disease symptoms are seen in plant leaves and fruits, yet their diagnosis by experts in the laboratory is expensive, tedious, labor-intensive, and time-consuming. Notably, how plant pests and diseases can be appropriately detected and timely prevented is a hotspot paradigm in smart, sustainable agriculture remains unknown. In recent years, deep transfer learning has demonstrated tremendous advances in the recognition accuracy of object detection and image classification systems since these frameworks utilize previously acquired knowledge to solve similar problems more effectively and quickly. Therefore, in this research, we introduce two plant disease detection (PDDNet) models of early fusion (AE) and the lead voting ensemble (LVE) integrated with nine pre-trained convolutional neural networks (CNNs) and fine-tuned by deep feature extraction for efficient plant disease identification and classification. The experiments were carried out on 15 classes of the popular PlantVillage dataset, which has 54,305 image samples of different plant disease species in 38 categories. Hyperparameter fine-tuning was done with popular pre-trained models, including DenseNet201, ResNet101, ResNet50, GoogleNet, AlexNet, ResNet18, EfficientNetB7, NASNetMobile, and ConvNeXtSmall. We test these CNNs on the stated plant disease detection and classification problem, both independently and as part of an ensemble. In the final phase, a logistic regression (LR) classifier is utilized to determine the performance of various CNN model combinations. A comparative analysis was also performed on classifiers, deep learning, the proposed model, and similar state-of-the-art studies. The experiments demonstrated that PDDNet-AE and PDDNet-LVE achieved 96.74% and 97.79%, respectively, compared to current CNNs when tested on several plant diseases, depicting its exceptional robustness and generalization capabilities and mitigating current concerns in plant disease detection and classification.
Collapse
Affiliation(s)
- Wasswa Shafik
- School of Digital Science, Universiti Brunei Darussalam, Tungku Link, Gadong, BE1410, Brunei
| | - Ali Tufail
- School of Digital Science, Universiti Brunei Darussalam, Tungku Link, Gadong, BE1410, Brunei.
| | | | | |
Collapse
|
48
|
De A, Mishra N, Chang HT. An approach to the dermatological classification of histopathological skin images using a hybridized CNN-DenseNet model. PeerJ Comput Sci 2024; 10:e1884. [PMID: 38435616 PMCID: PMC10909212 DOI: 10.7717/peerj-cs.1884] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/26/2023] [Accepted: 01/29/2024] [Indexed: 03/05/2024]
Abstract
This research addresses the challenge of automating skin disease diagnosis using dermatoscopic images. The primary issue lies in accurately classifying pigmented skin lesions, which traditionally rely on manual assessment by dermatologists and are prone to subjectivity and time consumption. By integrating a hybrid CNN-DenseNet model, this study aimed to overcome the complexities of differentiating various skin diseases and automating the diagnostic process effectively. Our methodology involved rigorous data preprocessing, exploratory data analysis, normalization, and label encoding. Techniques such as model hybridization, batch normalization and data fitting were employed to optimize the model architecture and data fitting. Initial iterations of our convolutional neural network (CNN) model achieved an accuracy of 76.22% on the test data and 75.69% on the validation data. Recognizing the need for improvement, the model was hybridized with DenseNet architecture and ResNet architecture was implemented for feature extraction and then further trained on the HAM10000 and PAD-UFES-20 datasets. Overall, our efforts resulted in a hybrid model that demonstrated an impressive accuracy of 95.7% on the HAM10000 dataset and 91.07% on the PAD-UFES-20 dataset. In comparison to recently published works, our model stands out because of its potential to effectively diagnose skin diseases such as melanocytic nevi, melanoma, benign keratosis-like lesions, basal cell carcinoma, actinic keratoses, vascular lesions, and dermatofibroma, all of which rival the diagnostic accuracy of real-world clinical specialists but also offer customization potential for more nuanced clinical uses.
Collapse
Affiliation(s)
- Anubhav De
- School of Computing Science & Engineering, VIT Bhopal University, Madhya Pradesh, India
| | - Nilamadhab Mishra
- School of Computing Science & Engineering, VIT Bhopal University, Madhya Pradesh, India
| | - Hsien-Tsung Chang
- Department of Computer Science and Information Engineering, Chang Gung University, Taoyuan, Taiwan
- Department of Physical Medicine and Rehabilitation, Chang Gung Memorial Hospital, Taoyuan, Taiwan
- Artificial Intelligence Research Center, Chang Gung University, Taoyuan, Taiwan
- Bachelor Program in Artificial Intelligence, Chang Gung University, Taoyuan, Taiwan
| |
Collapse
|
49
|
Perkgoz C. Identifying optical microscope images of CVD-grown two-dimensional MoS 2 by convolutional neural networks and transfer learning. PeerJ Comput Sci 2024; 10:e1885. [PMID: 38435565 PMCID: PMC10909165 DOI: 10.7717/peerj-cs.1885] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/08/2023] [Accepted: 01/29/2024] [Indexed: 03/05/2024]
Abstract
Background In Complementary Metal-Oxide Semiconductor (CMOS) technology, scaling down has been a key strategy to improve chip performance and reduce power losses. However, challenges such as sub-threshold leakage and gate leakage, resulting from short-channel effects, contribute to an increase in distributed static power. Two-dimensional transition metal dichalcogenides (2D TMDs) emerge as potential solutions, serving as channel materials with steep sub-threshold swings and lower power consumption. However, the production and development of these 2-dimensional materials require some time-consuming tasks. In order to employ them in different fields, including chip technology, it is crucial to ensure that their production meets the required standards of quality and uniformity; in this context, deep learning techniques show significant potential. Methods This research introduces a transfer learning-based deep convolutional neural network (CNN) to classify chemical vapor deposition (CVD) grown molybdenum disulfide (MoS2) flakes based on their uniformity or the occurrence of defects affecting electronic properties. Acquiring and labeling a sufficient number of microscope images for CNN training may not be realistic. To address this challenge, artificial images were generated using Fresnel equations to pre-train the CNN. Subsequently, accuracy was improved through fine-tuning with a limited set of real images. Results The proposed transfer learning-based CNN method significantly improved all measurement metrics with respect to the ordinary CNNs. The initial CNN, trained with limited data and without transfer learning, achieved 68% average accuracy for binary classification. Through transfer learning and artificial images, the same CNN achieved 85% average accuracy, demonstrating an average increase of approximately 17%. While this study specifically focuses on MoS2 structures, the same methodology can be extended to other 2-dimensional materials by simply incorporating their specific parameters when generating artificial images.
Collapse
Affiliation(s)
- Cahit Perkgoz
- Department of Computer Engineering, Eskisehir Technical University, Eskişehir, Turkey
| |
Collapse
|
50
|
Lee JH, Seok J, Kim JY, Kim HC, Kwon TK. Evaluating the Diagnostic Potential of Connected Speech for Benign Laryngeal Disease Using Deep Learning Analysis. J Voice 2024:S0892-1997(24)00018-3. [PMID: 38350806 DOI: 10.1016/j.jvoice.2024.01.015] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/30/2023] [Revised: 01/26/2024] [Accepted: 01/26/2024] [Indexed: 02/15/2024]
Abstract
OBJECTIVES This study aimed to evaluate the performance of artificial intelligence (AI) models using connected speech and vowel sounds in detecting benign laryngeal diseases. STUDY DESIGN Retrospective. METHODS Voice samples from 772 patients, including 502 with normal voices and 270 with vocal cord polyps, cysts, or nodules, were analyzed. We employed deep learning architectures, including convolutional neural networks (CNNs) and time series models, to process the speech data. The primary endpoint was the area under the receiver's operating characteristic curve for binary classification. RESULTS CNN models analyzing speech segments significantly outperformed those using vowel sounds in distinguishing patients with and without benign laryngeal diseases. The best-performing CNN model achieved areas under the receiver operating characteristic curve of 0.895 and 0.845 for speech and vowel sounds, respectively. Correlations between AI-generated disease probabilities and perceptual assessments were more pronounced in the connected-speech analyses. However, the time series models performed worse than the CNNs. CONCLUSION Connected speech analysis is more effective than traditional vowel sound analysis for the diagnosis of laryngeal voice disorders. This study highlights the potential of AI technologies in enhancing the diagnostic capabilities of speech, advocating further exploration, and validation in this field.
Collapse
Affiliation(s)
- Jeong Hoon Lee
- Department of Radiology, Stanford University School of Medicine, Stanford, California
| | - Jungirl Seok
- Department of Otorhinolaryngology-Head and Neck Surgery, Seoul National University College of Medicine, Seoul, Republic of Korea; Department of Otorhinolaryngology-Head and Neck Surgery, Seoul National University Hospital, Seoul, Republic of Korea
| | - Jae Yeong Kim
- Department of Otorhinolaryngology-Head and Neck Surgery, Seoul National University College of Medicine, Seoul, Republic of Korea
| | - Hee Chan Kim
- Department of Biomedical Engineering, Seoul National University College of Medicine, Seoul, Republic of Korea
| | - Tack-Kyun Kwon
- Department of Otorhinolaryngology-Head and Neck Surgery, Seoul National University College of Medicine, Seoul, Republic of Korea; Department of Otorhinolaryngology‑Head and Neck Surgery, Boramae Medical Center, Seoul, Republic of Korea.
| |
Collapse
|