1
|
Wang Y, Guo S, Guo J, Zhang J, Zhang W, Yan C, Zhang Y. Towards performance-maximizing neural network pruning via global channel attention. Neural Netw 2024; 171:104-113. [PMID: 38091754 DOI: 10.1016/j.neunet.2023.11.065] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/27/2023] [Revised: 11/24/2023] [Accepted: 11/29/2023] [Indexed: 01/29/2024]
Abstract
Network pruning has attracted increasing attention recently for its capability of transferring large-scale neural networks (e.g., CNNs) into resource-constrained devices. Such a transfer is typically achieved by removing redundant network parameters while retaining its generalization performance in a static or dynamic manner. Concretely, static pruning usually maintains a larger and fit-to-all (samples) compressed network by removing the same channels for all samples, which cannot maximally excavate redundancy in the given network. In contrast, dynamic pruning can adaptively remove (more) different channels for different samples and obtain state-of-the-art performance along with a higher compression ratio. However, since the system has to preserve the complete network information for sample-specific pruning, the dynamic pruning methods are usually not memory-efficient. In this paper, our interest is to explore a static alternative, dubbed GlobalPru, from a different perspective by respecting the differences among data. Specifically, a novel channel attention-based learn-to-rank framework is proposed to learn a global ranking of channels with respect to network redundancy. In this method, each sample-wise (local) channel attention is forced to reach an agreement on the global ranking among different data. Hence, all samples can empirically share the same ranking of channels and make the pruning statically in practice. Extensive experiments on ImageNet, SVHN, and CIFAR-10/100 demonstrate that the proposed GlobalPru achieves superior performance than state-of-the-art static and dynamic pruning methods by significant margins.
Collapse
Affiliation(s)
- Yingchun Wang
- BDKE Lab, School of Computer Science and Technology, Xi'an Jiaotong University, Xi'an, China; Department of Computing, The Hong Kong Polytechnic University, Hong Kong Special Administrative Region of China.
| | - Song Guo
- Department of Computing, The Hong Kong Polytechnic University, Hong Kong Special Administrative Region of China.
| | - Jingcai Guo
- Department of Computing, The Hong Kong Polytechnic University, Hong Kong Special Administrative Region of China.
| | - Jie Zhang
- Department of Computing, The Hong Kong Polytechnic University, Hong Kong Special Administrative Region of China.
| | - Weizhan Zhang
- BDKE Lab, School of Computer Science and Technology, Xi'an Jiaotong University, Xi'an, China.
| | - Caixia Yan
- BDKE Lab, School of Computer Science and Technology, Xi'an Jiaotong University, Xi'an, China
| | - Yuanhong Zhang
- BDKE Lab, School of Computer Science and Technology, Xi'an Jiaotong University, Xi'an, China
| |
Collapse
|
2
|
Mendes F, Mascarenhas M, Ribeiro T, Afonso J, Cardoso P, Martins M, Cardoso H, Andrade P, Ferreira JPS, Mascarenhas Saraiva M, Macedo G. Artificial Intelligence and Panendoscopy-Automatic Detection of Clinically Relevant Lesions in Multibrand Device-Assisted Enteroscopy. Cancers (Basel) 2024; 16:208. [PMID: 38201634 PMCID: PMC10778030 DOI: 10.3390/cancers16010208] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/05/2023] [Revised: 12/27/2023] [Accepted: 12/28/2023] [Indexed: 01/12/2024] Open
Abstract
Device-assisted enteroscopy (DAE) is capable of evaluating the entire gastrointestinal tract, identifying multiple lesions. Nevertheless, DAE's diagnostic yield is suboptimal. Convolutional neural networks (CNN) are multi-layer architecture artificial intelligence models suitable for image analysis, but there is a lack of studies about their application in DAE. Our group aimed to develop a multidevice CNN for panendoscopic detection of clinically relevant lesions during DAE. In total, 338 exams performed in two specialized centers were retrospectively evaluated, with 152 single-balloon enteroscopies (Fujifilm®, Porto, Portugal), 172 double-balloon enteroscopies (Olympus®, Porto, Portugal) and 14 motorized spiral enteroscopies (Olympus®, Porto, Portugal); then, 40,655 images were divided in a training dataset (90% of the images, n = 36,599) and testing dataset (10% of the images, n = 4066) used to evaluate the model. The CNN's output was compared to an expert consensus classification. The model was evaluated by its sensitivity, specificity, positive (PPV) and negative predictive values (NPV), accuracy and area under the precision recall curve (AUC-PR). The CNN had an 88.9% sensitivity, 98.9% specificity, 95.8% PPV, 97.1% NPV, 96.8% accuracy and an AUC-PR of 0.97. Our group developed the first multidevice CNN for panendoscopic detection of clinically relevant lesions during DAE. The development of accurate deep learning models is of utmost importance for increasing the diagnostic yield of DAE-based panendoscopy.
Collapse
Affiliation(s)
- Francisco Mendes
- Alameda Professor Hernâni Monteiro, Department of Gastroenterology, São João University Hospital, 4200-427 Porto, Portugal; (F.M.); (T.R.); (P.C.); (M.M.); (P.A.); (G.M.)
- WGO Gastroenterology and Hepatology Training Center, 4050-345 Porto, Portugal
| | - Miguel Mascarenhas
- Alameda Professor Hernâni Monteiro, Department of Gastroenterology, São João University Hospital, 4200-427 Porto, Portugal; (F.M.); (T.R.); (P.C.); (M.M.); (P.A.); (G.M.)
- WGO Gastroenterology and Hepatology Training Center, 4050-345 Porto, Portugal
- Faculty of Medicine, University of Porto, Alameda Professor Hernâni Monteiro, 4200-427 Porto, Portugal
| | - Tiago Ribeiro
- Alameda Professor Hernâni Monteiro, Department of Gastroenterology, São João University Hospital, 4200-427 Porto, Portugal; (F.M.); (T.R.); (P.C.); (M.M.); (P.A.); (G.M.)
- WGO Gastroenterology and Hepatology Training Center, 4050-345 Porto, Portugal
- Faculty of Medicine, University of Porto, Alameda Professor Hernâni Monteiro, 4200-427 Porto, Portugal
| | - João Afonso
- Alameda Professor Hernâni Monteiro, Department of Gastroenterology, São João University Hospital, 4200-427 Porto, Portugal; (F.M.); (T.R.); (P.C.); (M.M.); (P.A.); (G.M.)
- WGO Gastroenterology and Hepatology Training Center, 4050-345 Porto, Portugal
- Faculty of Medicine, University of Porto, Alameda Professor Hernâni Monteiro, 4200-427 Porto, Portugal
| | - Pedro Cardoso
- Alameda Professor Hernâni Monteiro, Department of Gastroenterology, São João University Hospital, 4200-427 Porto, Portugal; (F.M.); (T.R.); (P.C.); (M.M.); (P.A.); (G.M.)
- WGO Gastroenterology and Hepatology Training Center, 4050-345 Porto, Portugal
- Faculty of Medicine, University of Porto, Alameda Professor Hernâni Monteiro, 4200-427 Porto, Portugal
| | - Miguel Martins
- Alameda Professor Hernâni Monteiro, Department of Gastroenterology, São João University Hospital, 4200-427 Porto, Portugal; (F.M.); (T.R.); (P.C.); (M.M.); (P.A.); (G.M.)
- WGO Gastroenterology and Hepatology Training Center, 4050-345 Porto, Portugal
| | - Hélder Cardoso
- Alameda Professor Hernâni Monteiro, Department of Gastroenterology, São João University Hospital, 4200-427 Porto, Portugal; (F.M.); (T.R.); (P.C.); (M.M.); (P.A.); (G.M.)
- WGO Gastroenterology and Hepatology Training Center, 4050-345 Porto, Portugal
- Faculty of Medicine, University of Porto, Alameda Professor Hernâni Monteiro, 4200-427 Porto, Portugal
| | - Patrícia Andrade
- Alameda Professor Hernâni Monteiro, Department of Gastroenterology, São João University Hospital, 4200-427 Porto, Portugal; (F.M.); (T.R.); (P.C.); (M.M.); (P.A.); (G.M.)
- WGO Gastroenterology and Hepatology Training Center, 4050-345 Porto, Portugal
- Faculty of Medicine, University of Porto, Alameda Professor Hernâni Monteiro, 4200-427 Porto, Portugal
| | - João P. S. Ferreira
- Department of Mechanical Engineering, Faculty of Engineering, University of Porto, Rua Dr. Roberto Frias, 4200-465 Porto, Portugal;
- DigestAID—Digestive Artificial Intelligence Development, R. Alfredo Allen n°. 455/461, 4200-135 Porto, Portugal
| | | | - Guilherme Macedo
- Alameda Professor Hernâni Monteiro, Department of Gastroenterology, São João University Hospital, 4200-427 Porto, Portugal; (F.M.); (T.R.); (P.C.); (M.M.); (P.A.); (G.M.)
- WGO Gastroenterology and Hepatology Training Center, 4050-345 Porto, Portugal
- Faculty of Medicine, University of Porto, Alameda Professor Hernâni Monteiro, 4200-427 Porto, Portugal
| |
Collapse
|
3
|
Wiest IC, Gilbert S, Kather JN. From research to reality: The role of artificial intelligence applications in HCC care. Clin Liver Dis (Hoboken) 2024; 23:e0136. [PMID: 38567094 PMCID: PMC10986906 DOI: 10.1097/cld.0000000000000136] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 11/24/2023] [Accepted: 01/12/2024] [Indexed: 04/04/2024] Open
Affiliation(s)
- Isabella C. Wiest
- Else Kroener Fresenius Center for Digital Health, Medical Faculty Carl Gustav Carus, TUD Dresden University of Technology, Dresden, Germany
- Department of Medicine II, Medical Faculty Mannheim, Heidelberg University, Mannheim, Germany
| | - Stephen Gilbert
- Else Kroener Fresenius Center for Digital Health, Medical Faculty Carl Gustav Carus, TUD Dresden University of Technology, Dresden, Germany
| | - Jakob N. Kather
- Else Kroener Fresenius Center for Digital Health, Medical Faculty Carl Gustav Carus, TUD Dresden University of Technology, Dresden, Germany
- Medical Oncology, National Center for Tumor Diseases (NCT), University Hospital Heidelberg, Heidelberg, Germany
- Department of Medicine I, University Hospital Dresden, Dresden, Germany
| |
Collapse
|
4
|
Wei ZG, Bu PY, Zhang XD, Liu F, Qian Y, Wu FX. invMap: a sensitive mapping tool for long noisy reads with inversion structural variants. BIOINFORMATICS (OXFORD, ENGLAND) 2023; 39:btad726. [PMID: 38058196 DOI: 10.1093/bioinformatics/btad726] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 09/06/2023] [Revised: 11/02/2023] [Accepted: 12/05/2023] [Indexed: 12/08/2023]
Abstract
MOTIVATION Longer reads produced by PacBio or Oxford Nanopore sequencers could more frequently span the breakpoints of structural variations (SVs) than shorter reads. Therefore, existing long-read mapping methods often generate wrong alignments and variant calls. Compared to deletions and insertions, inversion events are more difficult to be detected since the anchors in inversion regions are nonlinear to those in SV-free regions. To address this issue, this study presents a novel long-read mapping algorithm (named as invMap). RESULTS For each long noisy read, invMap first locates the aligned region with a specifically designed scoring method for chaining, then checks the remaining anchors in the aligned region to discover potential inversions. We benchmark invMap on simulated datasets across different genomes and sequencing coverages, experimental results demonstrate that invMap is more accurate to locate aligned regions and call SVs for inversions than the competing methods. The real human genome sequencing dataset of NA12878 illustrates that invMap can effectively find more candidate variant calls for inversions than the competing methods. AVAILABILITY AND IMPLEMENTATION The invMap software is available at https://github.com/zhang134/invMap.git.
Collapse
Affiliation(s)
- Ze-Gang Wei
- School of Physics and Optoelectronics Technology, Baoji University of Arts and Sciences, Baoji 721016, China
- Division of Biomedical Engineering, Department of Computer Science and Department of Mechanical Engineering, University of Saskatchewan, Saskatoon, SK S7N 5A9, Canada
| | - Peng-Yu Bu
- School of Physics and Optoelectronics Technology, Baoji University of Arts and Sciences, Baoji 721016, China
| | - Xiao-Dan Zhang
- School of Physics and Optoelectronics Technology, Baoji University of Arts and Sciences, Baoji 721016, China
| | - Fei Liu
- School of Physics and Optoelectronics Technology, Baoji University of Arts and Sciences, Baoji 721016, China
| | - Yu Qian
- School of Physics and Optoelectronics Technology, Baoji University of Arts and Sciences, Baoji 721016, China
| | - Fang-Xiang Wu
- Division of Biomedical Engineering, Department of Computer Science and Department of Mechanical Engineering, University of Saskatchewan, Saskatoon, SK S7N 5A9, Canada
| |
Collapse
|