51
|
Fang F, Li J, Yuan Y, Zeng T, Zhang G. Multilevel Edge Features Guided Network for Image Denoising. IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS 2021; 32:3956-3970. [PMID: 32845847 DOI: 10.1109/tnnls.2020.3016321] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/11/2023]
Abstract
Image denoising is a challenging inverse problem due to complex scenes and information loss. Recently, various methods have been considered to solve this problem by building a well-designed convolutional neural network (CNN) or introducing some hand-designed image priors. Different from previous works, we investigate a new framework for image denoising, which integrates edge detection, edge guidance, and image denoising into an end-to-end CNN model. To achieve this goal, we propose a multilevel edge features guided network (MLEFGN). First, we build an edge reconstruction network (Edge-Net) to directly predict clear edges from the noisy image. Then, the Edge-Net is embedded as part of the model to provide edge priors, and a dual-path network is applied to extract the image and edge features, respectively. Finally, we introduce a multilevel edge features guidance mechanism for image denoising. To the best of our knowledge, the Edge-Net is the first CNN model specially designed to reconstruct image edges from the noisy image, which shows good accuracy and robustness on natural images. Extensive experiments clearly illustrate that our MLEFGN achieves favorable performance against other methods and plenty of ablation studies demonstrate the effectiveness of our proposed Edge-Net and MLEFGN. The code is available at https://github.com/MIVRC/MLEFGN-PyTorch.
Collapse
|
52
|
Abstract
High-throughput root phenotyping in the soil became an indispensable quantitative tool for the assessment of effects of climatic factors and molecular perturbation on plant root morphology, development and function. To efficiently analyse a large amount of structurally complex soil-root images advanced methods for automated image segmentation are required. Due to often unavoidable overlap between the intensity of fore- and background regions simple thresholding methods are, generally, not suitable for the segmentation of root regions. Higher-level cognitive models such as convolutional neural networks (CNN) provide capabilities for segmenting roots from heterogeneous and noisy background structures, however, they require a representative set of manually segmented (ground truth) images. Here, we present a GUI-based tool for fully automated quantitative analysis of root images using a pre-trained CNN model, which relies on an extension of the U-Net architecture. The developed CNN framework was designed to efficiently segment root structures of different size, shape and optical contrast using low budget hardware systems. The CNN model was trained on a set of 6465 masks derived from 182 manually segmented near-infrared (NIR) maize root images. Our experimental results show that the proposed approach achieves a Dice coefficient of 0.87 and outperforms existing tools (e.g., SegRoot) with Dice coefficient of 0.67 by application not only to NIR but also to other imaging modalities and plant species such as barley and arabidopsis soil-root images from LED-rhizotron and UV imaging systems, respectively. In summary, the developed software framework enables users to efficiently analyse soil-root images in an automated manner (i.e. without manual interaction with data and/or parameter tuning) providing quantitative plant scientists with a powerful analytical tool.
Collapse
|
53
|
Tian C, Xu Y, Zuo W, Du B, Lin CW, Zhang D. Designing and training of a dual CNN for image denoising. Knowl Based Syst 2021. [DOI: 10.1016/j.knosys.2021.106949] [Citation(s) in RCA: 30] [Impact Index Per Article: 10.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/28/2022]
|
54
|
Zhang Q, Yun KK, Wang H, Yoon SW, Lu F, Won D. Automatic cell counting from stimulated Raman imaging using deep learning. PLoS One 2021; 16:e0254586. [PMID: 34288972 PMCID: PMC8294532 DOI: 10.1371/journal.pone.0254586] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/22/2020] [Accepted: 06/29/2021] [Indexed: 11/28/2022] Open
Abstract
In this paper, we propose an automatic cell counting framework for stimulated Raman scattering (SRS) images, which can assist tumor tissue characteristic analysis, cancer diagnosis, and surgery planning processes. SRS microscopy has promoted tumor diagnosis and surgery by mapping lipids and proteins from fresh specimens and conducting a fast disclose of fundamental diagnostic hallmarks of tumors with a high resolution. However, cell counting from label-free SRS images has been challenging due to the limited contrast of cells and tissue, along with the heterogeneity of tissue morphology and biochemical compositions. To this end, a deep learning-based cell counting scheme is proposed by modifying and applying U-Net, an effective medical image semantic segmentation model that uses a small number of training samples. The distance transform and watershed segmentation algorithms are also implemented to yield the cell instance segmentation and cell counting results. By performing cell counting on SRS images of real human brain tumor specimens, promising cell counting results are obtained with > 98% of area under the curve (AUC) and R = 0.97 in terms of cell counting correlation between SRS and histological images with hematoxylin and eosin (H&E) staining. The proposed cell counting scheme illustrates the possibility and potential of performing cell counting automatically in near real time and encourages the study of applying deep learning techniques in biomedical and pathological image analyses.
Collapse
Affiliation(s)
- Qianqian Zhang
- Department of System Science and Industrial Engineering, State University of New York at Binghamton, Binghamton, NY, United States of America
| | - Kyung Keun Yun
- Department of System Science and Industrial Engineering, State University of New York at Binghamton, Binghamton, NY, United States of America
| | - Hao Wang
- Department of System Science and Industrial Engineering, State University of New York at Binghamton, Binghamton, NY, United States of America
| | - Sang Won Yoon
- Department of System Science and Industrial Engineering, State University of New York at Binghamton, Binghamton, NY, United States of America
| | - Fake Lu
- Department of Biomedical Engineering, State University of New York at Binghamton, Binghamton, NY, United States of America
| | - Daehan Won
- Department of System Science and Industrial Engineering, State University of New York at Binghamton, Binghamton, NY, United States of America
| |
Collapse
|
55
|
Wu H, Du P, Kokate R, Wang JX. A semi-analytical solution and AI-based reconstruction algorithms for magnetic particle tracking. PLoS One 2021; 16:e0254051. [PMID: 34242299 PMCID: PMC8270195 DOI: 10.1371/journal.pone.0254051] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/06/2021] [Accepted: 06/18/2021] [Indexed: 11/18/2022] Open
Abstract
Magnetic particle tracking is a recently developed technology that can measure the translation and rotation of a particle in an opaque environment like a turbidity flow and fluidized-bed flow. The trajectory reconstruction usually relies on numerical optimization or filtering, which involve artificial parameters or thresholds. Existing analytical reconstruction algorithms have certain limitations and usually depend on the gradient of the magnetic field, which is not easy to measure accurately in many applications. This paper discusses a new semi-analytical solution and the related reconstruction algorithm. The new method can be used for an arbitrary sensor arrangement. To reduce the measurement uncertainty in practical applications, deep neural network (DNN)-based models are developed to denoise the reconstructed trajectory. Compared to traditional approaches such as wavelet-based filtering, the DNN-based denoisers are more accurate in the position reconstruction. However, they often over-smooth the velocity signal, and a hybrid method that combines the wavelet and DNN model provides a more accurate velocity reconstruction. All the DNN-based and wavelet methods perform well in the orientation reconstruction.
Collapse
Affiliation(s)
- Huixuan Wu
- Department of Aerospace Engineering, School of Engineering, University of Kansas, Lawrence, Kansas, United States of America
| | - Pan Du
- Department of Aerospace and Mechanical Engineering, University of Notre Dame, Notre Dame, Indiana, United States of America
| | - Rohan Kokate
- Department of Aerospace Engineering, School of Engineering, University of Kansas, Lawrence, Kansas, United States of America
| | - Jian-Xun Wang
- Department of Aerospace and Mechanical Engineering, University of Notre Dame, Notre Dame, Indiana, United States of America
- * E-mail:
| |
Collapse
|
56
|
Related Study Based on Otsu Watershed Algorithm and New Squeeze-and-Excitation Networks for Segmentation and Level Classification of Tea Buds. Neural Process Lett 2021. [DOI: 10.1007/s11063-021-10501-1] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/21/2022]
|
57
|
Shen Z, Xi M, Tang C, Xu M, Lei Z. Double-path parallel convolutional neural network for removing speckle noise in different types of OCT images. APPLIED OPTICS 2021; 60:4345-4355. [PMID: 34143124 DOI: 10.1364/ao.419871] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 01/14/2021] [Accepted: 04/30/2021] [Indexed: 06/12/2023]
Abstract
Speckle noises widely exist in optical coherence tomography (OCT) images. We propose an improved double-path parallel convolutional neural network (called DPNet) to reduce speckles. We increase the network width to replace the network depth to extract deeper information from the original OCT images. In addition, we use dilated convolution and residual learning to increase the learning ability of our DPNet. We use 100 pairs of human retinal OCT images as the training dataset. Then we test the DPNet model for denoising speckles on four different types of OCT images, mainly including human retinal OCT images, skin OCT images, colon crypt OCT images, and quail embryo OCT images. We compare the DPNet model with the adaptive complex diffusion method, the curvelet shrinkage method, the shearlet-based total variation method, and the OCTNet method. We qualitatively and quantitatively evaluate these methods in terms of image smoothness, structural information protection, and edge clarity. Our experimental results prove the performance of the DPNet model, and it allows us to batch and quickly process different types of poor-quality OCT images without any parameter fine-tuning under a time-constrained situation.
Collapse
|
58
|
Attention Guided Low-Light Image Enhancement with a Large Scale Low-Light Simulation Dataset. Int J Comput Vis 2021. [DOI: 10.1007/s11263-021-01466-8] [Citation(s) in RCA: 39] [Impact Index Per Article: 13.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/26/2022]
|
59
|
Anoop B, Kalmady KS, Udathu A, Siddharth V, Girish G, Kothari AR, Rajan J. A cascaded convolutional neural network architecture for despeckling OCT images. Biomed Signal Process Control 2021. [DOI: 10.1016/j.bspc.2021.102463] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/22/2022]
|
60
|
Ilesanmi AE, Idowu OP, Chaumrattanakul U, Makhanov SS. Multiscale hybrid algorithm for pre-processing of ultrasound images. Biomed Signal Process Control 2021. [DOI: 10.1016/j.bspc.2020.102396] [Citation(s) in RCA: 8] [Impact Index Per Article: 2.7] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/26/2022]
|
61
|
Feng T, Wang C, Chen X, Fan H, Zeng K, Li Z. URNet: A U-Net based residual network for image dehazing. Appl Soft Comput 2021. [DOI: 10.1016/j.asoc.2020.106884] [Citation(s) in RCA: 6] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/24/2022]
|
62
|
Sun X, Luo H, Liu G, Chen C, Xu F. Lightweight Image Restoration Network for Strong Noise Removal in Nuclear Radiation Scenes. SENSORS (BASEL, SWITZERLAND) 2021; 21:1810. [PMID: 33807719 PMCID: PMC7961967 DOI: 10.3390/s21051810] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 02/01/2021] [Revised: 03/01/2021] [Accepted: 03/03/2021] [Indexed: 11/16/2022]
Abstract
In order to remove the strong noise with complex shapes and high density in nuclear radiation scenes, a lightweight network composed of a Noise Learning Unit (NLU) and Texture Learning Unit (TLU) was designed. The NLU is bilinearly composed of a Multi-scale Kernel Module (MKM) and a Residual Module (RM), which learn non-local information and high-level features, respectively. Both the MKM and RM have receptive field blocks and attention blocks to enlarge receptive fields and enhance features. The TLU is at the bottom of the NLU and learns textures through an independent loss. The entire network adopts a Mish activation function and asymmetric convolutions to improve the overall performance. Compared with 12 denoising methods on our nuclear radiation dataset, the proposed method has the fewest model parameters, the highest quantitative metrics, and the best perceptual satisfaction, indicating its high denoising efficiency and rich texture retention.
Collapse
Affiliation(s)
- Xin Sun
- School of Information Engineering, Southwest University of Science and Technology, Mianyang 621010, China; (X.S.); (C.C.); (F.X.)
| | - Hongwei Luo
- Shenzhen Launch Digital Technology Co., Ltd., Shenzhen 518000, China;
| | - Guihua Liu
- School of Information Engineering, Southwest University of Science and Technology, Mianyang 621010, China; (X.S.); (C.C.); (F.X.)
| | - Chunmei Chen
- School of Information Engineering, Southwest University of Science and Technology, Mianyang 621010, China; (X.S.); (C.C.); (F.X.)
| | - Feng Xu
- School of Information Engineering, Southwest University of Science and Technology, Mianyang 621010, China; (X.S.); (C.C.); (F.X.)
| |
Collapse
|
63
|
Recent developments in computational color image denoising with PDEs to deep learning: a review. Artif Intell Rev 2021. [DOI: 10.1007/s10462-021-09977-z] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 02/07/2023]
|
64
|
Liu Y, Zhu YH, Song X, Song J, Yu DJ. Why can deep convolutional neural networks improve protein fold recognition? A visual explanation by interpretation. Brief Bioinform 2021; 22:6127449. [PMID: 33537753 DOI: 10.1093/bib/bbab001] [Citation(s) in RCA: 9] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/12/2020] [Revised: 12/20/2020] [Accepted: 01/01/2021] [Indexed: 01/26/2023] Open
Abstract
As an essential task in protein structure and function prediction, protein fold recognition has attracted increasing attention. The majority of the existing machine learning-based protein fold recognition approaches strongly rely on handcrafted features, which depict the characteristics of different protein folds; however, effective feature extraction methods still represent the bottleneck for further performance improvement of protein fold recognition. As a powerful feature extractor, deep convolutional neural network (DCNN) can automatically extract discriminative features for fold recognition without human intervention, which has demonstrated an impressive performance on protein fold recognition. Despite the encouraging progress, DCNN often acts as a black box, and as such, it is challenging for users to understand what really happens in DCNN and why it works well for protein fold recognition. In this study, we explore the intrinsic mechanism of DCNN and explain why it works for protein fold recognition using a visual explanation technique. More specifically, we first trained a VGGNet-based DCNN model, termed VGGNet-FE, which can extract fold-specific features from the predicted protein residue-residue contact map for protein fold recognition. Subsequently, based on the trained VGGNet-FE, we implemented a new contact-assisted predictor, termed VGGfold, for protein fold recognition; we then visualized what features were extracted by each of the convolutional layers in VGGNet-FE using a deconvolution technique. Furthermore, we visualized the high-level semantic information, termed fold-discriminative region, of a predicted contact map from the localization map obtained from the last convolutional layer of VGGNet-FE. It is visually confirmed that VGGNet-FE could effectively extract distinct fold-discriminative regions for different types of protein folds, thereby accounting for the improved performance of VGGfold for protein fold recognition. In summary, this study is of great significance for both understanding the working principle of DCNNs in protein fold recognition and exploring the relationship between the predicted protein contact map and protein tertiary structure. This proposed visualization method is flexible and applicable to address other DCNN-based bioinformatics and computational biology questions. The online web server of VGGfold is freely available at http://csbio.njust.edu.cn/bioinf/vggfold/.
Collapse
Affiliation(s)
- Yan Liu
- School of Computer Science and Engineering, Nanjing University of Science and Technology, 200 Xiaolingwei, Nanjing, 210094, China
| | - Yi-Heng Zhu
- Department of Computer Science, Jiangnan University, No. 1800 Lihu Avenue, Wuxi, 214122, China
| | - Xiaoning Song
- Biomedicine Discovery Institute and Department of Biochemistry and Molecular Biology, Monash University, Melbourne, VIC 3800, Australia
| | - Jiangning Song
- Monash Centre for Data Science, Faculty of Information Technology, Monash University, Melbourne, VIC 3800, Australia
| | - Dong-Jun Yu
- School of Computer Science and Engineering, Nanjing University of Science and Technology, 200 Xiaolingwei, Nanjing, 210094, China
| |
Collapse
|
65
|
Malik J, Kiranyaz S, Gabbouj M. Self-organized operational neural networks for severe image restoration problems. Neural Netw 2021; 135:201-211. [PMID: 33401226 DOI: 10.1016/j.neunet.2020.12.014] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/10/2020] [Revised: 12/14/2020] [Accepted: 12/14/2020] [Indexed: 10/22/2022]
Abstract
Discriminative learning based on convolutional neural networks (CNNs) aims to perform image restoration by learning from training examples of noisy-clean image pairs. It has become the go-to methodology for tackling image restoration and has outperformed the traditional non-local class of methods. However, the top-performing networks are generally composed of many convolutional layers and hundreds of neurons, with trainable parameters in excess of several million. We claim that this is due to the inherently linear nature of convolution-based transformation, which is inadequate for handling severe restoration problems. Recently, a non-linear generalization of CNNs, called the operational neural networks (ONN), has been shown to outperform CNN on AWGN denoising. However, its formulation is burdened by a fixed collection of well-known non-linear operators and an exhaustive search to find the best possible configuration for a given architecture, whose efficacy is further limited by a fixed output layer operator assignment. In this study, we leverage the Taylor series-based function approximation to propose a self-organizing variant of ONNs, Self-ONNs, for image restoration, which synthesizes novel nodal transformations on-the-fly as part of the learning process, thus eliminating the need for redundant training runs for operator search. In addition, it enables a finer level of operator heterogeneity by diversifying individual connections of the receptive fields and weights. We perform a series of extensive ablation experiments across three severe image restoration tasks. Even when a strict equivalence of learnable parameters is imposed, Self-ONNs surpass CNNs by a considerable margin across all problems, improving the generalization performance by up to 3 dB in terms of PSNR.
Collapse
Affiliation(s)
- Junaid Malik
- Faculty of Information Technology and Communication Sciences, Tampere University, Tampere, Finland.
| | - Serkan Kiranyaz
- Department of Electrical Engineering, Qatar University, Doha, Qatar
| | - Moncef Gabbouj
- Faculty of Information Technology and Communication Sciences, Tampere University, Tampere, Finland
| |
Collapse
|
66
|
Wen J, Sun H, Fei L, Li J, Zhang Z, Zhang B. Consensus guided incomplete multi-view spectral clustering. Neural Netw 2020; 133:207-219. [PMID: 33227665 DOI: 10.1016/j.neunet.2020.10.014] [Citation(s) in RCA: 13] [Impact Index Per Article: 3.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/26/2020] [Revised: 10/25/2020] [Accepted: 10/29/2020] [Indexed: 10/23/2022]
Abstract
Incomplete multi-view clustering which aims to solve the difficult clustering challenge on incomplete multi-view data collected from diverse domains with missing views has drawn considerable attention in recent years. In this paper, we propose a novel method, called consensus guided incomplete multi-view spectral clustering (CGIMVSC), to address the incomplete clustering problem. Specifically, CGIMVSC seeks to explore the local information within every single-view and the semantic consistent information shared by all views in a unified framework simultaneously, where the local structure is adaptively obtained from the incomplete data rather than pre-constructed via a k-nearest neighbor approach in the existing methods. Considering the semantic consistency of multiple views, CGIMVSC introduces a co-regularization constraint to minimize the disagreement between the common representation and the individual representations with respect to different views, such that all views will obtain a consensus clustering result. Experimental comparisons with some state-of-the-art methods on seven datasets validate the effectiveness of the proposed method on incomplete multi-view clustering.
Collapse
Affiliation(s)
- Jie Wen
- PAMI Research Group, Department of Computer and Information Science, University of Macau, Taipa, Macau
| | - Huijie Sun
- Nanchang Institute of Technology, Nanchang 330044, China; Sun Yat-sen University, Guangzhou 510000, China
| | - Lunke Fei
- School of Computer Science and Technology, Guangdong University of Technology, Guangzhou 510006, China
| | - Jinxing Li
- School of Science and Engineering, Chinese University of Hong Kong (Shenzhen), Shenzhen, 518000, China
| | - Zheng Zhang
- Shenzhen Key Laboratory of Visual Object Detection and Recognition, Harbin Institute of Technology, Shenzhen, Shenzhen 518055, China; Peng Cheng Laboratory, Shenzhen 518055, China
| | - Bob Zhang
- PAMI Research Group, Department of Computer and Information Science, University of Macau, Taipa, Macau.
| |
Collapse
|
67
|
Wang Q, Lai J, Claesen L, Yang Z, Lei L, Liu W. A novel feature representation: Aggregating convolution kernels for image retrieval. Neural Netw 2020; 130:1-10. [PMID: 32589586 DOI: 10.1016/j.neunet.2020.06.010] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/08/2020] [Revised: 06/07/2020] [Accepted: 06/08/2020] [Indexed: 11/25/2022]
Abstract
Activated hidden units in convolutional neural networks (CNNs), known as feature maps, dominate image representation, which is compact and discriminative. For ultra-large datasets, high dimensional feature maps in float format not only result in high computational complexity, but also occupy massive memory space. To this end, a new image representation by aggregating convolution kernels (ACK) is proposed, where some convolution kernels capturing certain patterns are activated. The top-n index numbers of the convolution kernels are extracted directly as image representation in discrete integer values, which rebuild relationship between convolution kernels and image. Furthermore, a distance measurement is defined from the perspective of ordered sets to calculate position-sensitive similarities between image representations. Extensive experiments conducted on Oxford Buildings, Paris, and Holidays, etc., manifest that the proposed ACK achieves competitive performance on image retrieval with much lower computational cost, outperforming the ones using feature maps for image representation.
Collapse
Affiliation(s)
- Qi Wang
- Guangdong University of Technology, Guangzhou 510006, China; Hasselt University, Martelarenlaan 42, Hasselt 3500, Belgium.
| | - Jinxing Lai
- Guangdong University of Technology, Guangzhou 510006, China.
| | - Luc Claesen
- Hasselt University, Martelarenlaan 42, Hasselt 3500, Belgium.
| | - Zhenguo Yang
- Guangdong University of Technology, Guangzhou 510006, China.
| | - Liang Lei
- Guangdong University of Technology, Guangzhou 510006, China.
| | - Wenyin Liu
- Guangdong University of Technology, Guangzhou 510006, China.
| |
Collapse
|
68
|
Tian C, Zhuge R, Wu Z, Xu Y, Zuo W, Chen C, Lin CW. Lightweight image super-resolution with enhanced CNN. Knowl Based Syst 2020. [DOI: 10.1016/j.knosys.2020.106235] [Citation(s) in RCA: 34] [Impact Index Per Article: 8.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/08/2023]
|
69
|
Li J, Li M, Lu G, Zhang B, Yin H, Zhang D. Similarity and diversity induced paired projection for cross-modal retrieval. Inf Sci (N Y) 2020. [DOI: 10.1016/j.ins.2020.06.032] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/24/2022]
|
70
|
|
71
|
Zong M, Wang R, Chen Z, Wang M, Wang X, Potgieter J. Multi-cue based 3D residual network for action recognition. Neural Comput Appl 2020. [DOI: 10.1007/s00521-020-05313-8] [Citation(s) in RCA: 6] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/25/2022]
|
72
|
Liu X, Zeng Z, Wunsch Ii DC. Memristor-based LSTM network with in situ training and its applications. Neural Netw 2020; 131:300-311. [PMID: 32841836 DOI: 10.1016/j.neunet.2020.07.035] [Citation(s) in RCA: 14] [Impact Index Per Article: 3.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/24/2019] [Revised: 06/12/2020] [Accepted: 07/27/2020] [Indexed: 11/25/2022]
Abstract
Artificial neural networks (ANNs), such as the convolutional neural network (CNN) and long short-term memory (LSTM), have high complexity and contain large numbers of parameters. Memristor-based neural networks, which have the ability of in-memory and parallel computing, are therefore proposed to accelerate the operations of ANNs. In this paper, a memristor-based hardware realization of long short-term memory (LSTM) network with in situ training is presented. The designed memristor-based LSTM (MbLSTM) network is composed of memristor-based LSTM cell and memristor-based dense layer. Sigmoid and tanh (hyperbolic tangent) activation functions are approximately implemented through intentionally designing circuit parameters. A weight update scheme with row-parallel characteristic is put forward to update the conductance of memristors in crossbars. The highlights of MbLSTM include an effective hardware-based inference process and in situ training. The validity of MbLSTM is substantiated through classification tasks. The robustness of MbLSTM to conductance variations is also analyzed.
Collapse
Affiliation(s)
- Xiaoyang Liu
- School of Artificial Intelligence and Automation, Huazhong University of Science and Technology, Wuhan 430074, China; Key Laboratory of Image Processing and Intelligent Control of Education Ministry of China, Wuhan 430074, China
| | - Zhigang Zeng
- School of Artificial Intelligence and Automation, Huazhong University of Science and Technology, Wuhan 430074, China; Key Laboratory of Image Processing and Intelligent Control of Education Ministry of China, Wuhan 430074, China.
| | - Donald C Wunsch Ii
- Applied Computational Intelligence Laboratory, Department of Electrical and Computer Engineering, Missouri University of Science and Technology, Rolla, MO 65401, USA
| |
Collapse
|
73
|
Tian C, Fei L, Zheng W, Xu Y, Zuo W, Lin CW. Deep learning on image denoising: An overview. Neural Netw 2020; 131:251-275. [PMID: 32829002 DOI: 10.1016/j.neunet.2020.07.025] [Citation(s) in RCA: 158] [Impact Index Per Article: 39.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/16/2020] [Revised: 06/17/2020] [Accepted: 07/21/2020] [Indexed: 01/19/2023]
Abstract
Deep learning techniques have received much attention in the area of image denoising. However, there are substantial differences in the various types of deep learning methods dealing with image denoising. Specifically, discriminative learning based on deep learning can ably address the issue of Gaussian noise. Optimization models based on deep learning are effective in estimating the real noise. However, there has thus far been little related research to summarize the different deep learning techniques for image denoising. In this paper, we offer a comparative study of deep techniques in image denoising. We first classify the deep convolutional neural networks (CNNs) for additive white noisy images; the deep CNNs for real noisy images; the deep CNNs for blind denoising and the deep CNNs for hybrid noisy images, which represents the combination of noisy, blurred and low-resolution images. Then, we analyze the motivations and principles of the different types of deep learning methods. Next, we compare the state-of-the-art methods on public denoising datasets in terms of quantitative and qualitative analyses. Finally, we point out some potential challenges and directions of future research.
Collapse
Affiliation(s)
- Chunwei Tian
- Bio-Computing Research Center, Harbin Institute of Technology, Shenzhen, Shenzhen, 518055, Guangdong, China; Shenzhen Key Laboratory of Visual Object Detection and Recognition, Shenzhen, 518055, Guangdong, China
| | - Lunke Fei
- School of Computers, Guangdong University of Technology, Guangzhou, 510006, Guangdong, China
| | - Wenxian Zheng
- Tsinghua Shenzhen International Graduate School, Shenzhen, 518055, Guangdong, China
| | - Yong Xu
- Bio-Computing Research Center, Harbin Institute of Technology, Shenzhen, Shenzhen, 518055, Guangdong, China; Shenzhen Key Laboratory of Visual Object Detection and Recognition, Shenzhen, 518055, Guangdong, China; Peng Cheng Laboratory, Shenzhen, 518055, Guangdong, China.
| | - Wangmeng Zuo
- School of Computer Science and Technology, Harbin Institute of Technology, Harbin, 150001, Heilongjiang, China; Peng Cheng Laboratory, Shenzhen, 518055, Guangdong, China
| | - Chia-Wen Lin
- Department of Electrical Engineering and the Institute of Communications Engineering, National Tsing Hua University, Hsinchu, Taiwan
| |
Collapse
|
74
|
Liang QK, Peng JZ, Li ZW, Xie DQ, Sun W, Wang YN, Zhang D. Robust table recognition for printed document images. MATHEMATICAL BIOSCIENCES AND ENGINEERING : MBE 2020; 17:3203-3223. [PMID: 32987525 DOI: 10.3934/mbe.2020182] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/11/2023]
Abstract
The recognition and analysis of tables on printed document images is a popular research field of the pattern recognition and image processing. Existing table recognition methods usually require high degree of regularity, and the robustness still needs significant improvement. This paper focuses on a robust table recognition system that mainly consists of three parts: Image preprocessing, cell location based on contour mutual exclusion, and recognition of printed Chinese characters based on deep learning network. A table recognition app has been developed based on these proposed algorithms, which can transform the captured images to editable text in real time. The effectiveness of the table recognition app has been verified by testing a dataset of 105 images. The corresponding test results show that it could well identify high-quality tables, and the recognition rate of low-quality tables with distortion and blur reaches 81%, which is considerably higher than those of the existing methods. The work in this paper could give insights into the application of the table recognition and analysis algorithms.
Collapse
Affiliation(s)
- Qiao Kang Liang
- College of Electrical and Information Engineering, Hunan University, Changsha 410082, China
- National Engineering Laboratory for Robot Vision Perception and Control, Hunan University, Changsha 410082, China
| | - Jian Zhong Peng
- College of Electrical and Information Engineering, Hunan University, Changsha 410082, China
- National Engineering Laboratory for Robot Vision Perception and Control, Hunan University, Changsha 410082, China
| | - Zheng Wei Li
- Department of Mechanical Engineering, University of Alberta, Edmonton, AB T6G 2R3, Canada
| | - Da Qi Xie
- College of Electrical and Information Engineering, Hunan University, Changsha 410082, China
- National Engineering Laboratory for Robot Vision Perception and Control, Hunan University, Changsha 410082, China
| | - Wei Sun
- College of Electrical and Information Engineering, Hunan University, Changsha 410082, China
- National Engineering Laboratory for Robot Vision Perception and Control, Hunan University, Changsha 410082, China
| | - Yao Nan Wang
- College of Electrical and Information Engineering, Hunan University, Changsha 410082, China
- National Engineering Laboratory for Robot Vision Perception and Control, Hunan University, Changsha 410082, China
| | - Dan Zhang
- Department of Mechanical Engineering, York University, Toronto, ON M3J 1P3, Canada
| |
Collapse
|
75
|
Yuan D, Li X, He Z, Liu Q, Lu S. Visual object tracking with adaptive structural convolutional network. Knowl Based Syst 2020. [DOI: 10.1016/j.knosys.2020.105554] [Citation(s) in RCA: 47] [Impact Index Per Article: 11.8] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/24/2022]
|
76
|
Jia F, Wang X, Guan J, Liao Q, Zhang J, Li H, Qi S. Bi-Connect Net for salient object detection. Neurocomputing 2020. [DOI: 10.1016/j.neucom.2019.12.020] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/17/2022]
|
77
|
Yuan D, Fan N, He Z. Learning target-focusing convolutional regression model for visual object tracking. Knowl Based Syst 2020. [DOI: 10.1016/j.knosys.2020.105526] [Citation(s) in RCA: 45] [Impact Index Per Article: 11.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/16/2022]
|
78
|
Tian C, Xu Y, Li Z, Zuo W, Fei L, Liu H. Attention-guided CNN for image denoising. Neural Netw 2020; 124:117-129. [PMID: 31991307 DOI: 10.1016/j.neunet.2019.12.024] [Citation(s) in RCA: 110] [Impact Index Per Article: 27.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/13/2019] [Revised: 11/15/2019] [Accepted: 12/23/2019] [Indexed: 10/25/2022]
Abstract
Deep convolutional neural networks (CNNs) have attracted considerable interest in low-level computer vision. Researches are usually devoted to improving the performance via very deep CNNs. However, as the depth increases, influences of the shallow layers on deep layers are weakened. Inspired by the fact, we propose an attention-guided denoising convolutional neural network (ADNet), mainly including a sparse block (SB), a feature enhancement block (FEB), an attention block (AB) and a reconstruction block (RB) for image denoising. Specifically, the SB makes a tradeoff between performance and efficiency by using dilated and common convolutions to remove the noise. The FEB integrates global and local features information via a long path to enhance the expressive ability of the denoising model. The AB is used to finely extract the noise information hidden in the complex background, which is very effective for complex noisy images, especially real noisy images and bind denoising. Also, the FEB is integrated with the AB to improve the efficiency and reduce the complexity for training a denoising model. Finally, a RB aims to construct the clean image through the obtained noise mapping and the given noisy image. Additionally, comprehensive experiments show that the proposed ADNet performs very well in three tasks (i.e. synthetic and real noisy images, and blind denoising) in terms of both quantitative and qualitative evaluations. The code of ADNet is accessible at https://github.com/hellloxiaotian/ADNet.
Collapse
Affiliation(s)
- Chunwei Tian
- Bio-Computing Research Center, Harbin Institute of Technology, Shenzhen, 518055, Guangdong, China
| | - Yong Xu
- Bio-Computing Research Center, Harbin Institute of Technology, Shenzhen, 518055, Guangdong, China; Peng Cheng Laboratory, Shenzhen, 518055, Guangdong, China.
| | - Zuoyong Li
- Fujian Provincial Key Laboratory of Information Processing and Intelligent Control, Minjiang University, Fuzhou, 350121, Fujian, China.
| | - Wangmeng Zuo
- School of Computer Science and Technology, Harbin Institute of Technology, Harbin, 150001, Heilongjiang, China
| | - Lunke Fei
- School of Computer Science and Technology, Guangdong University of Technology, Guangzhou, 510006, Guangdong, China
| | - Hong Liu
- Engineering Lab on Intelligent Perception for Internet of Things, Shenzhen Graduate School, Peking University, Shenzhen, 518055, Guangdong, China
| |
Collapse
|
79
|
Alla Takam C, Samba O, Tchagna Kouanou A, Tchiotsop D. Spark Architecture for deep learning-based dose optimization in medical imaging. INFORMATICS IN MEDICINE UNLOCKED 2020. [DOI: 10.1016/j.imu.2020.100335] [Citation(s) in RCA: 13] [Impact Index Per Article: 3.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 02/07/2023] Open
|