Auto-Colorization of Historical Images Using Deep Convolutional Neural Networks

Study on the Application of Visual Communication Design in APP Interface Design in the Context of Deep Learning

Visual communication concepts enable linguistics or semiotics to the teaching of visual communication designs, creating graphic designs into an innovative and scientific discipline. The use of storyline techniques in visual communication not only inspires the imagination of designer but also arouses the visual memory of the audience. Besides, improving cultural heritage such as historical images is important to protect cultural diversity. Recently, the developments of deep learning (DL) and computer vision (CV) approaches make it possible for the automatic colorization of grayscale images into color images. Also, the usage of visual communication design in APP interface design has increased. With this motivation, this work introduces the enhanced deep learning-based automated historical image colorization (EDL-AHIC) technique for wireless network-enabled visual communication. The proposed EDL-AHIC technique intends to effectually convert the grayscale images into color images. The presented EDL-AHIC technique extracts the local as well as global features. For global feature extraction, the enhanced capsule network (ECN) model is applied. Finally, the fusion layer and decoding unit are employed to determine the output, i.e., chrominance component of the input image. A comprehensive experimental validation process is performed to ensure the betterment of the EDL-AHIC technique. The comparison study reported the supremacy of the EDL-AHIC technique over the other recent methods.

Multi-Output Learning Based on Multimodal GCN and Co-Attention for Image Aesthetics and Emotion Analysis

With the development of social networks and intelligent terminals, it is becoming more convenient to share and acquire images. The massive growth of the number of social images makes people have higher demands for automatic image processing, especially in the aesthetic and emotional perspective. Both aesthetics assessment and emotion recognition require a higher ability for the computer to simulate high-level visual perception understanding, which belongs to the field of image processing and pattern recognition. However, existing methods often ignore the prior knowledge of images and intrinsic relationships between aesthetic and emotional perspectives. Recently, machine learning and deep learning have become powerful methods for researchers to solve mathematical problems in computing, such as image processing and pattern recognition. Both images and abstract concepts can be converted into numerical matrices and then establish the mapping relations using mathematics on computers. In this work, we propose an end-to-end multi-output deep learning model based on multimodal Graph Convolutional Network (GCN) and co-attention for aesthetic and emotion conjoint analysis. In our model, a stacked multimodal GCN network is proposed to encode the features under the guidance of the correlation matrix, and a co-attention module is designed to help the aesthetics and emotion feature representation learn from each other interactively. Experimental results indicate that our proposed model achieves competitive performance on the IAE dataset. Progressive results on the AVA and ArtPhoto datasets also prove the generalization ability of our model.

Principal Component Wavelet Networks for Solving Linear Inverse Problems

In this paper we propose a novel learning-based wavelet transform and demonstrate its utility as a representation in solving a number of linear inverse problems—these are asymmetric problems, where the forward problem is easy to solve, but the inverse is difficult and often ill-posed. The wavelet decomposition is comprised of the application of an invertible 2D wavelet filter-bank comprising symmetric and anti-symmetric filters, in combination with a set of 1×1 convolution filters learnt from Principal Component Analysis (PCA). The 1×1 filters are needed to control the size of the decomposition. We show that the application of PCA across wavelet subbands in this way produces an architecture equivalent to a separable Convolutional Neural Network (CNN), with the principal components forming the 1×1 filters and the subtraction of the mean forming the bias terms. The use of an invertible filter bank and (approximately) invertible PCA allows us to create a deep autoencoder very simply, and avoids issues of overfitting. We investigate the construction and learning of such networks, and their application to linear inverse problems via the Alternating Direction of Multipliers Method (ADMM). We use our network as a drop-in replacement for traditional discrete wavelet transform, using wavelet shrinkage as the projection operator. The results show good potential on a number of inverse problems such as compressive sensing, in-painting, denoising and super-resolution, and significantly close the performance gap with Generative Adversarial Network (GAN)-based methods.