Deep octonion networks

Universal approximation theorem for vector- and hypercomplex-valued neural networks

The universal approximation theorem states that a neural network with one hidden layer can approximate continuous functions on compact sets with any desired precision. This theorem supports using neural networks for various applications, including regression and classification tasks. Furthermore, it is valid for real-valued neural networks and some hypercomplex-valued neural networks such as complex-, quaternion-, tessarine-, and Clifford-valued neural networks. However, hypercomplex-valued neural networks are a type of vector-valued neural network defined on an algebra with additional algebraic or geometric properties. This paper extends the universal approximation theorem for a wide range of vector-valued neural networks, including hypercomplex-valued models as particular instances. Precisely, we introduce the concept of non-degenerate algebra and state the universal approximation theorem for neural networks defined on such algebras.

PHNNs: Lightweight Neural Networks via Parameterized Hypercomplex Convolutions

Hypercomplex neural networks have proven to reduce the overall number of parameters while ensuring valuable performance by leveraging the properties of Clifford algebras. Recently, hypercomplex linear layers have been further improved by involving efficient parameterized Kronecker products. In this article, we define the parameterization of hypercomplex convolutional layers and introduce the family of parameterized hypercomplex neural networks (PHNNs) that are lightweight and efficient large-scale models. Our method grasps the convolution rules and the filter organization directly from data without requiring a rigidly predefined domain structure to follow. PHNNs are flexible to operate in any user-defined or tuned domain, from 1-D to [Formula: see text] regardless of whether the algebra rules are preset. Such a malleability allows processing multidimensional inputs in their natural domain without annexing further dimensions, as done, instead, in quaternion neural networks (QNNs) for 3-D inputs like color images. As a result, the proposed family of PHNNs operates with 1/n free parameters as regards its analog in the real domain. We demonstrate the versatility of this approach to multiple domains of application by performing experiments on various image datasets and audio datasets in which our method outperforms real and quaternion-valued counterparts. Full code is available at: https://github.com/eleGAN23/HyperNets.

An octonion-based nonlinear echo state network for speech emotion recognition in Metaverse

While the Metaverse is becoming a popular trend and drawing much attention from academia, society, and businesses, processing cores used in its infrastructures need to be improved, particularly in terms of signal processing and pattern recognition. Accordingly, the speech emotion recognition (SER) method plays a crucial role in creating the Metaverse platforms more usable​ and enjoyable for its users. However, existing SER methods continue to be plagued by two significant problems in the online environment. The shortage of adequate engagement and customization between avatars and users is recognized as the first issue and the second problem is related to the complexity of SER problems in the Metaverse as we face people and their digital twins or avatars. This is why developing efficient machine learning (ML) techniques specified for hypercomplex signal processing is essential to enhance the impressiveness and tangibility of the Metaverse platforms. As a solution, echo state networks (ESNs), which are an ML powerful tool for SER, can be an appropriate technique to enhance the Metaverse's foundations in this area. Nevertheless, ESNs have some technical issues restricting them from a precise and reliable analysis, especially in the aspect of high-dimensional data. The most significant limitation of these networks is the high memory consumption caused by their reservoir structure in face of high-dimensional signals. To solve all problems associated with ESNs and their application in the Metaverse, we have come up with a novel structure for ESNs empowered by octonion algebra called NO2GESNet. Octonion numbers have eight dimensions, compactly display high-dimensional data, and improve the network precision and performance in comparison to conventional ESNs. The proposed network also solves the weaknesses of the ESNs in the presentation of the higher-order statistics to the output layer by equipping it with a multidimensional bilinear filter. Three comprehensive scenarios to use the proposed network in the Metaverse have been designed and analyzed, not only do they show the accuracy and performance of the proposed approach, but also the ways how SER can be employed in the Metaverse platforms.

Fast Algorithms for Deep Octonion Networks

This brief presents the results of a study of the possibilities of reducing the arithmetic complexity of computing basic operations in octonionic neural networks and also proposes new algorithmic solutions for efficiently performing these operations. Here, we primarily mean the operation of multiplying octonions, the operation of computing the dot product of two octonion-valued vectors, and the operation of multiple multiplications of an octonion by several other octonions. In order to reduce the computational complexity of these operations, it is proposed to use the fast Walsh-Hadamard transform, which is well known in digital signal processing. Using this transform reduces the number of multiplications and additions of real numbers required to perform computations. Thus, the use of the proposed algorithms will speed up computations in octonion-valued neural networks.

On the Octonion Cross Wigner Distribution of 3-D Signals

This paper introduces definitions of the octonion cross Wigner distribution (OWD) and the octonion ambiguity function, forming a pair of octonion Fourier transforms. The main part is devoted to the study of the basic properties of the OWD. Among them are the properties concerning its nature (nonlinearity, parity, space support conservation, marginals) and some “geometric” transformations (space shift, space scaling) similar to the case of the complex Wigner distribution. This paper also presents specific forms of the modulation property and an extended discussion about the validity of Moyal’s formula and the uncertainty principle, accompanied by new theorems and examples. The paper is illustrated with examples of 3-D separable Gaussian and Gabor signals. The concept of the application of the OWD for the analysis of multidimensional analytic signals is also proposed. The theoretical results presented in the papers are summarized, and the possibility of further research is discussed.

Computationally Efficient Barycentric Interpolation of Large Grain Boundary Octonion Point Sets

We present a method for performing efficient barycentric interpolation for large grain boundary octonion point sets which reside on the surface of a hypersphere. This method includes removal of degenerate dimensions via singular value decomposition (SVD) transformations and linear projections, determination of intersecting facets via nearest neighbor (NN) searches, and interpolation. This method is useful for hyperspherical point sets for applications such as grain boundaries structure-property models, robotics, and specialized neural networks. We provide a case study of the method applied to the 7-sphere. We provide 1-sphere and 2-sphere visualizations to illustrate important aspects of these dimension reduction and interpolation methods. A MATLAB implementation is available at github.com/sgbaird-5dof/interp.•

Barycentric interpolation is combined with hypersphere facet intersections, dimensionality reduction, and linear projections to reduce computational complexity without loss of information

•

A max nearest neighbor threshold is used in conjunction with facet intersection determination to reduce computational runtime.

Region-Based Split Octonion Networks with Channel Attention Module for Tuna Classification

Tuna fish is a popular food because of its nutritional value and taste. Demand for various species of tuna increases over time, necessitating the development of a system to sort tuna fish into distinct species in export sectors in order to accelerate the process. The work proposes an automated tuna classification system based on split octonion network. The images are initially preprocessed and divided into region images. Each region image is applied to a split octonion network with eleven layers. In addition, a split octonion channel attention module is presented, which is fed to the last two convolutional layers. The features from the three octonion networks are fused and applied to a series of dense layers. In the last layer, a softmax classifier is utilized for final classification. Results show that the proposed region-based split octonion network with attention module gives an accuracy of 98.01% on tuna database. The region-based tuna classification model is fine-tuned for the categorization of six species from QUT-FishBase dataset and Fish-Pak dataset. The system shows accuracies of 97.83% and 98.17% on QUT-FishBase and Fish-Pak datasets, respectively. The proposed methodology is also compared with existing approaches using a variety of evaluation criteria.

A Parallel Algorithm for Dividing Octonions

The article presents a parallel hardware-oriented algorithm designed to speed up the division of two octonions. The advantage of the proposed algorithm is that the number of real multiplications is halved as compared to the naive method for implementing this operation. In the synthesis of the discussed algorithm, the matrix representation of this operation was used, which allows us to present the division of octonions by means of a vector–matrix product. Taking into account a specific structure of the matrix multiplicand allows for reducing the number of real multiplications necessary for the execution of the octonion division procedure.