Gated temporal convolutional neural network and expert features for diagnosing and explaining physiological time series: A case study on heart rates

A Review on Deep Sequential Models for Forecasting Time Series Data

Deep sequential (DS) models are extensively employed for forecasting time series data since the dawn of the deep learning era, and they provide forecasts for the values required in subsequent time steps. DS models, unlike other traditional statistical models for forecasting time series data, can learn hidden patterns in temporal sequences and have the memorizing data from prior time points. Given the widespread usage of deep sequential models in several domains, a comprehensive study describing their applications is necessary. This work presents a comprehensive review of contemporary deep learning time series models, their performance in diverse domains, and an investigation of the models that were employed in various applications. Three deep sequential models, namely, artificial neural network (ANN), long short-term memory (LSTM), and temporal-conventional neural network (TCNN) along with their applications for forecasting time series data, are elaborated. We showed a comprehensive comparison between such models in terms of application fields, model structure and activation functions, optimizers, and implementation, with a goal of learning more about the optimal model used. Furthermore, the challenges and perspectives of future development of deep sequential models are presented and discussed. We conclude that the LSTM model is widely employed, particularly in the form of a hybrid model, in which the most accurate predictions are made when the shape of hybrids is used as the model.

Small-Footprint Keyword Spotting Based on Gated Channel Transformation Sandglass Residual Neural Network

Keyword spotting plays a crucial role in realizing voice-based user interaction on intelligent equipment terminals and service robots. In this task, it remains challenging to achieve the balance between low memory and high precision. To better satisfy this requirement, we propose an end-to-end neural architecture with sandglass residual blocks embedded with the gated channel-wise attention mechanism. The sandglass residual blocks utilize 1D separable convolutions to extract bottleneck temporal features, which can effectively drive the model to focus more on the speech segment with lower parameters. Especially, the gated attention mechanism helps the model enhance the critical speech temporal features and suppress the useless ones and further focus on the most important part of the human speech region for keyword spotting. The experimental results on Google Speech Commands Dataset show that our proposed model has an accuracy of 97.4[Formula: see text] with only 46K parameters. Compared with the baseline method with the highest accuracy, our model parameters are decreased by 54[Formula: see text] and accuracy is increased by 0.8[Formula: see text]. That makes us take further step in achieving the goal of low memory and high precision.

Machine Learning-Based Algorithms to Knowledge Extraction from Time Series Data: A Review

To predict the future behavior of a system, we can exploit the information collected in the past, trying to identify recurring structures in what happened to predict what could happen, if the same structures repeat themselves in the future as well. A time series represents a time sequence of numerical values observed in the past at a measurable variable. The values are sampled at equidistant time intervals, according to an appropriate granular frequency, such as the day, week, or month, and measured according to physical units of measurement. In machine learning-based algorithms, the information underlying the knowledge is extracted from the data themselves, which are explored and analyzed in search of recurring patterns or to discover hidden causal associations or relationships. The prediction model extracts knowledge through an inductive process: the input is the data and, possibly, a first example of the expected output, the machine will then learn the algorithm to follow to obtain the same result. This paper reviews the most recent work that has used machine learning-based techniques to extract knowledge from time series data.

Addressing Noise and Skewness in Interpretable Health-Condition Assessment by Learning Model Confidence

Assessing the health condition has a wide range of applications in healthcare, military, aerospace, and industrial fields. Nevertheless, traditional feature-engineered techniques involve manual feature extraction, which are too cumbersome to adapt to the changes caused by the development of sensor network technology. Recently, deep-learning-based methods have achieved initial success in health-condition assessment research, but insufficient considerations for problems such as class skewness, noisy segments, and result interpretability make it difficult to apply them to real-world applications. In this paper, we propose a K-margin-based Interpretable Learning approach for health-condition assessment. In detail, a skewness-aware RCR-Net model is employed to handle problems of class skewness. Furthermore, we present a diagnosis model based on K-margin to automatically handle noisy segments by naturally exploiting expected consistency among the segments associated with each record. Additionally, a knowledge-directed interpretation method is presented to learn domain knowledge-level features automatically without the help of human experts which can be used as an interpretable decision-making basis. Finally, through experimental validation in the field of both medical and aerospace, the proposed method has a better generality and high efficiency with 0.7974 and 0.8005 F1 scores, which outperform all state-of-the-art deep learning methods for health-condition assessment task by 3.30% and 2.99%, respectively.