Time series forecasting of petroleum production using deep LSTM recurrent networks

Physical Reservoir Computing Using van der Waals Ferroelectrics for Acoustic Keyword Spotting

Acoustic keyword spotting (KWS) plays a pivotal role in the voice-activated systems of artificial intelligence (AI), allowing for hands-free interactions between humans and smart devices through information retrieval of the voice commands. The cloud computing technology integrated with the artificial neural networks has been employed to execute the KWS tasks, which however suffers from propagation delay and the risk of privacy breach. Here, we report a single-node reservoir computing (RC) system based on the CuInP2S6 (CIPS)/graphene heterostructure planar device for implementing the KWS task with low computation cost. Through deliberately tuning the Schottky barrier height at the ferroelectric CIPS interfaces for the thermionic injection and transport of the electrons, the typical nonlinear current response and fading memory characteristics are achieved in the device. Additionally, the device exhibits diverse synaptic plasticity with an excellent separation capability of the temporal information. We construct a RC system through employing the ferroelectric device as the physical node to spot the acoustic keywords, i.e., the natural numbers from 1 to 9 based on simulation, in which the system demonstrates outstanding performance with high accuracy rate (>94.6%) and recall rate (>92.0%). Our work promises physical RC in single-node configuration as a prospective computing platform to process the acoustic keywords, promoting its applications in the artificial auditory system at the edge.

Soft sensing of NOx emission from waste incineration process based on data de-noising and bidirectional long short-term memory neural networks

Continuous emission monitoring system is commonly employed to monitor NOx emissions in municipal solid waste incineration (MSWI) processes. However, it still encounters the challenges of regular maintenance and measurement lag. These issues significantly impact the accurate and stable control of NOx emissions. Therefore, developing a soft NOx emission sensor to complement hardware monitoring becomes imperative. Considering data noise, dynamic nonlinearity, time series characteristics and volatility in the MSWI process, this article introduces a soft sensor model for NOx emission prediction utilizing the complete ensemble empirical mode decomposition adaptive noise (CEEMDAN)-wavelet threshold (WT) method and bidirectional long short-term memory (Bi-LSTM). Firstly, the original data signal is decomposed into a group of intrinsic mode functions (IMFs) using the CEEMDAN. Subsequently, the WT processes the high-frequency IMFs that are noise-dominant. Then, all IMFs are reconstructed to obtain the denoized signal. Finally, the Bi-LSTM model is employed to predict NOx emissions. Compared to conventional modelling approaches, the model proposed in this article demonstrates the best predictive performance. The mean absolute percentage error, root-mean-squared error and average absolute error on the test set of the proposed model are 3.75%, 5.34 mg m-3 and 4.34 mg m-3, respectively. The proposed model provides a new method to soft sensing NOx emissions. It holds significant practical value for precise and stable monitoring of NOx emissions in MSWI processes and provides a reference for research on modelling key process parameters.

Impact of Biotic and Abiotic Factors on Listeria monocytogenes, Salmonella enterica, and Enterohemorrhagic Escherichia coli in Agricultural Soil Extracts

Outbreaks of Enterohemorrhagic Escherichia coli (EHEC), Salmonella enterica, and Listeria monocytogenes linked to fresh produce consumption pose significant food safety concerns. These pathogens can contaminate pre-harvest produce through various routes, including contaminated water. Soil physicochemical properties and flooding can influence pathogen survival in soils. We investigated survival of EHEC, S. enterica, and L. monocytogenes in soil extracts designed to represent soils with stagnant water. We hypothesized pathogen survival would be influenced by soil extract nutrient levels and the presence of native microbes. A chemical analysis revealed higher levels of total nitrogen, phosphorus, and carbon in high-nutrient soil extracts compared to low-nutrient extracts. Pathogen survival was enhanced in high-nutrient, sterile soil extracts, while the presence of native microbes reduced pathogen numbers. A microbiome analysis showed greater diversity in low-nutrient soil extracts, with distinct microbial compositions between extract types. Our findings highlight the importance of soil nutrient composition and microbial dynamics in influencing pathogen behavior. Given key soil parameters, a long short-term memory model (LSTM) effectively predicted pathogen survival. Integrating these factors can aid in developing predictive models for pathogen persistence in agricultural systems. Overall, our study contributes to understanding the complex interplay in agricultural ecosystems, facilitating informed decision-making for crop production and food safety enhancement.

Synergistic Enhancement of Water-Splitting Performance Using MOF-Derived Ceria-Modified g-C3N4 Nanocomposites: Synthesis, Performance Evaluation, and Stability Prediction with Machine Learning

Diminishing the charge recombination rate by improving the photoelectrochemical (PEC) performance of graphitic carbon nitride (g-C3N4) is essential for better water oxidation. In this concern, this research explores the competent approach to enhance the PEC performance of g-C3N4 nanosheets (NSs), creating their nanocomposites (NCs) with metal-organic framework (MOF)-derived porous CeO2 nanobars (NBs) along with ZnO nanorods (NRs) and TiO2 nanoparticles (NPs). The synthesis involved preparing CeO2 NBs and g-C3N4 NSs through the calcination of respective precursors, while the sol-gel method is employed for ZnO NRs and TiO2 NPs. Following the subsequent analysis of the physicochemical properties of the materials, the binder-free brush-coating method is deployed to fabricate NC-based photoanodes, followed by an evaluation of the PEC performance through various electrochemical techniques. Remarkably, the binary g-C3N4/CeO2 NCs with 20 wt % CeO2 NBs (gC20 NCs) exhibited a significantly enhanced current density of 0.460 mA/cm2 at 1.23 V vs reversible hydrogen electrode, which is 2.3 times greater than that of bare g-C3N4 NSs (0.195 mA/cm2). Further improvements are observed with ternary gC20/TiO2 (gCT50) and gC20/ZnO (gCZ50) NCs, achieving current densities of 1.810 and 1.440 mA/cm2, respectively. These enhanced current densities are attributed to increased donor densities, reduced charge transfer resistances, and efficient charge transport within the NCs. In addition, higher surface areas with beneficial instinctive defects are perceived for gCT50 and gCZ50 NCs, as revealed by Brunauer-Emmett-Teller and electron spin resonance analysis. Finally, the stability of gCZ50 and gCT50 NC-based photoanodes is predicted and forecasted with the help of the recurrent neural network-based long short-term memory technique. Overall, this study demonstrates the efficacy of organic-inorganic hybrids for efficient photoanodes, facilitating advancements in water-splitting studies.

Alternative muscle synergy patterns of upper limb amputees

Myoelectric hand prostheses are effective tools for upper limb amputees to regain hand functions. Much progress has been made with pattern recognition algorithms to recognize surface electromyography (sEMG) patterns, but few attentions was placed on the amputees' motor learning process. Many potential myoelectric prostheses users could not fully master the control or had declined performance over time. It is possible that learning to produce distinct and consistent muscle activation patterns with the residual limb could help amputees better control the myoelectric prosthesis. In this study, we observed longitudinal effect of motor skill learning with 2 amputees who have developed alternative muscle activation patterns in response to the same set of target prosthetic actions. During a 10-week program, amputee participants were trained to produce distinct and constant muscle activations with visual feedback of live sEMG and without interaction with prosthesis. At the end, their sEMG patterns were different from each other and from non-amputee control groups. For certain intended hand motion, gradually reducing root mean square (RMS) variance was observed. The learning effect was also assessed with a CNN-LSTM mixture classifier designed for mobile sEMG pattern recognition. The classification accuracy had a rising trend over time, implicating potential performance improvement of myoelectric prosthesis control. A follow-up session took place 6 months after the program and showed lasting effect of the motor skill learning in terms of sEMG pattern classification accuracy. The results indicated that with proper feedback training, amputees could learn unique muscle activation patterns that allow them to trigger intended prosthesis functions, and the original motor control scheme is updated. The effect of such motor skill learning could help to improve myoelectric prosthetic control performance.

Short-term forecasting approach of single well production based on multi-intelligent agent hybrid model

The short-term prediction of single well production can provide direct data support for timely guiding the optimization and adjustment of oil well production parameters and studying and judging oil well production conditions. In view of the coupling effect of complex factors on the daily output of a single well, a short-term prediction method based on a multi-agent hybrid model is proposed, and a short-term prediction process of single well output is constructed. First, CEEMDAN method is used to decompose and reconstruct the original data set, and the sliding window method is used to compose the data set with the obtained components. Features of components by decomposition are described as feature vectors based on values of fuzzy entropy and autocorrelation coefficient, through which those components are divided into two groups using cluster algorithm for prediction with two sub models. Optimized online sequential extreme learning machine and the deep learning model based on encoder-decoder structure using self-attention are developed as sub models to predict the grouped data, and the final predicted production comes from the sum of prediction values by sub models. The validity of this method for short-term production prediction of single well daily oil production is verified. The statistical value of data deviation and statistical test methods are introduced as the basis for comparative evaluation, and comparative models are used as the reference model to evaluate the prediction effect of the above multi-agent hybrid model. Results indicated that the proposed hybrid model has performed better with MAE value of 0.0935, 0.0694 and 0.0593 in three cases, respectively. By comparison, the short-term prediction method of single well production based on multi-agent hybrid model has considerably improved the statistical value of prediction deviation of selected oil well data in different periods. Through statistical test, the multi-agent hybrid model is superior to the comparative models. Therefore, the short-term prediction method of single well production based on a multi-agent hybrid model can effectively optimize oilfield production parameters and study and judge oil well production conditions.

Developing a multivariate time series forecasting framework based on stacked autoencoders and multi-phase feature

Time series forecasting across different domains has received massive attention as it eases intelligent decision-making activities. Recurrent neural networks and various deep learning algorithms have been applied to modeling and forecasting multivariate time series data. Due to intricate non-linear patterns and significant variations in the randomness of characteristics across various categories of real-world time series data, achieving effectiveness and robustness simultaneously poses a considerable challenge for specific deep-learning models. We have proposed a novel prediction framework with a multi-phase feature selection technique, a long short-term memory-based autoencoder, and a temporal convolution-based autoencoder to fill this gap. The multi-phase feature selection is applied to retrieve the optimal feature selection and optimal lag window length for different features. Moreover, the customized stacked autoencoder strategy is employed in the model. The first autoencoder is used to resolve the random weight initialization problem. Additionally, the second autoencoder models the temporal relation between non-linear correlated features with convolution networks and recurrent neural networks. Finally, the model's ability to generalize, predict accurately, and perform effectively is validated through experimentation with three distinct real-world time series datasets. In this study, we conducted experiments on three real-world datasets: Energy Appliances, Beijing PM2.5 Concentration, and Solar Radiation. The Energy Appliances dataset consists of 29 attributes with a training size of 15,464 instances and a testing size of 4239 instances. For the Beijing PM2.5 Concentration dataset, there are 18 attributes, with 34,952 instances in the training set and 8760 instances in the testing set. The Solar Radiation dataset comprises 11 attributes, with 22,857 instances in the training set and 9797 instances in the testing set. The experimental setup involved evaluating the performance of forecasting models using two distinct error measures: root mean square error and mean absolute error. To ensure robust evaluation, the errors were calculated at the identical scale of the data. The results of the experiments demonstrate the superiority of the proposed model compared to existing models, as evidenced by significant advantages in various metrics such as mean squared error and mean absolute error. For PM2.5 air quality data, the proposed model's mean absolute error is 7.51 over 12.45, about ∼40% improvement. Similarly, the mean square error for the dataset is improved from 23.75 to 11.62, which is ∼51%of improvement. For the solar radiation dataset, the proposed model resulted in ∼34.7% improvement in means squared error and ∼75% in mean absolute error. The recommended framework demonstrates outstanding capabilities in generalization and outperforms datasets spanning multiple indigenous domains.

Meteorological factors cannot be ignored in machine learning-based methods for predicting dengue, a systematic review

In recent years, there has been a rapid increase in the application of machine learning methods about predicting the incidence of dengue fever. However, the predictive factors and models employed in different studies vary greatly. Hence, we conducted a systematic review to summarize machine learning methods and predictors in previous studies. We searched PubMed, ScienceDirect, and Web of Science databases for articles published up to July 2023. The selected papers included not only the forecast of dengue incidence but also machine learning methods. A total of 23 papers were included in this study. Predictive factors included meteorological factors (22, 95.7%), historical dengue data (14, 60.9%), environmental factors (4, 17.4%), socioeconomic factors (4, 17.4%), vector surveillance data (2, 8.7%), and internet search data (3, 13.0%). Among meteorological factors, temperature (20, 87.0%), rainfall (20, 87.0%), and relative humidity (14, 60.9%) were the most commonly used. We found that Support Vector Machine (SVM) (6, 26.1%), Long Short-Term Memory (LSTM) (5, 21.7%), Random Forest (RF) (4, 17.4%), Least Absolute Shrinkage and Selection Operator (LASSO) (2, 8.7%), ensemble model (2, 8.7%), and other models (4, 17.4%) were identified as the best models based on evaluation metrics used in each article. These results indicate that meteorological factors are important predictors that cannot be ignored and SVM and LSTM algorithms are the most commonly used models in dengue fever prediction with good predictive performance. This review will contribute to the development of more robust early dengue warning systems and promote the application of machine learning methods in predicting climate-related infectious diseases.

A CNN-LSTM based deep learning model with high accuracy and robustness for carbon price forecasting: A case of Shenzhen's carbon market in China

Accurately predicting carbon trading prices using deep learning models can help enterprises understand the operational mechanisms and regulations of the carbon market. This is crucial for expanding the industries covered by the carbon market and ensuring its stable and healthy development. To ensure the accuracy and reliability of the predictions in practical applications, it is important to evaluate the model's robustness. In this paper, we built models with different parameters to predict carbon trading prices, and proposed models with high accuracy and robustness. The accuracy of the models was assessed using traditional survey indicators. The robustness of the CNN-LSTM model was compared to that of the LSTM model using Z-scores. The CNN-LSTM model with the best prediction performance was compared to a single LSTM model, resulting in a 9% reduction in MSE and a 0.0133 shortening of the Z-score range. Furthermore, the CNN-LSTM model achieved a level of accuracy comparable to other popular models such as CEEMDAN, Boosting, and GRU. It also demonstrated a training speed improvement of at least 40% compared to the aforementioned methods. These results suggest that the CNN-LSTM enhances model resilience. Moreover, the practicality of using Z-score to evaluate model robustness is confirmed.

Remote sensing-enabled machine learning for river water quality modeling under multidimensional uncertainty

Two fundamental problems have inhibited progress in the simulation of river water quality under climate (and other) uncertainty: 1) insufficient data, and 2) the inability of existing models to account for the complexity of factors (e.g., hydro-climatic, basin characteristics, land use features) affecting river water quality. To address these concerns this study presents a technique for augmenting limited ground-based observations of water quality variables with remote-sensed surface reflectance data by leveraging a machine learning model capable of accommodating the multidimensionality of water quality influences. Total Suspended Solids (TSS) can serve as a surrogate for chemical and biological pollutants of concern in surface water bodies. Historically, TSS data collection in the United States has been limited to the location of water treatment plants where state or federal agencies conduct regularly-scheduled water sampling. Mathematical models relating riverine TSS concentration to the explanatory factors have therefore been limited and the relationships between climate extremes and water contamination events have not been effectively diagnosed. This paper presents a method to identify these issues by utilizing a Long Short-Term Memory Network (LSTM) model trained on Moderate Resolution Imaging Spectroradiometer (MODIS) satellite reflectance data, which is calibrated to TSS data collected by the Ohio River Valley Water Sanitation Commission (ORSANCO). The methodology developed enables a thorough empirical analysis and data-driven algorithms able to account for spatial variability within the watershed and provide effective water quality prediction under uncertainty.

Research on the Application of Integrated Learning Models in Oilfield Production Forecasting

Forecasting oil production is crucially important in oilfield management. Currently, multifeature-based modeling methods are widely used, but such modeling methods are not universally applicable due to the different actual conditions of oilfields in different places. In this paper, a time series forecasting method based on an integrated learning model is proposed, which combines the advantages of linearity and nonlinearity and is only concerned with the internal characteristics of the production curve itself, without considering other factors. The method includes processing the production history data using singular spectrum analysis, training the autoregressive integrated moving average model and Prophet, training the wavelet neural network, and forecasting oil production. The method is validated using historical production data from the J oilfield in China from 2011 to 2021, and compared with single models, Arps model, and mainstream time series forecasting models. The results show that in the early prediction, the difference in prediction error between the integrated learning model and other models is not obvious, but in the late prediction, the integrated model still predicts stably and the other models compared with it will show more obvious fluctuations. Therefore, the model in this article can make stable and accurate predictions.

PM2.5 concentration prediction using weighted CEEMDAN and improved LSTM neural network

As the core of pollution prevention and management, accurate PM2.5 concentration prediction is crucial for human survival. However, due to the nonstationarity and nonlinearity of PM2.5 concentration data, the accurate prediction for PM2.5 concentration remains a challenge. In this study, a PM2.5 concentration prediction method using weighted complementary ensemble empirical mode decomposition with adaptive noise (WCEEMDAN) and improved long and short-term memory (ILSTM) neural network is proposed. Firstly, a novel WCEEMDAN method is proposed to correctly identify the non-stationary and non-linear characteristics and divide the PM2.5 sequences into various layers. Through the correlation analysis with PM2.5 data, these sub-layers are given different weights. Secondly, the adaptive mutation particle swarm optimization (AMPSO) algorithm is developed to obtain the main hyperparameters of the long short-term memory network (LSTM) neural network, improving the prediction accuracy of PM2.5 concentration. The optimization convergence speed and accuracy are improved by adjusting the inertia weight and introducing the mutation mechanism to enhance the global optimization ability. Finally, three groups of PM2.5 concentration data are utilized to verify the effectiveness of the proposed model. Compared with other methods, the experimental results demonstrate the superiority of the proposed model. The source code can be downloaded from https://github.com/zhangli190227/WCEENDAM-ILSTM .

Achieving Optimal Paper Properties: A Layered Multiscale kMC and LSTM-ANN-Based Control Approach for Kraft Pulping

The growing demand for various types of paper highlights the importance of optimizing the kraft pulping process to achieve desired paper properties. This work proposes a novel multiscale model to optimize the kraft pulping process and obtain desired paper properties. The model combines mass and energy balance equations with a layered kinetic Monte Carlo (kMC) algorithm to predict the degradation of wood chips, the depolymerization of cellulose, and the spatio-temporal evolution of the Kappa number and cellulose degree of polymerization (DP). A surrogate LSTM-ANN model is trained on data generated from the multiscale model under different operating conditions, dealing with both time-varying and time-invariant inputs, and an LSTM-ANN-based model predictive controller is designed to achieve desired set-point values of the Kappa number and cellulose DP while considering process constraints. The results show that the LSTM-ANN-based controller is able to drive the process to desired set-point values with the use of a computationally faster surrogate model with high accuracy and low offset.

Forecasting metro rail transit passenger flow with multiple-attention deep neural networks and surrounding vehicle detection devices

In the rapid development of public transportation led, the traffic flow prediction has become one of the most crucial issues, especially estimating the number of passengers using the Mass Rapid Transit (MRT) system. In general, predicting the passenger flow of traffic is a time-series problem that requires external information to improve accuracy. Because many MRT passengers take cars or buses to MRT stations, this study used external information from vehicle detection (VD) devices to improve the prediction of passenger flow. This study proposed a deep learning architecture, called a multiple-attention deep neural network (MADNN) model, based on historical MRT passenger flow and the flow from surrounding VD devices that estimates the weights of the vehicle detection devices. The model consists of (1) an MRT attention layer (MRT-AL) that generate hidden features for MRT stations, (2) a surrounding VD (SVD) attention layer (SVD-AL) that generate hidden features for SVD devices, and (3) an MRT-SVD attention layer (MRT-SVD-AL) that generate attention weights for each VD device in an MRT station. The results of the investigation indicated that the MADNN model outperformed the models without multiple-attention mechanisms in predicting the passenger flow of MRT traffic.

Investigating Deep Stock Market Forecasting with Sentiment Analysis

When forecasting financial time series, incorporating relevant sentiment analysis data into the feature space is a common assumption to increase the capacities of the model. In addition, deep learning architectures and state-of-the-art schemes are increasingly used due to their efficiency. This work compares state-of-the-art methods in financial time series forecasting incorporating sentiment analysis. Through an extensive experimental process, 67 different feature setups consisting of stock closing prices and sentiment scores were tested on a variety of different datasets and metrics. In total, 30 state-of-the-art algorithmic schemes were used over two case studies: one comparing methods and one comparing input feature setups. The aggregated results indicate, on the one hand, the prevalence of a proposed method and, on the other, a conditional improvement in model efficiency after the incorporation of sentiment setups in certain forecast time frames.

From data to interpretable models: machine learning for soil moisture forecasting

Soil moisture is critical to agricultural business, ecosystem health, and certain hydrologically driven natural disasters. Monitoring data, though, is prone to instrumental noise, wide ranging extrema, and nonstationary response to rainfall where ground conditions change. Furthermore, existing soil moisture models generally forecast poorly for time periods greater than a few hours. To improve such forecasts, we introduce two data-driven models, the Naive Accumulative Representation (NAR) and the Additive Exponential Accumulative Representation (AEAR). Both of these models are rooted in deterministic, physically based hydrology, and we study their capabilities in forecasting soil moisture over time periods longer than a few hours. Learned model parameters represent the physically based unsaturated hydrological redistribution processes of gravity and suction. We validate our models using soil moisture and rainfall time series data collected from a steep gradient, post-wildfire site in southern California. Data analysis is complicated by rapid landscape change observed in steep, burned hillslopes in response to even small to moderate rain events. The proposed NAR and AEAR models are, in forecasting experiments, shown to be competitive with several established and state-of-the-art baselines. The AEAR model fits the data well for three distinct soil textures at variable depths below the ground surface (5, 15, and 30 cm). Similar robust results are demonstrated in controlled, laboratory-based experiments. Our AEAR model includes readily interpretable hydrologic parameters and provides more accurate forecasts than existing models for time horizons of 10-24 h. Such extended periods of warning for natural disasters, such as floods and landslides, provide actionable knowledge to reduce loss of life and property.

Hierarchical attention network for multivariate time series long-term forecasting

Multivariate time series long-term forecasting has always been the subject of research in various fields such as economics, finance, and traffic. In recent years, attention-based recurrent neural networks (RNNs) have received attention due to their ability of reducing error accumulation. However, the existing attention-based RNNs fail to eliminate the negative influence of irrelevant factors on prediction, and ignore the conflict between exogenous factors and target factor. To tackle these problems, we propose a novel Hierarchical Attention Network (HANet) for multivariate time series long-term forecasting. At first, HANet designs a factor-aware attention network (FAN) and uses it as the first component of the encoder. FAN weakens the negative impact of irrelevant exogenous factors on predictions by assigning small weights to them. Then HANet proposes a multi-modal fusion network (MFN) as the second component of the encoder. MFN employs a specially designed multi-modal fusion gate to adaptively select how much information about the expression of current time come from target and exogenous factors. Experiments on two real-world datasets reveal that HANet not only outperforms state-of-the-art methods, but also provides interpretability for prediction.

TSF-transformer: a time series forecasting model for exhaust gas emission using transformer

Monitoring and prediction of exhaust gas emissions for heavy trucks is a promising way to solve environmental problems. However, the emission data acquisition is time delayed and the pattern of emission is usually irregular, which makes it very difficult to accurately predict the emission state. To deal with these problems, in this paper, we interpret emission prediction as a time series prediction problem and explore a deep learning model, a time-series forecasting Transformer (TSF-Transformer) for exhaust gas emission prediction. The exhaust emission of the heavy truck is not directly predicted, but indirectly predicted by predicting the temperature and pressure changes of the exhaust pipe under the working state of the truck. The basis of our research is based on real-time data feeds from temperature and pressure sensors installed on the exhaust pipe of approximately 12,000 heavy trucks. Therefore, the task of time series forecasting consists of two key stages: monitoring and prediction. The former utilizes the server to receive the data sent by the sensors in real-time, and the latter uses these data as samples for network training and testing. The training of the network throughout the prediction process is done in an unsupervised manner. Also, to visualize the forecast results, we weight the forecast data with the truck trajectories and present them as heatmaps. To the best of our knowledge, this is the first case of using the Transformer as the core component of the prediction model to complete the task of exhaust emissions prediction from heavy trucks. Experiments show that the prediction model outperforms other state-of-the-art methods in prediction accuracy.

Evaluating the Applications of Dendritic Neuron Model with Metaheuristic Optimization Algorithms for Crude-Oil-Production Forecasting

The forecasting and prediction of crude oil are necessary in enabling governments to compile their economic plans. Artificial neural networks (ANN) have been widely used in different forecasting and prediction applications, including in the oil industry. The dendritic neural regression (DNR) model is an ANNs that has showed promising performance in time-series prediction. The DNR has the capability to deal with the nonlinear characteristics of historical data for time-series forecasting applications. However, it faces certain limitations in training and configuring its parameters. To this end, we utilized the power of metaheuristic optimization algorithms to boost the training process and optimize its parameters. A comprehensive evaluation is presented in this study with six MH optimization algorithms used for this purpose: whale optimization algorithm (WOA), particle swarm optimization algorithm (PSO), genetic algorithm (GA), sine-cosine algorithm (SCA), differential evolution (DE), and harmony search algorithm (HS). We used oil-production datasets for historical records of crude oil production from seven real-world oilfields (from Tahe oilfields, in China), provided by a local partner. Extensive evaluation experiments were carried out using several performance measures to study the validity of the DNR with MH optimization methods in time-series applications. The findings of this study have confirmed the applicability of MH with DNR. The applications of MH methods improved the performance of the original DNR. We also concluded that the PSO and WOA achieved the best performance compared with other methods.

Wavelet LSTM for Fault Forecasting in Electrical Power Grids

An electric power distribution utility is responsible for providing energy to consumers in a continuous and stable way. Failures in the electrical power system reduce the reliability indexes of the grid, directly harming its performance. For this reason, there is a need for failure prediction to reestablish power in the shortest possible time. Considering an evaluation of the number of failures over time, this paper proposes performing failure prediction during the first year of the pandemic in Brazil (2020) to verify the feasibility of using time series forecasting models for fault prediction. The long short-term memory (LSTM) model will be evaluated to obtain a forecast result that an electric power utility can use to organize maintenance teams. The wavelet transform has shown itself to be promising in improving the predictive ability of LSTM, making the wavelet LSTM model suitable for the study at hand. The assessments show that the proposed approach has better results regarding the error in prediction and has robustness when statistical analysis is performed.

Fault Prediction Based on Leakage Current in Contaminated Insulators Using Enhanced Time Series Forecasting Models

To improve the monitoring of the electrical power grid, it is necessary to evaluate the influence of contamination in relation to leakage current and its progression to a disruptive discharge. In this paper, insulators were tested in a saline chamber to simulate the increase of salt contamination on their surface. From the time series forecasting of the leakage current, it is possible to evaluate the development of the fault before a flashover occurs. In this paper, for a complete evaluation, the long short-term memory (LSTM), group method of data handling (GMDH), adaptive neuro-fuzzy inference system (ANFIS), bootstrap aggregation (bagging), sequential learning (boosting), random subspace, and stacked generalization (stacking) ensemble learning models are analyzed. From the results of the best structure of the models, the hyperparameters are evaluated and the wavelet transform is used to obtain an enhanced model. The contribution of this paper is related to the improvement of well-established models using the wavelet transform, thus obtaining hybrid models that can be used for several applications. The results showed that using the wavelet transform leads to an improvement in all the used models, especially the wavelet ANFIS model, which had a mean RMSE of 1.58 ×10-3, being the model that had the best result. Furthermore, the results for the standard deviation were 2.18 ×10-19, showing that the model is stable and robust for the application under study. Future work can be performed using other components of the distribution power grid susceptible to contamination because they are installed outdoors.

A neural network based approach to classify VLF signals as rock rupture precursors

The advent of novel technologies revealed that other geophysical signals than those directly related to fault motion could be used to probe the state of deformation of the Earth's crust. Electromagnetic signals belonging to this category have been increasingly investigated in the last decade in association to natural earthquakes and laboratory rock fractures. These studies are hampered by the lack of continuous recordings and a systematic mathematical processing of large data sets. Indeed, electromagnetic signals exhibit characteristic patterns on a specific frequency band (the very low frequency, VLF) that correlate uniquely with the paroxistic rupture of rocks specimens under uniaxial laboratory tests and were also detected in the atmosphere, in association to moderate magnitude earthquakes. The similarity of laboratory and atmospheric VLF offers an unique opportunity to study the relation between VLF and rock deformation on at least two different scales and to enlarge the dataset by combining laboratory and atmospheric data. In this paper we show that the enlarged VLF dataset can be successfully used, with a neural network approach based on LSTM neural networks to investigate the potential of the VLF spectrum in classifying rock rupture precursors both in nature and in the laboratory. The proposed approach lays foundation to the automatic detection of interesting VLF patterns for monitoring deformations in the seismically active Earth's crust.

Reducing Detrimental Communication Failure Impacts in Microgrids by Using Deep Learning Techniques

A Microgrid (MG), like any other smart and interoperable power system, requires device-to-device (D2D) communication structures in order to function effectively. This communication system, however, is not immune to intentional or unintentional failures. This paper discusses the effects of communication link failures on MG control and management and proposes solutions based on enhancing message content to mitigate their detritus impact. In order to achieve this goal, generation and consumption forecasting using deep learning (DL) methods at the next time steps is used. The architecture of an energy management system (EMS) and an energy storage system (ESS) that are able to operate in coordination is introduced and evaluated by simulation tests, which show promising results and illustrate the efficacy of the proposed methods. It is important to mention that, in this paper, three dissimilar topics namely MG control/management, DL-based forecasting, and D2D communication architectures are employed and this combination is proven to be capable of achieving the aforesaid objective.

Deep Learning for Forecasting Electricity Demand in Taiwan

According to the World Energy Investment 2018 report, the global annual investment in renewable energy exceeded USD 200 billion for eight consecutive years until 2017. In this paper, a deep-learning-based time-series prediction method, namely a gated recurrent unit (GRU)-based prediction method, is proposed to predict energy generation in Taiwan. Data on thermal power (coal, oil, and gas power), renewable energy (conventional hydropower, solar power, and wind power), pumped hydropower, and nuclear power generation for 1991 to 2020 were obtained from the Bureau of Energy, Ministry of Economic Affairs, Taiwan, and the Taiwan Power Company. The proposed GRU-based method was compared with six common forecasting methods: autoregressive integrated moving average, exponential smoothing (ETS), Holt–Winters ETS, support vector regression (SVR), whale-optimization-algorithm-based SVR, and long short-term memory. Among the methods compared, the proposed method had the lowest mean absolute percentage error and root mean square error and thus the highest accuracy. Government agencies and power companies in Taiwan can use the predictions of accurate energy forecasting models as references to formulate energy policies and design plans for the development of alternative energy sources.

Forecasting Oil Production Flowrate Based on an Improved Backpropagation High-Order Neural Network with Empirical Mode Decomposition

Developing a forecasting model for oilfield well production plays a significant role in managing mature oilfields as it can help to identify production loss earlier. It is very common that mature fields need more frequent production measurements to detect declining production. This study proposes a machine learning system based on a hybrid empirical mode decomposition backpropagation higher-order neural network (EMD-BP-HONN) for oilfields with less frequent measurement. With the individual well characteristic of stationary and non-stationary data, it creates a unique challenge. By utilizing historical well production measurement as a time series feature and then decomposing it using empirical mode decomposition, it generates a simpler pattern to be learned by the model. In this paper, various algorithms were deployed as a benchmark, and the proposed method was eventually completed to forecast well production. With proper feature engineering, it shows that the proposed method can be a potentially effective method to improve forecasting obtained by the traditional method.

Meta-learning for few-shot time series forecasting

Time series forecasting (TSF) is significant for many applications, therefore the exploration and study for this problem has been proceeding. With the advances of computing power, deep neural networks (DNNs) have shown powerful performance on many machine learning tasks when considerable amounts of data can be used. However, sufficient data may be unavailable in some scenarios, which leads to performance degradation or even not working of DNN-based models. In this paper, we focus on few-shot time series forecasting task and propose to employ meta-learning to alleviate the problems caused by insufficient training data. Therefore, we propose a meta-learning-based prediction mechanism for few-shot time series forecasting task, which mainly consists of meta-training and meta-testing. The meta-training phase uses first-order model-agnostic meta-learning algorithm (MAML) as a core component to conduct cross-task training, and thus our method also inherits the advantages of the MAML, i.e., model-agnostic, in the sense that our method is compatible with any model trained with gradient descent. In the meta-testing phase, the DNN-based models are fine-tuned by the small number of time series data from an unseen task in the meta-training phase. We design two groups of comparison models to validate the effectiveness of our method. The first group, as the baseline models, is trained directly on specific time series dataset from target task. The second group, as comparison models, is trained by our proposed method. Also, we conduct data sensitivity study to validate the robustness of our method. The experimental results indicate the second group models outperform the first in different degrees in terms of prediction accuracy and convergence speed, and our method has strong robustness for forecast horizons and data scales.

Generative Adversarial Network to evaluate quantity of information in financial markets

Nowadays, the information obtainable from the markets are potentially limitless. Economic theory has always supported the possible advantage obtainable from having more information than competitors, however quantifying the advantage that these can give has always been a problem. In particular, in this paper we study the amount of information obtainable from the markets taking into account only the time series of the prices, through the use of a specific Generative Adversarial Network. We consider two types of financial instruments traded on the market, stocks and cryptocurrencies: the first are traded in a market subject to opening and closing hours, whereas cryptocurrencies are traded in a 24/7 market. Our goal is to use this GAN to be able to “convert” the amount of information that the different instruments can have in discriminative and predictive power, useful to improve forecast. Finally, we demonstrate that by using the initial dataset with the 5 most important feature useds by traders, the prices of cryptocurrencies present higher discriminatory and predictive power than stocks, while by adding a feature the situation can be completely reversed.

Modeling and forecasting United States oil production along with the social cost of carbon: conventional and unconventional oil

Purpose

The USA is one of the largest oil producers in the world. For this purpose, the authors model and predict the US conventional and unconventional oil production during the period 2000–2030.

Design/methodology/approach

In this research, the system dynamics (SD) model has been used. In this model, economic, technical, geopolitical, learning-by-doing and environmental (social costs of carbon) issues are considered.

Findings

The results of the simulation, after successfully passing the validation test, show that the US unconventional oil production rate under the optimistic scenario (high oil prices) in 2030 is about 12.62 million barrels/day (mb/day), under the medium oil price scenario is about 11.4 mb/day and under the pessimistic scenario (low oil price) is about 10.18 mb/day. The results of US conventional oil production forecasting under these three scenarios (high, medium and low oil prices) show oil production of 4.62, 4.26 and 3.91 mb/day, respectively.

Originality/value

The contribution of this study is important in several respects: First, by modeling SD that technical, economic, proven reserves and technology factors are considered, this paper models US conventional and unconventional oil production separately. In this modeling, nonlinear relationships and feedback loops are presented to better understand the relationships between variables. Second, given the importance of environmental issues, the modeling of social costs of CO2 emissions per barrel of oil is also presented and considered as a part of oil production costs. Third, conventional and unconventional US oil production by 2030 is forecast separately, the results of this study could help policymakers to develop unconventional oil and plan for energy self-sufficiency.

Development of a Deep Learning-Based Prediction Model for Water Consumption at the Household Level

The importance of efficient water resource supply has been acknowledged, and it is essential to predict short-term water consumption in the future. Recently, it has become possible to obtain data on water consumption at the household level through smart water meters. The pattern of these data is nonlinear due to various factors related to human activities, such as holidays and weather. However, it is difficult to accurately predict household water consumption with a nonlinear pattern with the autoregressive integrated moving average (ARIMA) model, a traditional time series prediction model. Thus, this study used a deep learning-based long short-term memory (LSTM) approach to develop a water consumption prediction model for each customer. The proposed model considers several variables to learn nonlinear water consumption patterns. We developed an ARIMA model and an LSTM model in the training dataset for customers with four different water-use types (detached houses, apartment, restaurant, and elementary school). The performances of the two models were evaluated using a test dataset that was not used for model learning. The LSTM model outperformed the ARIMA model in all households (correlation coefficient: mean 89% and root mean square error: mean 5.60 m3). Therefore, it is expected that the proposed model can predict customer-specific water consumption at the household level depending on the type of use.

Hyperspectral Monitoring Driven by Machine Learning Methods for Grassland Above-Ground Biomass

Above-ground biomass (AGB) is a key indicator for studying grassland productivity and evaluating carbon sequestration capacity; it is also a key area of interest in hyperspectral ecological remote sensing. In this study, we use data from a typical alpine meadow in the Qinghai–Tibet Plateau during the main growing season (July–September), compare the results of various feature selection algorithms to extract an optimal subset of spectral variables, and use machine learning methods and data mining techniques to build an AGB prediction model and realize the optimal inversion of above-ground grassland biomass. The results show that the Lasso and RFE_SVM band filtering machine learning models can effectively select the global optimal feature and improve the prediction effect of the model. The analysis also compares the support vector machine (SVM), least squares regression boosting (LSB), and Gaussian process regression (GPR) AGB inversion models; our findings show that the results of the three models are similar, with the GPR machine learning model achieving the best outcomes. In addition, through the analysis of different data combinations, it is found that the accuracy of AGB inversion can be significantly improved by combining the spectral characteristics with the growing season. Finally, by constructing a machine learning interpretable model to analyze the specific role of features, it was found that the same band plays different roles in different records, and the related results can provide a scientific basis for the research of grassland resource monitoring and estimation.

An Attention-Guided Spatiotemporal Graph Convolutional Network for Sleep Stage Classification

Sleep staging has been widely used as an approach in sleep diagnoses at sleep clinics. Graph neural network (GNN)-based methods have been extensively applied for automatic sleep stage classifications with significant results. However, the existing GNN-based methods rely on a static adjacency matrix to capture the features of the different electroencephalogram (EEG) channels, which cannot grasp the information of each electrode. Meanwhile, these methods ignore the importance of spatiotemporal relations in classifying sleep stages. In this work, we propose a combination of a dynamic and static spatiotemporal graph convolutional network (ST-GCN) with inter-temporal attention blocks to overcome two shortcomings. The proposed method consists of a GCN with a CNN that takes into account the intra-frame dependency of each electrode in the brain region to extract spatial and temporal features separately. In addition, the attention block was used to capture the long-range dependencies between the different electrodes in the brain region, which helps the model to classify the dynamics of each sleep stage more accurately. In our experiments, we used the sleep-EDF and the subgroup III of the ISRUC-SLEEP dataset to compare with the most current methods. The results show that our method performs better in accuracy from 4.6% to 5.3%, in Kappa from 0.06 to 0.07, and in macro-F score from 4.9% to 5.7%. The proposed method has the potential to be an effective tool for improving sleep disorders.

Application of a Deep Learning Network for Joint Prediction of Associated Fluid Production in Unconventional Hydrocarbon Development

Machine learning (ML) approaches have risen in popularity for use in many oil and gas (O&G) applications. Time series-based predictive forecasting of hydrocarbon production using deep learning ML strategies that can generalize temporal or sequence-based information within data is fast gaining traction. The recent emphasis on hydrocarbon production provides opportunities to explore the use of deep learning ML to other facets of O&G development where dynamic, temporal dependencies exist and that also hold implications to production forecasting. This study proposes a combination of supervised and unsupervised ML approaches as part of a framework for the joint prediction of produced water and natural gas volumes associated with oil production from unconventional reservoirs in a time series fashion. The study focuses on the pay zones within the Spraberry and Wolfcamp Formations of the Midland Basin in the U.S. The joint prediction model is based on a deep neural network architecture leveraging long short-term memory (LSTM) layers. Our model has the capability to both reproduce and forecast produced water and natural gas volumes for wells at monthly resolution and has demonstrated 91 percent joint prediction accuracy to held out testing data with little disparity noted in prediction performance between the training and test datasets. Additionally, model predictions replicate water and gas production profiles to wells in the test dataset, even for circumstances that include irregularities in production trends. We apply the model in tandem with an Arps decline model to generate cumulative first and five-year estimates for oil, gas, and water production outlooks at the well and basin-levels. Production outlook totals are influenced by well completion, decline curve, and spatial and reservoir attributes. These types of model-derived outlooks can aid operators in formulating management or remedial solutions for the volumes of fluids expected from unconventional O&G development.

Optimization of Well Control during Gas Flooding Using the Deep-LSTM-Based Proxy Model: A Case Study in the Baoshaceng Reservoir, Tarim, China

Gas flooding has proven to be a promising method of enhanced oil recovery (EOR) for mature water-flooding reservoirs. The determination of optimal well control parameters is an essential step for proper and economic development of underground hydrocarbon resources using gas injection. Generally, the optimization of well control parameters in gas flooding requires the use of compositional numerical simulation for forecasting the production dynamics, which is computationally expensive and time-consuming. This paper proposes the use of a deep long-short-term memory neural network (Deep-LSTM) as a proxy model for a compositional numerical simulator in order to accelerate the optimization speed. The Deep-LSTM model was integrated with the classical covariance matrix adaptive evolutionary (CMA-ES) algorithm to conduct well injection and production optimization in gas flooding. The proposed method was applied in the Baoshaceng reservoir of the Tarim oilfield, and shows comparable accuracy (with an error of less than 3%) but significantly improved efficiency (reduced computational duration of ~90%) against the conventional numerical simulation method.

Liquid State Machine on SpiNNaker for Spatio-Temporal Classification Tasks

Liquid State Machines (LSMs) are computing reservoirs composed of recurrently connected Spiking Neural Networks which have attracted research interest for their modeling capacity of biological structures and as promising pattern recognition tools suitable for their implementation in neuromorphic processors, benefited from the modest use of computing resources in their training process. However, it has been difficult to optimize LSMs for solving complex tasks such as event-based computer vision and few implementations in large-scale neuromorphic processors have been attempted. In this work, we show that offline-trained LSMs implemented in the SpiNNaker neuromorphic processor are able to classify visual events, achieving state-of-the-art performance in the event-based N-MNIST dataset. The training of the readout layer is performed using a recent adaptation of back-propagation-through-time (BPTT) for SNNs, while the internal weights of the reservoir are kept static. Results show that mapping our LSM from a Deep Learning framework to SpiNNaker does not affect the performance of the classification task. Additionally, we show that weight quantization, which substantially reduces the memory footprint of the LSM, has a small impact on its performance.

The Application of Neural Networks to Forecast Radial Jet Drilling Effectiveness

This paper aims to study the applicability of machine-learning algorithms, specifically neural networks, for forecasting the effectiveness of Improved recovery methods. Radial jet drilling is the case operation in this study. Understanding changes in reservoir flow properties and their effect on liquid flow rate is essential to evaluate the radial jet drilling effectiveness. Therefore, liquid flow rate after radial jet drilling is the target variable, while geological and process parameters have been taken as features. The effect of various network parameters on learning quality has been assessed. As a result, conclusions on the applicability of neural networks to evaluate the radial jet drilling potential of wells in various geological conditions of carbonate reservoirs have been made.