Reservoir Production Prediction Model Based on a Stacked LSTM Network and Transfer Learning

Long short-term memory models of water quality in inland water environments

Water quality is substantially influenced by a multitude of dynamic and interrelated variables, including climate conditions, landuse and seasonal changes. Deep learning models have demonstrated predictive power of water quality due to the superior ability to automatically learn complex patterns and relationships from variables. Long short-term memory (LSTM), one of deep learning models for water quality prediction, is a type of recurrent neural network that can account for longer-term traits of time-dependent data. It is the most widely applied network used to predict the time series of water quality variables. First, we reviewed applications of a standalone LSTM and discussed its calculation time, prediction accuracy, and good robustness with process-driven numerical models and the other machine learning. This review was expanded into the LSTM model with data pre-processing techniques, including the Complete Ensemble Empirical Mode Decomposition with Adaptive Noise method and Synchrosqueezed Wavelet Transform. The review then focused on the coupling of LSTM with a convolutional neural network, attention network, and transfer learning. The coupled networks demonstrated their performance over the standalone LSTM model. We also emphasized the influence of the static variables in the model and used the transformation method on the dataset. Outlook and further challenges were addressed. The outlook for research and application of LSTM in hydrology concludes the review.

An Advanced Long Short-Term Memory (LSTM) Neural Network Method for Predicting Rate of Penetration (ROP)

Rate of penetration (ROP) is an essential factor in drilling optimization and reducing the drilling cycle. Most of the traditional ROP prediction methods are based on building physical model and single intelligent algorithms, and the efficiency and accuracy of these prediction methods are very low. With the development of artificial intelligence, high-performance algorithms make reliable prediction possible from the data perspective. To improve ROP prediction efficiency and accuracy, this paper presents a method based on particle swarm algorithm for optimization of long short-term memory (LSTM) neural networks. In this paper, we consider the Tuha Shengbei block oilfield as an example. First, the Pearson correlation coefficient is used to measure the correlation between the characteristics and eight parameters are screened out, namely, the depth of the well, gamma, formation density, pore pressure, well diameter, drilling time, displacement, and drilling fluid density. Second, the PSO algorithm is employed to optimize the super-parameters in the construction of the LSTM model to the predict ROP. Third, we assessed model performance using the determination coefficient (R ²), root mean square error (RMSE), and mean absolute percentage error (MAPE). The evaluation results show that the optimized LSTM model achieves an R ² of 0.978 and RMSE and MAPE are 0.287 and 12.862, respectively, hence overperforming the existing methods. The average accuracy of the optimized LSTM model is also improved by 44.2%, indicating that the prediction accuracy of the optimized model is higher. This proposed method can help to drill engineers and decision makers to better plan the drilling operation scheme and reduce the drilling cycle.

Prediction of Oil Saturation during Water and Gas Injection Using Controllable Convolutional Long Short-Term Memory

Oil saturation is a kind of spatiotemporal sequence that changes dynamically with time, and it is affected not only by the reservoir properties, but also by the injection–production parameters. When predicting oil saturation during water and gas injection, the influence of time, space and injection–production parameters should be considered. Aiming at this issue, a prediction method based on a controllable convolutional long short-term memory network (Ctrl-CLSTM) is proposed in this paper. The Ctrl-CLSTM is an unsupervised learning model whose input is the previous spatiotemporal sequence together with the controllable factors of corresponding moments, and the output is the sequence to be predicted. In this way, future oil saturation can be generated from the historical context. Concretely, the convolution operation is embedded into each unit to describe the interaction between temporal features and spatial structures of oil saturation, thus the Ctrl-CLSTM realizes the unified modeling of the spatiotemporal features of oil saturation. In addition, a novel control gate structure is introduced in each Ctrl-CLSTM unit to take the injection–production parameters as controllable influencing factors and establish the nonlinear relationship between oil saturation and injection–production parameters according to the coordinates of each well location. Therefore, different oil saturation prediction results can be obtained by changing the injection–production parameters. Finally, experiments on real oilfields show that the Ctrl-CLSTM comprehensively considers the influence of artificial controllable factors such as injection–production parameters, accomplishes accurate prediction of oil saturation with a structure similarity of more than 98% and is more time efficient than reservoir numerical simulation.

Research on an Optimization Method for Injection-Production Parameters Based on an Improved Particle Swarm Optimization Algorithm

The optimization of injection–production parameters is an important step in the design of gas injection development schemes, but there are many influencing factors and they are difficult to determine. To solve this problem, this paper optimizes injection-production parameters by combining an improved particle swarm optimization algorithm to study the relationship between injection-production parameters and the net present value. In the process of injection-production parameter optimization, the particle swarm optimization algorithm has shortcomings, such as being prone to fall into local extreme points and slow in convergence speed. Curve adaptive and simulated annealing particle swarm optimization algorithms are proposed to further improve the optimization ability of the particle swarm optimization algorithm. Taking the Tarim oil field as an example, in different stages, the production time, injection volume and flowing bottom hole pressure were used as input variables, and the optimal net present value was taken as the goal. The injection-production parameters were optimized by improving the particle swarm optimization algorithm. Compared with the particle swarm algorithm, the net present value of the improved scheme was increased by about 3.3%.

Dense Residual LSTM-Attention Network for Boiler Steam Temperature Prediction with Uncertainty Analysis

Flexible operation of large-scale boilers for electricity generation is essential in modern power systems. An accurate prediction of boiler steam temperature is of great importance to the operational efficiency of boiler units to prevent the occurrence of overtemperature. In this study, a dense, residual long short-term memory network (LSTM)-attention model is proposed for steam temperature prediction. In particular, the residual elements in the proposed model have a great advantage in improving the accuracy by adding short skip connections between layers. To provide overall information for the steam temperature prediction, uncertainty analysis based on the proposed model is performed to quantify the uncertainties in steam temperature variations. Our results demonstrate that the proposed method exhibits great performance in steam temperature prediction with a mean absolute error (MAE) of less than 0.6 °C. Compared to algorithms such as support-vector regression (SVR), ridge regression (RIDGE), the recurrent neural network (RNN), the gated recurrent unit (GRU), and LSTM, the prediction accuracy of the proposed model outperforms by 32, 16, 12, 10, and 11% in terms of MAE, respectively. According to our analysis, the dense residual LSTM-attention model is shown to provide an accurate early warning of overtemperature, enabling the development of real-time steam temperature control.