Adaptive stock trading with dynamic asset allocation using reinforcement learning

Multi-objective reward generalization: improving performance of Deep Reinforcement Learning for applications in single-asset trading

We investigate the potential of Multi-Objective, Deep Reinforcement Learning for stock and cryptocurrency single-asset trading: in particular, we consider a Multi-Objective algorithm which generalizes the reward functions and discount factor (i.e., these components are not specified a priori, but incorporated in the learning process). Firstly, using several important assets (BTCUSD, ETHUSDT, XRPUSDT, AAPL, SPY, NIFTY50), we verify the reward generalization property of the proposed Multi-Objective algorithm, and provide preliminary statistical evidence showing increased predictive stability over the corresponding Single-Objective strategy. Secondly, we show that the Multi-Objective algorithm has a clear edge over the corresponding Single-Objective strategy when the reward mechanism is sparse (i.e., when non-null feedback is infrequent over time). Finally, we discuss the generalization properties with respect to the discount factor. The entirety of our code is provided in open-source format.

Reinforcement Learning for Quantitative Trading

Quantitative trading (QT), which refers to the usage of mathematical models and data-driven techniques in analyzing the financial market, has been a popular topic in both academia and financial industry since 1970s. In the last decade, reinforcement learning (RL) has garnered significant interest in many domains such as robotics and video games, owing to its outstanding ability on solving complex sequential decision making problems. RL’s impact is pervasive, recently demonstrating its ability to conquer many challenging QT tasks. It is a flourishing research direction to explore RL techniques’ potential on QT tasks. This paper aims at providing a comprehensive survey of research efforts on RL-based methods for QT tasks. More concretely, we devise a taxonomy of RL-based QT models, along with a comprehensive summary of the state of the art. Finally, we discuss current challenges and propose future research directions in this exciting field.

ProtInteract: A deep learning framework for predicting protein-protein interactions

Proteins mainly perform their functions by interacting with other proteins. Protein-protein interactions underpin various biological activities such as metabolic cycles, signal transduction, and immune response. However, due to the sheer number of proteins, experimental methods for finding interacting and non-interacting protein pairs are time-consuming and costly. We therefore developed the ProtInteract framework to predict protein-protein interaction. ProtInteract comprises two components: first, a novel autoencoder architecture that encodes each protein's primary structure to a lower-dimensional vector while preserving its underlying sequence attributes. This leads to faster training of the second network, a deep convolutional neural network (CNN) that receives encoded proteins and predicts their interaction under three different scenarios. In each scenario, the deep CNN predicts the class of a given encoded protein pair. Each class indicates different ranges of confidence scores corresponding to the probability of whether a predicted interaction occurs or not. The proposed framework features significantly low computational complexity and relatively fast response. The contributions of this work are twofold. First, ProtInteract assimilates the protein's primary structure into a pseudo-time series. Therefore, we leverage the nature of the time series of proteins and their physicochemical properties to encode a protein's amino acid sequence into a lower-dimensional vector space. This approach enables extracting highly informative sequence attributes while reducing computational complexity. Second, the ProtInteract framework utilises this information to identify protein interactions with other proteins based on its amino acid configuration. Our results suggest that the proposed framework performs with high accuracy and efficiency in predicting protein-protein interactions.

Development of a stock trading system based on a neural network using highly volatile stock price patterns

This paper proposes a pattern-based stock trading system using ANN-based deep learning and utilizing the results to analyze and forecast highly volatile stock price patterns. Three highly volatile price patterns containing at least a record of the price hitting the daily ceiling in the recent trading days are defined. The implications of each pattern are briefly analyzed using chart examples. The training of the neural network was conducted with stock data filtered in three patterns and trading signals were generated using the prediction results of those neural networks. Using data from the KOSPI and KOSDAQ markets, It was found that that the proposed pattern-based trading system can achieve better trading performances than domestic and overseas stock indices. The significance of this study is the development of a stock price prediction model that exceeds the market index to help overcome the continued freezing of interest rates in Korea. Also, the results of this study can help investors who fail to invest in stocks due to the information gap.

Innovative deep matching algorithm for stock portfolio selection using deep stock profiles

Construction of a reliable stock portfolio remains an open issue in quantitative investment. Multiple machine learning models have been trained for stock portfolio selection, but their practical applicability remains limited due to the challenges posed by the characteristic of a low signal-to-noise ratio (SNR), the nature of time-series data, and non-independent identical distribution in financial data. Here, we transformed the stock selection task into a matching problem between a group of stocks and a stock selection target. We proposed a novel representation algorithm of stock selection target and a novel deep matching algorithm (TS-Deep-LtM). Then we proposed a deep stock profiling method to extract the optimal feature combination and trained a deep matching model based on TS-Deep-LtM algorithm for stock portfolio selection. Especially, TS-Deep-LtM algorithm was obtained by setting statistical indicators to filter and integrate three deep text matching algorithms. This parallel framework design made it good at capturing signals from time-series data and adapting to non-independent identically distributed data. Finally, we applied the proposed model to stock selection and tested long-only portfolio strategies from 2010 to 2017. We demonstrated that the risk-adjusted returns obtained by our portfolio strategies outperformed those obtained by the CSI300 index and learning-to-rank approaches during the same period.

Reinforcement Learning in Financial Markets

Recently there has been an exponential increase in the use of artificial intelligence for trading in financial markets such as stock and forex. Reinforcement learning has become of particular interest to financial traders ever since the program AlphaGo defeated the strongest human contemporary Go board game player Lee Sedol in 2016. We systematically reviewed all recent stock/forex prediction or trading articles that used reinforcement learning as their primary machine learning method. All reviewed articles had some unrealistic assumptions such as no transaction costs, no liquidity issues and no bid or ask spread issues. Transaction costs had significant impacts on the profitability of the reinforcement learning algorithms compared with the baseline algorithms tested. Despite showing statistically significant profitability when reinforcement learning was used in comparison with baseline models in many studies, some showed no meaningful level of profitability, in particular with large changes in the price pattern between the system training and testing data. Furthermore, few performance comparisons between reinforcement learning and other sophisticated machine/deep learning models were provided. The impact of transaction costs, including the bid/ask spread on profitability has also been assessed. In conclusion, reinforcement learning in stock/forex trading is still in its early development and further research is needed to make it a reliable method in this domain.