Proof of RL viability for trading with data leakage

[e-gluzman]

Hi guys,

Your FinRL project has been very helpful - I have been using the StockTradingEnv to make sure I do not mess up my environment.

However, I am encountering very low performance with RL algorithms. In order to test if the RL models are working properly I have created features that leak data about the future returns in the next 1,2,3,5 days. In theory, this should make the task very easy - if future returns are low, sell. However, the model is not able to learn any strategy other than buy and hold.

To replicate:

• Take the notebook https://github.com/AI4Finance-Foundation/FinRL-Tutorials/blob/master/1-Introduction/Stock_NeurIPS2018_SB3.ipynb
• Set ticker list to single stock
• Disable tech indicators, vix, turbulence and add indicators of type close.pct_change(-1), close.pct_change(-5)
• Run a2c model

Do you know why the standard RL algorithm is failing even when given future information? Could you show a notebook where it is able to outperform a buy and hold strategy on a stock, while using information from the future?

Thank you,
Evgeny.

Contact: gluzman64@gmail.com

[zhumingpassional]

@e-gluzman

The stability of RL algorithms is an important issue. Pls use some tricks such as ensemble strategy, dynamic datasets.

Setting the tick list to single tick is not a good method. you can use multiple ticks, and after training only use the action of the single tick.

We will recruit a research assistant to maintain this project.

[BruceYanghy]

Hi guys,

Your FinRL project has been very helpful - I have been using the StockTradingEnv to make sure I do not mess up my environment.

However, I am encountering very low performance with RL algorithms. In order to test if the RL models are working properly I have created features that leak data about the future returns in the next 1,2,3,5 days. In theory, this should make the task very easy - if future returns are low, sell. However, the model is not able to learn any strategy other than buy and hold.

To replicate:

• Take the notebook https://github.com/AI4Finance-Foundation/FinRL-Tutorials/blob/master/1-Introduction/Stock_NeurIPS2018_SB3.ipynb
• Set ticker list to single stock
• Disable tech indicators, vix, turbulence and add indicators of type close.pct_change(-1), close.pct_change(-5)
• Run a2c model

Do you know why the standard RL algorithm is failing even when given future information? Could you show a notebook where it is able to outperform a buy and hold strategy on a stock, while using information from the future?

Thank you, Evgeny.

Contact: gluzman64@gmail.com

Thank you for bringing up the issue. Currently, the FinRL library is extremely poorly maintained. Rest assured, I will reorganize a team to ensure its proper maintenance.

Best regards,

Bruce Yang