Why I can't converge the network when I use your default setting(terminal: python run_atari_dqn.py)?

[nizhihao]

Hello, first of all thanks for sharing your amazing job. I want to know Why I can't converge the network when I use your default setting(terminal: python run_atari_dqn.py)? the average 100 reward value always in about -20.9, which can't increase as the time-step increase. And Can you share your code about how to draw good pictures like you folder ./imgs/ pictures. Thank your very much!!

[BY571]

You are running PongNoFrameskip-v4? and for how long did you let it train?

I have another repository for how to do the plottings: https://github.com/BY571/Tensorboard-Plotting

[nizhihao]

Yeah, it's the PongNoFrameskip-v4, I use the default parameters, because of I use python==2.7 and torch==1.0.0, I only modify the function random.choices() in dqn_agent.py. And I have train the code with default frame==5e6. The average100 reward graph I get from tensorboard is put up:

[BY571]

I checked yesterday and noticed the same. for CartPole-v0 and LunarLander-v2 it converges nicely. But as you noticed for Pong it was struggling. I'll have a look later might be something with the wrappers of the environment. Once I found out whats the problem I come back to you :)

[nizhihao]

Em, Ok , Thanks！Let me try the CartPole-v0 and LunarLander-v2 environment firstly, and then try others. thanks for your answer :)

[BY571]

how did your runs work out for cartpole and lunarlander? sry i was quite busy and couldn't dig deeper in the problem yet