DQN with Network Modification

Propose fully convolutional network with skip connection which is deeper than the network used in vanilla DQN. Apply the network to DQN, DDQN and Deuling-DDQN. Compare the performance with vanilla algorithms in 6 Atari games. And introduce 3 useful debuging method.

Network

With skip connection, the network can catch the details (eg. bullets, balls). Although layers of the network is increasing, the parameters are still smaller than the previous network (With inputs of the Atari games, the number of parameters in new network is 138,020 and in previous network is 211,620).

6 Atari Games

Open AI gym Atari

• Breakout
• SpaceInvaders
• Pong
• Enduro
• Atlantis
• BeamRider

Result

Please check here - Experiment Report (Korean)

Breakout

Duration : 1d 19h 33m 39s

SpaceInvaders

Duration : 1d 4h 41m 19s

Pong

Duration : 2d 10h 7m 49s

BeamRider

Duration : 3d 4h 18m 7s

Enduro

Duration :

• Proposed : 3d 3h 7m 23s
• Vanilla : 2d 20h 14s

Atlantis

Duration :

• Proposed : 4d 48m 41s
• Vanilla : 1d 6h 9m 40s

Useful Debuging Method

• Check Input Frames
• Check Saliency Map (Grad-CAM)
• Check Log and Plot

Usage

Training

python ./DQN/train.py

Test

python ./DQN/test.py

requirements

tensorflow==2.2.0
scikit-learn==0.23.2
matplotlib==3.3.3
gym[atari]
opencv-python