This project is an implementation of a deep reinforcement learning model that uses a general neural network architecture to train an agent to play Atari 2600 games.
The model uses a Deep Q-Network (DQN) that consists of two convolutional layers and two fully connected layers. The model uses the a Double DQN and Prioritized Experience Replay for training. If you are interested in training the model on an Atari game, please refer to the Setup section to get started.
If you want to create your own DRL model, I recommend that you check out the Acknowledgements section for some resources to get you pointed in the right direction.
Clone this repository and change to the project directory to get started.
Go to the roms directory in the project folder and extract the files from the Roms.rar file to get the Atari ROMs. The .rar file was obtained from here.
$ cd Atari-DRL
$ python -m venv venv
$ venv\Scripts\activate
$ pip install ipykernel
$ ipython kernel install --user --name=atari
$ pip install -r requirements.txt
$ python -m atari_py.import_roms roms/
$ jupyter-notebook
Open the sample_code.ipynb file to get started. Make sure to switch the kernel to atari by going to Kernel > Change kernel > atari on the jupyter notebook menu bar.
-
DEEPLIZARD: This website gives a good introduction to Q-learning and Deep Reinforcement Learning. This implementation of a DRL agent was a heavily adapted version of the tutorial offered by this website.
-
jcwleo: My implementation of prioritized experience replay was based off this source code.
-
Human Level Control Through Deep Reinforcement Learning: This paper was the first to present a deep learning model that successfully trained a DRL agent to play Atari games. It provides a good starting point for implementing a DRL model and offers some data preprocessing techniques.
-
Deep Reinforcement Learning with Double Q-learning: This paper demonstrates the use of a Double Deep Q-Network and provides a mathematical formula for getting the reward subject to the Double Q-learning algorithm.
-
Prioritized Experience Replay: This paper discusses the concept of using error to prioritize which replay experiences are sampled, creating the notion of prioritized experience replay. This paper provides some methods for implementation.