To avoid any conflict with your existing Python setup, it is suggested to work in a virtual environment with virtualenv
. To install virtualenv
:
pip install --upgrade virtualenv
Create a virtual environment, activate it and install the requirements in requirements.txt
.
virtualenv venv
source venv/bin/activate
pip install -r requirements.txt
In this work, we experiment with few datasets such as PointMass environment, 2-D Navigation environment and Metaworld. PointMass environment can be used upon installation of garage. 2-D Navigation environment does not require any installation, and can be used as is. Metaworld environment has been used as prescribed in their paper:
@inproceedings{yu2019meta,
title={Meta-World: A Benchmark and Evaluation for Multi-Task and Meta Reinforcement Learning},
author={Tianhe Yu and Deirdre Quillen and Zhanpeng He and Ryan Julian and Karol Hausman and Chelsea Finn and Sergey Levine},
booktitle={Conference on Robot Learning (CoRL)},
year={2019}
eprint={1910.10897},
archivePrefix={arXiv},
primaryClass={cs.LG}
url={https://arxiv.org/abs/1910.10897}
}
Note that, as long as the packages from the requirements are installed, you could run all of our experiments without any additional hassle!
We have carefully organized our codes under scripts
.
The TE-PPO model can be trained on all seeds in a parallel fashion as follows:
sbatch scripts/train_te_ppo_<dataset>.sh
Similarly, our ATE-PPO models can be tested on a fixed set of tasks in a parallel fashion as follows:
sbatch scripts/train_ate_ppo_<dataset>.sh
All our codes were run with 1 GPUs (TitanRTX or A100). Furthermore, all our codes have been optimized to be run on less than 30Gb RAM, including our experiments on Meta-World!
If you find our codes useful, do consider citing our paper:
@misc{kumar2022rethinking,
title={Rethinking Learning Dynamics in RL using Adversarial Networks},
author={Ramnath Kumar and Tristan Deleu and Yoshua Bengio},
year={2022},
eprint={2201.11783},
archivePrefix={arXiv},
primaryClass={cs.LG}
}
Our repository makes use of various open-source codes. Most of which have been documented at Garage If you find the respective codes useful, do cite their respective papers as well:
@misc{garage,
author = {The garage contributors},
title = {Garage: A toolkit for reproducible reinforcement learning research},
year = {2019},
publisher = {GitHub},
journal = {GitHub repository},
howpublished = {\url{https://github.com/rlworkgroup/garage}},
commit = {be070842071f736eb24f28e4b902a9f144f5c97b}
}