Decrease gamma for SAC/TQC on Swimmer-v4 (#447)

* Decrease swimmer gamma some experiments indicate that gamma=0.999 is better for swimmer, in agreement with https://arxiv.org/pdf/2208.07587 Fig. 4 * Update TQC Swimmer and changelog --------- Co-authored-by: Antonin Raffin <antonin.raffin@ensta.org>
DLR-RM · May 3, 2024 · 27e081e · 27e081e
1 parent e06914e
commit 27e081e
Show file tree

Hide file tree

Showing 4 changed files with 15 additions and 8 deletions.
diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -1,17 +1,24 @@
-## Release 2.3.0 (2024-03-31)
+## Release 2.4.0a0 (WIP)
 
 ### Breaking Changes
-- Updated defaults hyperparameters for TD3/DDPG to be more consistent with SAC
-- Upgraded MuJoCo envs hyperparameters to v4 (pre-trained agents need to be updated)
-- Upgraded to SB3 >= 2.3.0
+- Updated defaults hyperparameters for TQC/SAC for Swimmer-v4 (decrease gamma for more consistent results) (@JacobHA) [W&B report](https://wandb.ai/openrlbenchmark/sbx/reports/SAC-MuJoCo-Swimmer-v4--Vmlldzo3NzM5OTk2)
 
 ### New Features
 
-
 ### Bug fixes
 
 ### Documentation
 
+### Other
+
+
+## Release 2.3.0 (2024-03-31)
+
+### Breaking Changes
+- Updated defaults hyperparameters for TD3/DDPG to be more consistent with SAC
+- Upgraded MuJoCo envs hyperparameters to v4 (pre-trained agents need to be updated)
+- Upgraded to SB3 >= 2.3.0
+
 ### Other
 - Added test dependencies to `setup.py` (@power-edge)
 - Simplify dependencies of `requirements.txt` (remove duplicates from `setup.py`)

diff --git a/hyperparams/sac.yml b/hyperparams/sac.yml
@@ -209,7 +209,7 @@ Humanoid-v4:
 
 Swimmer-v4:
   <<: *mujoco-defaults
-  gamma: 0.9999
+  gamma: 0.999
 
 # === HER Robotics GoalEnvs ===
 

diff --git a/hyperparams/tqc.yml b/hyperparams/tqc.yml
@@ -161,7 +161,7 @@ Humanoid-v4:
 
 Swimmer-v4:
   <<: *mujoco-defaults
-  gamma: 0.9999
+  gamma: 0.999
 
 # === HER Robotics GoalEnvs ===
 FetchReach-v1:

diff --git a/rl_zoo3/version.txt b/rl_zoo3/version.txt
@@ -1 +1 @@
-2.3.0
+2.4.0a0