Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Mixed double precision for PPO_RNN algorithm #172

Open
wants to merge 2 commits into
base: develop
Choose a base branch
from

Conversation

lopatovsky
Copy link
Contributor

@lopatovsky lopatovsky commented Jul 15, 2024

Mixed precision

Motivation:

Inspired by RLGames, we implemented automatic mixed double precision to boost performance of PPO_RNN especially for big models.

Sources:

https://pytorch.org/docs/stable/amp.html

https://pytorch.org/docs/stable/notes/amp_examples.html

Speed eval:

  • model with one layer of lstm (hidden size: 768, seq_len 128) followed by mlp units: [2048, 1024, 1024, 512]
  • trained with isaac-sim simulation (so the speed up on skrl side is actually higher than what this test shows)
Mixed-Precision Time (s) Speed Factor
No 155 1x
Yes 105 0.677x

Quality eval:

  • We trained a policy for our task with each of the configurations multiple times. We didn’t observe any statistically significant difference in quality of the final results.

@lopatovsky lopatovsky changed the base branch from main to develop July 15, 2024 12:50
@lopatovsky lopatovsky changed the title Mixed double precision for PPO algorithm Mixed double precision for PPO_RNN algorithm Jul 15, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants