Questions and Suggestions Regarding SKRL Library Implementation and Performance

[umfundii]

Hi @Toni-SM,

Thanks for this amazing library. I have a few questions/suggestions.
I've read the documentation, examples, and discussions, but I still have a few questions outlined below:

1. After training, how do I load the trained policy to visualize its performance on the environment? For example, after running "jax_gymnasium_cartpole_cem.py," how do I load the saved model/policy and visualize its performance on the environment? I noticed that all the examples focus on training and evaluation but not on loading. Could you provide an example for the entire procedure: training, evaluating, and loading/deploying in a simulator for a simple environment like Gymnasium's "CartPole-v1"? Best practices to get the most out of SKRL would be appreciated especially for new users like myself.

2. Second question, what is the performance of the JAX/Torch implementation? Are there any benefits to using one over the other?

Thanks.

[Toni-SM]

Hi @umfundii

1. In the Loading checkpoints section of the skrl documentation you can find details about how to load saved models/policies.
   Furthermore, you can find some code for loading the saved model/policy in the Isaac Gym examples (this code can be used for other wrapped environments):

   skrl/docs/source/examples/isaacgym/jax_cartpole_ppo.py

   Lines 124 to 135 in 631613a

   	# # ---------------------------------------------------------
   	# # comment the code above: `trainer.train()`, and...
   	# # uncomment the following lines to evaluate a trained agent
   	# # ---------------------------------------------------------
   	# from skrl.utils.huggingface import download_model_from_huggingface
   	# # download the trained agent's checkpoint from Hugging Face Hub and load it
   	# path = download_model_from_huggingface("skrl/IsaacGymEnvs-Cartpole-PPO", filename="agent.pickle")
   	# agent.load(path)
   	# # start evaluation
   	# trainer.eval()

2. The following chart shows the performance of the JAX/Torch implementation for the Isaac Gym Environments (higher is better).
   Please note that the data used to generate this graphic was collected several months ago (I don't have data to compare the latest version of the involved ML frameworks). Also, since Isaac Sim environments are implemented in PyTorch, for the JAX version there is an extra (although minimal in performance metric) conversion execution (from JAX to PyTorch and viceversa) on each training step.