[Question] Question about VecEnv using a custom environment in SB3 #1968
Comments
|
Yes, from stable_baselines3.common.env_util import make_vec_env
vec_env = make_vec_env(CustomEnv, n_envs=4)is the way to go. If you want to use from stable_baselines3.common.vec_env import SubprocVecEnv
vec_env = make_vec_env(CustomEnv, n_envs=4, vec_env_cls=SubprocVecEnv)answer in the doc: Multiprocessing Unleashing the Power of Vectorized Environments EDIT: pass the env class instead of the env. |
|
Well, there is a small problem in the above code since 'env' is not callable. However, even following the tutorial it does not work for me and I traced the problem to However, if I do the same in my main code it looks as expected: I do this under Windows using stable-baseline3 v2.3.2 (which uses 'spawn' as start_method). Thanx |
|
Wait, |
|
I'm not sure to understand your issue.
Why is it a problem? |
Indeed, you are right. I didn't spot that in your code :-( But does the following line of code work for you ?
|
|
With a full example it may be clearer (above comment edited): import gymnasium as gym
from gymnasium import spaces
from stable_baselines3.common.env_util import make_vec_env
from stable_baselines3.common.vec_env import SubprocVecEnv
class CustomEnv(gym.Env):
"""Custom Environment that follows gym interface."""
def __init__(self):
super().__init__()
self.action_space = spaces.Discrete(2)
self.observation_space = spaces.Box(low=-1, high=1, shape=(2,))
def step(self, action):
return self.observation_space.sample(), 0.0, False, False, {}
def reset(self, seed=None, options=None):
return self.observation_space.sample(), {}
if __name__ == "__main__":
vec_env = make_vec_env(CustomEnv, n_envs=4, vec_env_cls=SubprocVecEnv) |
|
When I run your code I get the following error: Process SpawnProcess-1: During handling of the above exception, another exception occurred: Traceback (most recent call last): Process finished with exit code 1 |
|
Please provide your system info (see bug issue template) |
|
sb3.get_system_info() gives:
|
This example does not work. |
No we don't. gym env aren't meant to be callable |
Can you double-check that you've actually ran the example that I gave? What the following outputs?: import gymnasium as gym
from gymnasium import spaces
from stable_baselines3.common.env_util import make_vec_env
from stable_baselines3.common.vec_env import SubprocVecEnv
class CustomEnv(gym.Env):
"""Custom Environment that follows gym interface."""
def __init__(self):
super().__init__()
self.action_space = spaces.Discrete(2)
self.observation_space = spaces.Box(low=-1, high=1, shape=(2,))
def step(self, action):
return self.observation_space.sample(), 0.0, False, False, {}
def reset(self, seed=None, options=None):
return self.observation_space.sample(), {}
if __name__ == "__main__":
env = CustomEnv()
print(isinstance(env, gym.Env)) |
That was surely your example! However, as I mentioned in my first post "isinstance(env, gym.Env)" inside _patch_env() is then False. |
|
To summarize, this is what you're encountering?: import gymnasium as gym
from gymnasium import spaces
from stable_baselines3.common.env_util import _patch_env
class CustomEnv(gym.Env):
def __init__(self):
super().__init__()
self.action_space = spaces.Discrete(2)
self.observation_space = spaces.Box(low=-1, high=1, shape=(2,))
def step(self, action):
return self.observation_space.sample(), 0.0, False, False, {}
def reset(self, seed=None, options=None):
return self.observation_space.sample(), {}
if __name__ == "__main__":
env = CustomEnv()
print(isinstance(env, gym.Env)) # True
_patch_env(env) # Fails The environment is of type <class 'main.CustomEnv'>, not a Gymnasium ... |
|
Do you also have the issue with make_vec_env(CustomEnv, n_envs=4) |
|
The direct call to _patch_env(env) does NOT give an error, only when it happens inside make_vec_env(). make_vec_env() with DummyVecEnv works fine. P.S. I have to go now. Will check back tomorrow. |
|
From my side: without the By the way the function: returns True |
|
@wilhem @AxKo It's getting pretty complicated to figure out who's getting what error. So to sum up, the problem is that under Windows, you get: from gymnasium import spaces
import gymnasium as gym
from stable_baselines3.common.env_util import _patch_env, make_vec_env, SubprocVecEnv
class CustomEnv(gym.Env):
def __init__(self):
super().__init__()
self.action_space = spaces.Discrete(2)
self.observation_space = spaces.Box(low=-1, high=1, shape=(2,))
def step(self, action):
return self.observation_space.sample(), 0.0, False, False, {}
def reset(self, seed=None, options=None):
return self.observation_space.sample(), {}
if __name__ == "__main__":
print(isinstance(CustomEnv(), gym.Env)) # True
_patch_env(CustomEnv()) # OK
make_vec_env(CustomEnv, n_envs=4) # OK
make_vec_env(CustomEnv, n_envs=4, vec_env_cls=SubprocVecEnv) # FailsTraceback: System info: OS: Windows-10-10.0.22631-SP0 10.0.22631 Can you confirm? From now on, if your comment doesn't relate to this issue precisely, please open a new issue. |
qgallouedec
added
bug
|
For some unknown reasons it works, if the environment is registered, like here: Please try that and remove the Source: link About my system: |
|
So it's not Windows only but Ubuntu as well? |
|
@qgallouedec: Yes, your above summary is correct for me. In addition, using a registered using a registered environment also works for me: However, registration does NOT work for me: P.S. can you actually reproduce my problem under Windows?? |
|
Stupid question: is the class |
|
The same file! |
|
Can you post your code entirely removing the unnecessary parts? |
|
What happens if you call the class and not only the name of the class? |
|
It makes no difference if I use CustomEnv or CustomEnv(). |
|
Can you put the |
|
That WORKED !! I only get this warning about render_mode (which is an unrelated and minor issue): Thanks ! |
|
I don't know... but I think that SB3 is extremely picky compared to other frameworks |
|
Closing as the original question has been answered here: #1968 (comment) |
I also have problems about render_mode Did you solve it? |
|
this comes from Gymnasium not sb3, please have a look at their doc |
In your custom environment put the following line: |
❓ Question
I have a custom environment (inherited from Gymnasium and yes
check_envruns without any errors or warnings) and now I'm trying to migrate it to a vectorized environment.My question is: since the official documentation shows an example using a standard CartPole-v1 environment and not really a custom class or function, how do I adapt my environment class to be vectorized?
A minimal example is:
The question now is: if I do a
print(type(vec_env))the output is a
DummyVecEnvIs it correct or I'm doing something wrong?
Checklist
The text was updated successfully, but these errors were encountered: