[Вопрос] how to train 2 models in parallel?

[kozlolet]

❓ Question

I made 2 models that will compete on 1 stadium in unity, and in order for them to compete I need them to train together at the same time. each model has its own environment each of them is connected to this stadium unity so that if 1 model wins, it gets an award, the same with the 2nd model. So I want to test my work with this code.

episodes = 5
for episode in range(1, episodes+1):
    v_state = v_env.reset()
    i_state = i_env.reset()
    done = False
    v_score = 0
    i_score = 0 
    
    while not done:
        i_action = i_env.action_space.sample()
        v_action = v_env.action_space.sample()

        i_n_state, i_reward, done, _, i_info = i_env.step(i_action)
        v_n_state, v_reward, done, _, v_info = v_env.step(v_action)

        i_score+=i_reward
        v_score+=v_reward

    print('Episode:{} | i_score:{}, v_score:{}'.format(episode, i_score, v_score))

It's designed so that each model takes a random action step by step. And so after the launch, both models started running and moving around the stadium.

main point

And finally after all this, I wondered: How do you train these models?

Because if you just write:

i_model = PPO('MultiInputPolicy', i_env, verbose=1)
v_model = PPO('MultiInputPolicy', v_env, verbose=1)

i_model.learn(total_timesteps=1000)
v_model.learn(total_timesteps=1000)

then model 1 will be trained first, and then model 2.

I tried to find a way to train these models step-by-step but found no results

Checklist

• I have checked that there is no similar issue in the repo
• I have read the documentation
• If code there is, it is minimal and working
• If code there is, it is formatted using the markdown code blocks for both code and stack traces.

[araffin]

Duplicate of #957 (comment) and others like #1094