Towards a 3D BabyAI-like environment, starting with PickUp

[Near32]

Description

Following the same motivations as the BabyAI benchmark, this pull request aims to emulate the BabyAI's Pickup environment, but this time using a 3D environment.

Why is it important to propose a 3D BabyAI-like environment?
[Hill et al., 2020]'s work identified the egocentric viewpoint within a 3D environment as an important factor impacting trained agent's systematicity (cf. Section 4.3 and 4.4).
Given that the BabyAI benchmark is a prominent systematicity benchmark, it would be interesting to be able to verify that insight in an apple-to-apple comparison, whcih was not possible until now since there exists no environment like a 3D BabyAI.
This pull request proposes to start to fix that gap.

Dependencies have not changed.

Type of change

Please delete options that are not relevant.

• New feature (non-breaking change which adds functionality)
• This change requires a documentation update

Screenshots

Please attach before and after screenshots of the change if applicable.

Checklist:

• I have run the pre-commit checks with pre-commit run --all-files (see CONTRIBUTING.md instructions to set it up)
• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation
• My changes generate no new warnings
• I have added tests that prove my fix is effective or that my feature works
• New and existing unit tests pass locally with my changes

[pseudo-rnd-thoughts]

Hey, this looks really cool.

Do you have any training performance with the environment? Could you add some rendered images of the environment to understand what the world looks like

[Near32]

I have not yet run any training on the environment, so no training performance to show yet.

Here is a screenshot of the environment when using the manual_control.py script (with the mission/instruction being printed out in the terminal stdout) :

[Near32]

Hello,

Still new to all of this, thus, may I inquire what would remains for me to do to make this pull request progress, please ?

[pseudo-rnd-thoughts]

Hello,

Still new to all of this, thus, may I inquire what would remains for me to do to make this pull request progress, please ?

Hi, sorry, we have been a bit slow
Also, could you run some training algorithm on the environment to actually see if it works and agents can learn on it. Then we should be good to merge

[pseudo-rnd-thoughts]

@Near32 Hey, have you had time to do any training on the environment yet?