Feature/ Smac wrapper Update, MADQN/QMIX/VDN upgrades and Dockerfile improvements

[KaleabTessera]

What?

• Smac & Wrapper:
  • Switched to PZ SMAC wrapper (https://github.com/oxwhirl/smac/tree/master/smac/examples/pettingzoo).
  • Minor bug fixes to PZ core wrapper.
• MADQN/QMIX/VDN:
  • Updates and minor fixes to action selectors.
  • Moved epsilon decay out of trainers to executors and evaluators.
  • Added support for timestep based epsilon decay e.g. decay for 50000 time steps.
• All Systems:
  • Added optional LR decay and examples of usage.
• Other:
  • Redesigned dockerfiles (i.e. dockerfile per env) & makefiles. @DriesSmit @arnupretorius @mmorris44 Please check the redesigned dockerfiles :)
  • Tested and updated versions of most packages.
  • Changed how github actions are done.

Why?

• Fixes and more features.

How?

Extra

[mmorris44]

Remaining problems:

• Robocup sudo not found -> fix by installing sudo in Dockerfile: RUN apt-get -y install sudo -> install now works, but run broken, can't find server file -> Robocup example itself is broken (Dries fixed in another PR), but Robocup build also broken
• SC2 working fine -> turned out to just be a couple of faulty downloads on my side

[mmorris44]

Error during robocup install that gets ignored, probably causing the issues:

#15 264.1 libtool: link: g++ -std=c++14 -W -Wall -g -O2 -o .libs/rcssserver audio.o bodysender.o coach.o csvsaver.o
 dispsender.o field.o fullstatesender.o heteroplayer.o initsender.o initsendercoach.o initsenderlogger.o initsender
monitor.o initsenderonlinecoach.o initsenderplayer.o landmarkreader.o logger.o main.o monitor.o pcombuilder.o pcomp
arser.o player.o playerparam.o object.o referee.o remoteclient.o resultsaver.o serializer.o serializercoachstdv1.o 
serializercoachstdv7.o serializercoachstdv8.o serializercoachstdv13.o serializercoachstdv14.o serializercommonstdv1
.o serializercommonstdv7.o serializercommonstdv8.o serializermonitor.o serializeronlinecoachstdv1.o serializeronlin
ecoachstdv6.o serializeronlinecoachstdv7.o serializeronlinecoachstdv8.o serializeronlinecoachstdv13.o serializeronl
inecoachstdv14.o serializerplayerstdv1.o serializerplayerstdv7.o serializerplayerstdv8.o serializerplayerstdv13.o s
erializerplayerstdv14.o serverparam.o stadium.o stdoutsaver.o stdtimer.o synctimer.o team.o utility.o visualsenderc
oach.o visualsenderplayer.o weather.o xmlreader.o xpmholder.o player_command_parser.o player_command_tok.o  -L../rc
ssbase/conf -L../rcssbase/net -L../rcssbase/gzip -L/usr/lib/x86_64-linux-gnu -lrcssclangparser /home/app/mava/rcsss
erver-rcssserver-16.0.0/rcssbase/conf/.libs/librcssconfparser.so /home/app/mava/rcssserver-rcssserver-16.0.0/rcssba
se/net/.libs/librcssnet.so /home/app/mava/rcssserver-rcssserver-16.0.0/rcssbase/gzip/.libs/librcssgz.so -lboost_fil
esystem -lboost_system -lz -lm
#15 264.3 /usr/bin/ld: cannot find -lrcssclangparser
#15 264.3 collect2: error: ld returned 1 exit status
#15 264.3 make[3]: *** [Makefile:913: rcssserver] Error 1
#15 264.3 make[3]: Leaving directory '/home/app/mava/rcssserver-rcssserver-16.0.0/src'
#15 264.3 make[2]: *** [Makefile:778: all] Error 2
#15 264.3 make[2]: Leaving directory '/home/app/mava/rcssserver-rcssserver-16.0.0/src'
#15 264.3 make[1]: *** [Makefile:439: all-recursive] Error 1
#15 264.3 make[1]: Leaving directory '/home/app/mava/rcssserver-rcssserver-16.0.0'
#15 264.3 make: *** [Makefile:371: all] Error 2
#15 264.3 --2021-11-25 09:58:11--  https://github.com/rcsoccersim/rcssmonitor/archive/rcssmonitor-16.0.0.tar.gz
#15 264.3 Resolving github.com (github.com)... 140.82.121.4
#15 264.3 Connecting to github.com (github.com)|140.82.121.4|:443... connected.```

[mmorris44]

I assume the reason for this change is that the docker container doesn't need root access to run these commands. However, won't making this change cause problems for when people try to install robocup outside of docker?

[KaleabTessera]

Good point. I will re-add sudo to the install. 👍

[mmorris44]

Can't find any further things that I would change, but my Mava knowledge isn't good enough for this to be terribly meaningful.
Nevertheless, great job @KaleabTessera!

[DriesSmit]

This PR looks good, thanks @KaleabTessera 🔥 A lot of changes in here. Did you run some regression tests on this PR?

[arnupretorius]

This is a big one @KaleabTessera! Looks good to me, a massive effort 🔥 Good that we had multiple iterations across sections of the code because it is a big PR. @mmorris44 also thanks for your detailed checks and testing some of the changes. In the future, we can perhaps try to keep the PRs smaller and just more frequent.

For reference - Kale-ab confirmed we have test runs for the new smac wrapper and the eps scheduling changes for MADQN/QMIX/VDN and the learning rate decay which all look good.