How to control (or at least speed up) time in the simulator?

[yannbouteiller]

Hello everyone,

So, I am experimenting with Reinforcement Learning strategies to develop a controller for the competition. Since this is very sample-intensive, I need to speed Airsim up as much as possible to generate as many samples as possible (for now I understand it only runs real-time?).

Is it possible to basically control time in airsimneurips, please?

This is an obvious requirement for making the engine more useful than the real world in training Machine Learning algorithms in general, and it seems people tried this in microsoft/AirSim#901 modifying the setting.json file, but I don't think we have access to this file?

Regards,
Yann.

[rmst]

Second that question. Also, how will time be handled in the competition? If it is "real-time" how do we deal with the fact that our controller that has potentially been trained on different hardware has to suddenly work on a new timescale? (A problem that would be unlikely to occur on a real drone as those usually use real-time operating systems with predicable and constant time stepping)

[msb336]

You can access your settings.json file inside of Documents/AirSim and modify the clock speed by adding the following line to your settings file:

  "ClockSpeed": {your desired clock speed},

Note that the default clock speed (1) will be used for both the qualifiers and the live competition. We will be releasing the hardware specs and estimated performance of the live competition machine in the near future.

[rmst]

Okay, cool, thanks. Having the hardware specs helps in the real-time case. It could still trip up the agent, in case the system needs to swap memory or the python garbage collector kicks in. I don't know how easy/hard it is to do in Unreal Engine but using proper stepping would be cleaner. By that I mean between each two observations (and actions) the exact same amount of game time passes.

[msb336]

Using a stepper clock is a good idea. I'll bring it up with AirSimNeurips team.

[yannbouteiller]

This part of the official airsim documentation is worrying on this matter, it implies that what is being simulated between two time steps is clockspeed-dependent and hardware-dependent?

This setting allows you to set the speed of simulation clock with respect to wall clock. For example, value of 5.0 would mean simulation clock has 5 seconds elapsed when wall clock has 1 second elapsed (i.e. simulation is running faster). The value of 0.1 means that simulation clock is 10X slower than wall clock. The value of 1 means simulation is running in real time. It is important to realize that quality of simulation may decrease as the simulation clock runs faster. You might see artifacts like object moving past obstacles because collision is not detected. However slowing down simulation clock (i.e. values < 1.0) generally improves the quality of simulation.

[patrickhart]

Is there any update/progress being made regarding this issue?

[yannbouteiller]

I have been struggling on this issue for quite a while now and I didn't come out with a satisfactory solution yet. I think it is overly difficult to get a deterministic control over time in Airsim and to speed it up without changing the actual simulation, which makes the simulator very impractical for Reinforcement Learning, sadly.

[SavitGupta]

"You might see artifacts like object moving past obstacles because collision is not detected." Is there any easy way of checking when this has occurred in a given simulation? I was trying to see what clock speed can my hardware reliably handle.

[yannbouteiller]

Actually I am trying to make something that is clockspeed-independent and hardware-independent at least in simulated time (not accounting for the 'quality' of the simulation) but this is very challenging.

I have implemented 2 ways of running the simulator for a should-be-constant amount of simulated time.

The first one is the following:

simPause(False)
(:/)
f = API_control_method(duration=simulated_time_step)
f.join()
(:/)
simPause(True)

However I am afraid things still happen in simulation where the (:/) are, and that it gets worse when we set a higher clockspeed, am I correct?

The second one is the following:

simPause(True)
f = API_control_method(duration=simulated_time_step)
simContinueForTime(therotical_cpu_time_step) (:/)
while not simIsPause():
pass

The problem here is that what actually happens in simContinueForTime() is very hardware dependent, in the sense that a simulation running with e.g. clockspeed=100.0 won't be running actually exactly 100 times faster than real-time, am I correct?

Could we have a way to make this predictable?

[madratman]

Hey everyone
I just updating the v0.3 training binaries for linux which should really pause everything when simPause(True) is called. I recommend using simPause(True) and simPause(False).
See microsoft/AirSim#2282 (comment) for details.
Note that the race wall clock will still be running when you pause things.

simContinueForTime seems redundant API design to me and is causing confusion. I am leaning on removing it. Effectively it's doing something on the lines of simPause(False), let things run for n seconds, then simPause(True). I'd rather client side code do things like this.

[yannbouteiller]

simContinueForTime seems redundant API design to me and is causing confusion. I am leaning on removing it. Effectively it's doing something on the lines of simPause(False), let things run for n seconds, then simPause(True). I'd rather client side code do things like this.

Really? I had the feeling that using f.join() and then simPause(True) had this problem of things keeping being simulated between the two calls (e.g. when we set a high clockspeed), which didn't seem to happen with simContinueForTime() ?

(Also what I actually do is use a duration for actions that is larger than the duration of simContinueForTime() so I don't fall in this regime where the drone has finished its action and falls back to the default action for a small amount of time)

[yannbouteiller]

Well, there are annoying problems with the new Linux binaries.

When using simPause(True), everything gets so much paused that it becomes impossible to use e.g. F10 to get the mouse back when it is caught in the window without opening a terminal to kill the simulator, and when using simContinueForTime() (which I do for the aforementioned reasons) the simulator seems to run but the visualization window stays frozen.

[madratman]

When using simPause(True), everything gets so much paused that it becomes impossible to use e.g. F10 to get the mouse back
Well, you can just press backtick to get the mouse back (unreal console commands still work when simPause(True) is called).

Looking into simContinueForTime

[madratman]

when using simContinueForTime() (which I do for the aforementioned reasons) the simulator seems to run but the visualization window stays frozen.
I don't see any frozen windows on my machine, by I fixed simContinueForTime to pause everything as it should with the updated simPause microsoft/AirSim#2299. Please re-download linux training binaries.