Improve "Operator crashes on startup with OOMKilled" docs section

[ppf2]

Add an example of an OOMKilled event and share best practice on setting memory limits: recommend setting spec.containers[].resources.limits.memory and spec.containers[].resources.requests.memory to the same value.

cc: @kunisen

[barkbay]

May I ask your opinion to see if we should also explicitly set that for the operator too, please? @pebrc

For the JVM it is indeed a best practice because:

• The JVM sets the Heap size according to the limit, not the request
• Elasticsearch is started with -XX:+AlwaysPreTouch

Therefore, memory pages of an Elasticsearch Pod are immediately mapped, up to the limit. Said differently, if the request is lower than the limit then an Elasticsearch Pod may immediately be a candidate for eviction, a soon as it is started (update: AlwaysPreTouch only applies to the heap, not to the native memory. It still means that a significant amount of memory is already mapped at startup)

The situation is a bit different for the operator since it is not running in a virtual machine, with a fixed amount of memory mapped. There may be memory usage spikes, but memory may eventually be returned to the operating system. Hence, having different values might make sense. That being said, we can indeed remind the user that to avoid any disruption it is recommend to:

• set the request and the limit to a same value (or just set the limit without any request)
• monitor the RSS memory to adjust the allocated resources

[ppf2]

thx for the suggestions!

[barkbay]

LGTM

[alaudazzi]

Left a minor editing suggestion. Otherwise LGTM.