Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Revert global systemd unit timeout defaults #1450

Closed
zmrow opened this issue Apr 5, 2021 · 2 comments · Fixed by #1483
Closed

Revert global systemd unit timeout defaults #1450

zmrow opened this issue Apr 5, 2021 · 2 comments · Fixed by #1483
Assignees
Labels
type/enhancement New feature or request
Milestone

Comments

@zmrow
Copy link
Contributor

zmrow commented Apr 5, 2021

In this commit we created a default start/stop timeout for systemd units of 10 seconds. In the process of testing VMWare, we have found that kubelet requires more time to start. We have also observed journald crash looping as it needs more than 10 seconds to restart and recover a corrupted journal.

We should just remove the system-wide timeout and only enforce it where it is really necessary.

@tjkirch
Copy link
Contributor

tjkirch commented Apr 5, 2021

How confident are we that moving to 90s (the default) is better for most of our services? Services taking longer than 10s are rare in Bottlerocket. It seems like to do the correct thing without that default, we'd be adding much lower timeouts for many services. It seems simpler and safer to me to keep a lower default that's reasonable for most things and raise it in exceptional cases.

@jhaynes jhaynes added this to the next milestone Apr 5, 2021
@bcressey
Copy link
Contributor

bcressey commented Apr 5, 2021

We are partly addressing this with the default reservations for kubelet, but under heavy enough load pretty much any service could take a long time to start. We also want to avoid a death spiral where services are repeatedly started and killed because they're just over the limit, which adds additional resource pressure.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
type/enhancement New feature or request
Projects
None yet
Development

Successfully merging a pull request may close this issue.

5 participants