[KEP-2400] Node swap updates, GA criterias and clarifications

[iholder101]

• One-line PR description:
  Add updates, GA criterias and clarifications

• Issue link: Node memory swap support #2400

• Other comments:

This PR updates the KEP in the following ways:

Emphasize that this KEP is about basic swap enablement
The original KEP indicated that pod-level swap APIs are out of scope:

enhancements/keps/sig-node/2400-node-swap/README.md

Lines 163 to 166 in 155a949

	- Allocating swap on a per-workload basis with accounting (e.g. pod-level
	specification of swap). If desired, this should be designed and implemented
	as part of a follow-up KEP. This KEP is a prerequisite for that work. Hence,
	swap will be an overcommitted resource in the context of this KEP.

enhancements/keps/sig-node/2400-node-swap/README.md

Lines 142 to 144 in 155a949

	This KEP will be limited in scope to the first two scenarios. The third can be
	addressed in a follow-up KEP. The enablement work that is in scope for this KEP
	will be necessary to implement the third scenario.

However, the lack of APIs and the implicit nature of the current implementation sometimes brings suggestions to extend the API under this KEP.

This KEP focuses on a basic swap enablement. Follow-up KEPs regarding several topics (e.g. customization, zram/zswap suport, and more) will be introduced in the near future, in which we would be able to design and implement each extension in a focused way.

To ensure we're on the same page, this topic was recently raised in a sig-node meeting. In this meeting there was a very broad consensus that this approach makes sense, especially since the NodeSwap feature is important to many different parties which want it to "just work".

This PR updates the KEP to emphasize this approach.

GA criterias
The PR adds GA criterias, alongside the intent to GA in version 1.32.

Make sure PRR is ready

Updates
Since the last KEP updates, many improvements were made and many concerns were addressed. For example:

• Memory-backed volumes
• Added metrics
• Kubelet Configuration examples
• more

This PR updates the KEP to reflect these updates.

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: iholder101
Once this PR has been reviewed and has the lgtm label, please assign dchen1107 for approval. For more information see the Kubernetes Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

• keps/sig-node/OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[iholder101]

/cc @SergeyKanzhelev @haircommander @harche

[deads2k]

Please update https://github.com/kubernetes/enhancements/blob/master/keps/prod-readiness/sig-node/2400.yaml and update missing bits of the PRR questionaire.

[iholder101]

Thanks @deads2k!

Please update master/keps/prod-readiness/sig-node/2400.yaml

I see you're the assigned approver for alpha/beta.
Is it OK to also assign you as the approver for GA?

and update missing bits of the PRR questionaire.

Done! PTAL :)

[sftim]

/retitle [KEP-2400] Node swap ppdates, GA criterias and clarifications

[sftim]

D'oh

/retitle [KEP-2400] Node swap updates, GA criterias and clarifications

[sftim]

Thanks for the PR.

Here's a mix of feedback; I hope it is all useful.

[sftim]

Suggested change

	stable: "v1.32"
	#stable: "v1.32"

(AIUI)
We can put in a suggestion, but until we're planning that release, it's not a commitment. If we are planning that release, we should change latest-milestone and stage I think.

[iholder101]

I think that we should indeed plan it for next release.

The latest-milestone comments read:

# The most recent milestone for which work toward delivery of this KEP has been
# done. This can be the current (upcoming) milestone, if it is being actively
# worked on.

The milestone's documentation reads:

# The milestone at which this feature was, or is targeted to be, at each stage.

Since we target 1.32, miltstone seemed like the right place. I've added it now to latest-milestone.

What's important to me is that we indeed plan and commmit to releasing in 1.32, which there's a broad consensus of doing. I've even volunteered to not pursue GAing in 1.31 to reduce pressure from the sig-node approvers and ensure we do so in 1.32 with no pressure.

[aojea]

but it will mount it anyway, right?

[kannon92]

AFAIK kubelet will fail to mount if the kernel does not support this feature. So it won’t be mounted in the “correct” way due to a limitation of the kernel.

[iholder101]

but it will mount it anyway, right?

Not sure I understand, mount what?
This "dummy mount" will fail if the tmpfs noswap is not supported. It is basically a method to determine whether the noswap option is supported or not. If it is supported, it'll be used to mount memory-backed volumes. If not, a warning log entry will be raised, and memory-backed volumes will be mounted without the noswap option.

Hope this answers your question, let me know if you need more info

[iholder101]

@sftim @deads2k @haircommander @SergeyKanzhelev @mrunalp

Would you kindly have another look at this? Is this anything missing? I believe that most of this PR has already been agreed upon in previous sig-node calls. Thanks!

[jabdoa2]

We have been using this feature for a while in production and it generally performs well. We still need to set --experimental-allocatable-ignore-eviction on kubelet to allow setting memory eviction threshold to 0. Otherwise, kubelet might evict pods before swapping them out. With that option it has been pretty solid.

[haircommander]

/milestone 1.32

[k8s-ci-robot]

@haircommander: The provided milestone is not valid for this repository. Milestones in this repository: [v1.25, v1.27, v1.28, v1.29, v1.30, v1.31, v1.32, v1.33, v1.34]

Use /milestone clear to clear the milestone.

In response to this:

/milestone 1.32

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

[haircommander]

/milestone v1.32

[iholder101]

I wish to update that we agreed [1] that NFD support for swap brings enough debugability for this KEP. We'll definitely circle back to improving this as part of the follow-up API extension KEP. Thanks @kannon92 for bringing swap support to NFD, and thanks @dchen1107 for helping pushing the discussion forward!

With this, I believe that this PR is entirely ready to get in.
@sftim @deads2k @haircommander @SergeyKanzhelev @mrunalp - can you please have another look so we can push this forward? Thanks in advance!

[1] The discussion started at the sig-node meeting, then continued to this comment.