chore: port all non deprecated, non apiserver/controller-manager referenced features to versioned

[Jefftree]

What type of PR is this?

/kind cleanup

What this PR does / why we need it:

Continuation of #126878 and #125031. Migrate all features that are not deprecated and not redefined in other places like apiserver and controller manager (they have their own feature gates).

As of 8/29/2024, there are 150 unversioned features and 2 versioned features. After this PR, there will be 45 unversioned features and 107 versioned features. The cardinality is the same so we can guarantee that all features that existed before still exist after the PR.

Obviously this doesn't 100% guarantee that version information is correct, I've taken the version information in the comments from https://github.com/kubernetes/kubernetes/blob/master/pkg/features/kube_features.go#L28 and constructed it in versioned_feature_gates.go, preserving the current comments like // remove after 1.33 as much as possible. These were manually cross referenced by both me and aaron-prindle.

Before

kubernetes ❯ ./hack/verify-featuregates.sh 
found 150 features in FeatureSpecMap var defaultKubernetesFeatureGates in file: /usr/local/google/home/jying/gospace/src/k8s.io/kubernetes/pkg/features/kube_features.go
...
found 2 features in FeatureSpecMap var defaultVersionedKubernetesFeatureGates in file: /usr/local/google/home/jying/gospace/src/k8s.io/kubernetes/pkg/features/versioned_kube_features.go

After:

kubernetes ❯ ./hack/verify-featuregates.sh
found 45 features in FeatureSpecMap var defaultKubernetesFeatureGates in file: /usr/local/google/home/jying/gospace/src/k8s.io/kubernete
s/pkg/features/kube_features.go
...
found 107 features in FeatureSpecMap var defaultVersionedKubernetesFeatureGates in file: /usr/local/google/home/jying/gospace/src/k8s.io/kubernetes/pkg/features/versioned_kube_features.go

Which issue(s) this PR fixes:

Fixes #

Special notes for your reviewer:

Follow ups:

• CSIMigrationPortworx needs a beta switch in the website

Does this PR introduce a user-facing change?

NONE

Additional documentation e.g., KEPs (Kubernetes Enhancement Proposals), usage docs, etc.:

[k8s-ci-robot]

This issue is currently awaiting triage.

If a SIG or subproject determines this is a relevant issue, they will accept it by applying the triage/accepted label and provide further guidance.

The triage/accepted label can be added by org members by writing /triage accepted in a comment.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

[Jefftree]

/retest

[Jefftree]

/retest

[aaron-prindle]

LegacyServiceAccountTokenCleanUp doesn't seem to be ported over when it possibly should be IIUC. It has this comment:

// GA in 1.30; remove in 1.32

but I believe you ported over other features with this comment and it doesn't fit into the groups to omit IIUC. Just wanted to raise this, not sure if it should be ported over atm

[aaron-prindle]

This has the following comment in kube_feautures.go

	// owner: @danielvegamyhre
	// kep: https://kep.k8s.io/2413
	// beta: v1.27
	//
	// Allows mutating spec.completions for Indexed job when done in tandem with
	// spec.parallelism. Specifically, spec.completions is mutable iff spec.completions
	// equals to spec.parallelism before and after the update.
	ElasticIndexedJob featuregate.Feature = "ElasticIndexedJob"

This says it is in beta but perhaps the comment is incorrect, just wanted to raise this in case

[Jefftree]

be410c0#diff-71e3b98f9a6bbf5b8421e26a7ba0c079f397cd8d49abacdad943c66a4f44f03d

Graduation is here, thanks for catching this will update the comment

[aaron-prindle]

The code here says beta: v1.30 but the comment from kube_features.go says beta: 1.31. I think this may need to be changed.

	// owner: @munnerz
	// kep: http://kep.k8s.io/4193
	// alpha: v1.29
	// beta: v1.31
	//
	// Controls whether the apiserver supports binding service account tokens to Node objects.
	ServiceAccountTokenNodeBinding featuregate.Feature = "ServiceAccountTokenNodeBinding"

[Jefftree]

ServiceAccountTokenNodeBinding looks to be correct at 1.31 https://github.com/kubernetes/kubernetes/blob/master/pkg/features/kube_features.go#L719. You're looking at ServiceAccountTokenJTI which is beta at 1.30 and that seems to match the comments https://github.com/kubernetes/kubernetes/blob/master/pkg/features/kube_features.go#L710

[aaron-prindle]

Ah sorry, +1 it is correct as is

[Jefftree]

LegacyServiceAccountTokenCleanUp doesn't seem to be ported over when it possibly should be IIUC. It has this comment:

// GA in 1.30; remove in 1.32

but I believe you ported over other features with this comment and it doesn't fit into the groups to omit IIUC. Just wanted to raise this, not sure if it should be ported over atm

#126839 removes the feature gate. IMO features that transitioned to GA in or before 1.31 should be safe to remove because they cannot be turned off both in 1.31 and future versions.

[pacoxu]

Generally, we use GA: v1.31.

A quick search shows that we have five stable: v1.x in this file.

[Jefftree]

Renamed all to GA

[aaron-prindle]

I believe this PR has addressed most of the deltas discovered in the below gist in a go through of kube_features.go vs kubernetes/website vs actual k8s releases from discussion w/ @Jefftree offline. My findings from my going through of all of the features in kube_features.go and comparing to the values in kubernets/website (and direct k8s releases) are here:
https://gist.github.com/aaron-prindle/8a56b33f70f7275a142d618efacea779

Going to create an issue + PR against kubernetes/website to fix the deltas I discovered there.

I believe the only outstanding item from this for this PR is changing StatefulSetAutoDeletePVC from alpha: 1.22 -> alpha: 1.23

(feature was added and then removed in 1.22 - fb5b966)

[aaron-prindle]

/lgtm

[k8s-ci-robot]

@aaron-prindle: changing LGTM is restricted to collaborators

In response to this:

/lgtm

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

[jpbetz]

/lgtm

[k8s-ci-robot]

LGTM label has been added.

Git tree hash: e922dc3d82bd568734258808484640835f2a8a37

[Jefftree]

/sig architecture
/hold cancel

[Jefftree]

/retest