Timed out after 180.001s. waiting for cluster deletion timed out

[adilGhaffarDev]

Which jobs are flaking?

• periodic-cluster-api-e2e-mink8s-release-1-8
• periodic-cluster-api-e2e-main
• periodic-cluster-api-e2e-latestk8s-main
• periodic-cluster-api-e2e-release-1-8

Which tests are flaking?

• capi-e2e [It] When testing Cluster API working on self-hosted clusters using ClusterClass with a HA control plane [ClusterClass] Should pivot the bootstrap cluster to a self-hosted cluster
• capi-e2e [It] When testing Cluster API working on self-hosted clusters using ClusterClass [ClusterClass] Should pivot the bootstrap cluster to a self-hosted cluster
• capi-e2e [It] When testing ClusterClass changes [ClusterClass] Should successfully rollout the managed topology upon changes to the ClusterClass

Since when has it been flaking?

there were few before, more flakes after 6-9-2024

Testgrid link

https://prow.k8s.io/view/gs/kubernetes-jenkins/logs/periodic-cluster-api-e2e-release-1-8/1833344003316125696

https://storage.googleapis.com/k8s-triage/index.html?job=.*-cluster-api-.*&xjob=.*-provider-.*%7C.*-operator-.*#8899ccb732f9f0e048cb

Reason for failure (if possible)

MachinePool deletion is stuck

Anything else we need to know?

No response

Label(s) to be applied

/kind flake
One or more /area label. See https://github.com/kubernetes-sigs/cluster-api/labels?q=area for the list of labels.

[adilGhaffarDev]

cc @kubernetes-sigs/cluster-api-release-team

[sbueringer]

I took an initial look and the cluster deletion is stuck because MachinePools are stuck in deletion

cc @jackfrancis @willie-yao @Jont828
(feel free to cc other folks that are interested in help maintaining MachinePools)

[jackfrancis]

@sbueringer thx for triaging

[sbueringer]

Given how often this test fails and how long it has been failing, I think we should consider removing machinepools from the affected tests.

There's a high chance we are missing other issues in the rest of Cluster API because of this flake.

[willie-yao]

I can take a look at this if there are no other more urgent MP-related issues @sbueringer

[sbueringer]

I can't prioritize MP issues for MP maintainers. But from a CI stability perspective this one is really important.

I think either we can fix it soon, or we have to disable MPs for all affected tests. I just don't want to take the risk for much longer that this flake is hiding other issues.