Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

DNM: break kubelet to verify CI #1905

Closed
wants to merge 1 commit into from
Closed

DNM: break kubelet to verify CI #1905

wants to merge 1 commit into from

Conversation

jlebon
Copy link
Member

@jlebon jlebon commented Mar 5, 2024

This will sanity-check in a much more straight-forward way that the kubelet built from PRs is what's actually getting tested in CI.

See also
#1805 (comment) and following.

@openshift-ci-robot openshift-ci-robot added the backports/unvalidated-commits Indicates that not all commits come to merged upstream PRs. label Mar 5, 2024
@openshift-ci-robot
Copy link

@jlebon: the contents of this pull request could not be automatically validated.

The following commits could not be validated and must be approved by a top-level approver:

Comment /validate-backports to re-evaluate validity of the upstream PRs, for example when they are merged upstream.

@openshift-ci openshift-ci bot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Mar 5, 2024
@openshift-ci OpenShift CI
Copy link

openshift-ci bot commented Mar 5, 2024

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

@jlebon
Copy link
Member Author

jlebon commented Mar 5, 2024

/test ci/prow/e2e-gcp

@openshift-ci OpenShift CI
Copy link

openshift-ci bot commented Mar 5, 2024

@jlebon: The specified target(s) for /test were not found.
The following commands are available to trigger required jobs:

  • /test artifacts
  • /test configmap-scale
  • /test e2e-aws-crun-wasm
  • /test e2e-aws-jenkins
  • /test e2e-aws-ovn-cgroupsv2
  • /test e2e-aws-ovn-crun
  • /test e2e-aws-ovn-downgrade
  • /test e2e-aws-ovn-fips
  • /test e2e-aws-ovn-serial
  • /test e2e-aws-ovn-upgrade
  • /test e2e-azure-ovn-upgrade
  • /test e2e-gcp
  • /test e2e-gcp-ovn-upgrade
  • /test images
  • /test integration
  • /test k8s-e2e-conformance-aws
  • /test k8s-e2e-gcp-ovn
  • /test k8s-e2e-gcp-serial
  • /test unit
  • /test verify
  • /test verify-commits

The following commands are available to trigger optional jobs:

  • /test e2e-agnostic-ovn-cmd
  • /test e2e-aws
  • /test e2e-aws-csi
  • /test e2e-aws-disruptive
  • /test e2e-aws-multitenant
  • /test e2e-aws-ovn
  • /test e2e-aws-single-node
  • /test e2e-azure
  • /test e2e-metal-ipi
  • /test e2e-metal-ipi-ovn-dualstack
  • /test e2e-metal-ipi-ovn-ipv6
  • /test e2e-openstack
  • /test e2e-openstack-csi-cinder
  • /test e2e-openstack-csi-manila
  • /test e2e-vsphere
  • /test k8s-e2e-aws
  • /test k8s-e2e-aws-ovn-serial
  • /test k8s-e2e-gcp-five-control-plane-replicas

Use /test all to run the following jobs that were automatically triggered:

  • pull-ci-openshift-kubernetes-master-e2e-agnostic-ovn-cmd
  • pull-ci-openshift-kubernetes-master-e2e-aws-crun-wasm
  • pull-ci-openshift-kubernetes-master-e2e-aws-csi
  • pull-ci-openshift-kubernetes-master-e2e-aws-ovn-cgroupsv2
  • pull-ci-openshift-kubernetes-master-e2e-aws-ovn-crun
  • pull-ci-openshift-kubernetes-master-e2e-aws-ovn-fips
  • pull-ci-openshift-kubernetes-master-e2e-aws-ovn-serial
  • pull-ci-openshift-kubernetes-master-e2e-gcp
  • pull-ci-openshift-kubernetes-master-e2e-gcp-ovn-upgrade
  • pull-ci-openshift-kubernetes-master-images
  • pull-ci-openshift-kubernetes-master-integration
  • pull-ci-openshift-kubernetes-master-k8s-e2e-aws-ovn-serial
  • pull-ci-openshift-kubernetes-master-k8s-e2e-conformance-aws
  • pull-ci-openshift-kubernetes-master-k8s-e2e-gcp-ovn
  • pull-ci-openshift-kubernetes-master-k8s-e2e-gcp-serial
  • pull-ci-openshift-kubernetes-master-unit
  • pull-ci-openshift-kubernetes-master-verify
  • pull-ci-openshift-kubernetes-master-verify-commits

In response to this:

/test ci/prow/e2e-gcp

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@jlebon
Copy link
Member Author

jlebon commented Mar 5, 2024

/test e2e-gcp

@openshift-ci OpenShift CI
Copy link

openshift-ci bot commented Mar 5, 2024

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: jlebon
Once this PR has been reviewed and has the lgtm label, please assign soltysh for approval. For more information see the Kubernetes Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@jlebon
DNM: break kubelet to verify CI
6182578 
This will sanity-check in a much more straight-forward way that the
kubelet built from PRs is what's actually getting tested in CI.

See also
openshift#1805 (comment)
and following.
@openshift-ci-robot
Copy link

@jlebon: the contents of this pull request could not be automatically validated.

The following commits could not be validated and must be approved by a top-level approver:

Comment /validate-backports to re-evaluate validity of the upstream PRs, for example when they are merged upstream.

@jlebon
Copy link
Member Author

jlebon commented Mar 5, 2024

/test e2e-gcp

@jlebon
Copy link
Member Author

jlebon commented Mar 5, 2024

We can see in e.g. control-plane/10.0.0.3/journals/kubelet.log:

Mar 05 19:31:20 ci-op-h7qm5jnf-b2c4f-p5d6k-master-1 systemd[1]: Starting Kubernetes Kubelet...
Mar 05 19:31:20 ci-op-h7qm5jnf-b2c4f-p5d6k-master-1 systemd[1]: kubelet.service: Main process exited, code=exited, status=1/FAILURE
Mar 05 19:31:20 ci-op-h7qm5jnf-b2c4f-p5d6k-master-1 kubenswrapper[2227]: in your kubelet, breaking things
Mar 05 19:31:20 ci-op-h7qm5jnf-b2c4f-p5d6k-master-1 systemd[1]: kubelet.service: Failed with result 'exit-code'.
Mar 05 19:31:20 ci-op-h7qm5jnf-b2c4f-p5d6k-master-1 systemd[1]: Failed to start Kubernetes Kubelet.

So this looks good.

We'll redo this test once openshift/release#49156 and #1805 are in.

@jlebon jlebon mentioned this pull request Mar 5, 2024
Merged
@soltysh
Copy link

soltysh commented Mar 6, 2024

/test e2e-gcp

@openshift-ci OpenShift CI
Copy link

openshift-ci bot commented Mar 6, 2024

@jlebon: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name Commit Details Required Rerun command
ci/prow/e2e-gcp 6182578 link true /test e2e-gcp

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@jlebon
Copy link
Member Author

jlebon commented Mar 6, 2024 

Mar 06 20:09:58 ci-op-5n55s9xn-b2c4f-np95d-master-0 systemd[1]: kubelet.service: Scheduled restart job, restart counter is at 1.
Mar 06 20:09:58 ci-op-5n55s9xn-b2c4f-np95d-master-0 systemd[1]: Stopped Kubernetes Kubelet.
Mar 06 20:09:58 ci-op-5n55s9xn-b2c4f-np95d-master-0 systemd[1]: Starting Kubernetes Kubelet...
Mar 06 20:09:58 ci-op-5n55s9xn-b2c4f-np95d-master-0 kubenswrapper[2420]: in your kubelet, breaking things
Mar 06 20:09:58 ci-op-5n55s9xn-b2c4f-np95d-master-0 systemd[1]: kubelet.service: Main process exited, code=exited, status=1/FAILURE
Mar 06 20:09:58 ci-op-5n55s9xn-b2c4f-np95d-master-0 systemd[1]: kubelet.service: Failed with result 'exit-code'.
Mar 06 20:09:58 ci-op-5n55s9xn-b2c4f-np95d-master-0 systemd[1]: Failed to start Kubernetes Kubelet.

🎉

I think we can close this now.
Thanks for all your help @soltysh!

@jlebon jlebon closed this Mar 6, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
backports/unvalidated-commits Indicates that not all commits come to merged upstream PRs. do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@jlebon @openshift-ci-robot @soltysh