Skip to content

Commit

Permalink
revert
Browse files Browse the repository at this point in the history
  • Loading branch information
eaudetcobello committed Sep 6, 2024
1 parent 0f2f9c5 commit 2550fe8
Show file tree
Hide file tree
Showing 14 changed files with 11 additions and 1,352 deletions.
9 changes: 2 additions & 7 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -92,15 +92,14 @@ KUSTOMIZE := $(TOOLS_BIN_DIR)/$(KUSTOMIZE_BIN)-$(KUSTOMIZE_VER)
# Ginkgo
TEST_DIR := $(shell pwd)/test
ARTIFACTS ?= $(shell pwd)/_artifacts
GINKGO_FOCUS ?= Workload cluster creation
GINKGO_FOCUS ?=
GINKGO_SKIP ?=
GINKGO_NODES ?= 1 # GINKGO_NODES is the number of parallel nodes to run
# when running the e2e tests, 1 means no parallelism
GINKGO_TIMEOUT ?= 2h
GINKGO_POLL_PROGRESS_AFTER ?= 60m
GINKGO_POLL_PROGRESS_INTERVAL ?= 5m
E2E_INFRA ?= aws
E2E_CONF_FILE ?= $(TEST_DIR)/e2e/config/ck8s-$(E2E_INFRA).yaml
E2E_CONF_FILE ?= $(TEST_DIR)/e2e/config/ck8s-docker.yaml
SKIP_RESOURCE_CLEANUP ?= false
USE_EXISTING_CLUSTER ?= false
GINKGO_NOCOLOR ?= false
Expand Down Expand Up @@ -408,7 +407,3 @@ $(CONTROLLER_GEN): ## Build controller-gen from tools folder.

$(CONVERSION_GEN): ## Build conversion-gen from tools folder.
GOBIN=$(TOOLS_BIN_DIR) $(GO_INSTALL) k8s.io/code-generator/cmd/conversion-gen $(CONVERSION_GEN_BIN) $(CONVERSION_GEN_VER)

.PHONY: nuke
nuke:
aws-nuke -c ~/.config/aws-nuke/nuke-config.yaml --no-dry-run
83 changes: 0 additions & 83 deletions test/e2e/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,89 +21,6 @@ To run a specific e2e test, such as `[PR-Blocking]`, use the `GINKGO_FOCUS` envi
make GINKGO_FOCUS="\\[PR-Blocking\\]" test-e2e # only run e2e test with `[PR-Blocking]` in its spec name
```

### Use an existing cluster as the management cluster

This is useful if you want to use a cluster managed by Tilt.

```shell
make USE_EXISTING_CLUSTER=true test-e2e
```

### Running the tests with Tilt

This section explains how to run the E2E tests on AWS using a management cluster run by Tilt.

This section assumes you have *kind* and *Docker* installed. (See [Prerequisites](https://cluster-api.sigs.k8s.io/developer/tilt#prerequisites))

First, clone the upstream cluster-api and cluster-api-provider-aws repositories.
```shell
git clone https://github.com/kubernetes-sigs/cluster-api.git
git clone https://github.com/kubernetes-sigs/cluster-api-provider-aws.git
```

Next, you need to create a `tilt-settings.yaml` file inside the `cluster-api` directory.
The kustomize_substitutions you provide here are automatically applied to the *management cluster*.
```shell
default_registry: "ghcr.io/canonical/cluster-api-k8s"
provider_repos:
- ../cluster-api-k8s
- ../cluster-api-provider-aws
enable_providers:
- aws
- ck8s-bootstrap
- ck8s-control-plane
kustomize_substitutions:
EXP_CLUSTER_RESOURCE_SET: "true" # todo(eac): revisit these exp. features
EXP_MACHINE_POOL: "true"
CAPA_EKS_IAM: "false"
CAPA_EKS_ADD_ROLES: "false"
AWS_B64ENCODED_CREDENTIALS: ""
```

Tilt will know how to run the aws provider controllers because the `cluster-api-provider-aws` repository has a `tilt-provider.json` file at it's root. Canonical Kubernetes also provides this file at the root of the repository. The CK8s provider names, ck8s-bootstrap and ck8s-control-plane, are defined in CK8's `tilt-provider.yaml` file.

Next, you have to customize the variables that will be substituted into the cluster templates applied by the tests (these are under `test/e2e/data/infrastructure-aws`). You can customize the variables in the `test/e2e/config/ck8s-aws.yaml` file under the `variables` key.

Finally, in one terminal, go into the `cluster-api` directory and run `make tilt-up`. You should see a kind cluster be created, and finally a message indicating that Tilt is available at a certain address.

In a second terminal in the `cluster-api-k8s` directory, run `make USE_EXISTING_CLUSTER=true test-e2e`.

### Cleaning up after an e2e test

The test framework tries it's best to cleanup resources after a test suite, but it is possible that
cloud resources are left over. This can be very problematic especially if you run the tests multiple times
while iterating on development (see [Cluster API Book - Tear down](https://cluster-api.sigs.k8s.io/developer/e2e#tear-down)).

You can use a tool like [aws-nuke](https://github.com/rebuy-de/aws-nuke) to cleanup your AWS account after a test. Here is a config. you can use that should cover most resources:

```yaml
regions:
- us-east-2

account-blocklist:
- "<your prod account number>"

accounts:
"<your account number>": {}

resource-types:
targets:
- EC2Instance
- EC2SecurityGroup
- EC2Volume
- EC2InternetGateway
- EC2NATGateway
- EC2RouteTable
- EC2Subnet
- EC2VPC
- EC2VPCEndpoint
- EC2VPCEndpointServiceConfiguration
- EC2ElasticIP
- EC2NetworkInterface
- ELBv2
- ELBv2TargetGroup
```
## Develop an e2e test

Refer to [Developing E2E tests](https://cluster-api.sigs.k8s.io/developer/e2e) for a complete guide for developing e2e tests.
Expand Down
5 changes: 2 additions & 3 deletions test/e2e/cluster_upgrade_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,6 @@ package e2e
import (
. "github.com/onsi/ginkgo/v2"
"k8s.io/utils/ptr"
"sigs.k8s.io/cluster-api/test/framework/clusterctl"
)

var _ = Describe("Workload cluster upgrade [CK8s-Upgrade]", func() {
Expand All @@ -39,7 +38,7 @@ var _ = Describe("Workload cluster upgrade [CK8s-Upgrade]", func() {
BootstrapClusterProxy: bootstrapClusterProxy,
ArtifactFolder: artifactFolder,
SkipCleanup: skipCleanup,
InfrastructureProvider: ptr.To(clusterctl.DefaultInfrastructureProvider),
InfrastructureProvider: ptr.To("docker"),
ControlPlaneMachineCount: ptr.To[int64](1),
WorkerMachineCount: ptr.To[int64](2),
}
Expand All @@ -54,7 +53,7 @@ var _ = Describe("Workload cluster upgrade [CK8s-Upgrade]", func() {
BootstrapClusterProxy: bootstrapClusterProxy,
ArtifactFolder: artifactFolder,
SkipCleanup: skipCleanup,
InfrastructureProvider: ptr.To(clusterctl.DefaultInfrastructureProvider),
InfrastructureProvider: ptr.To("docker"),
ControlPlaneMachineCount: ptr.To[int64](3),
WorkerMachineCount: ptr.To[int64](1),
}
Expand Down
132 changes: 0 additions & 132 deletions test/e2e/config/ck8s-aws.yaml

This file was deleted.

2 changes: 1 addition & 1 deletion test/e2e/create_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -48,7 +48,7 @@ var _ = Describe("Workload cluster creation", func() {
Expect(e2eConfig.Variables).To(HaveKey(KubernetesVersion))

clusterName = fmt.Sprintf("capick8s-create-%s", util.RandomString(6))
infrastructureProvider = clusterctl.DefaultInfrastructureProvider
infrastructureProvider = "docker"

// Setup a Namespace where to host objects for this spec and create a watcher for the namespace events.
namespace, cancelWatches = setupSpecNamespace(ctx, specName, bootstrapClusterProxy, artifactFolder)
Expand Down
Loading

0 comments on commit 2550fe8

Please sign in to comment.