CPE operator is a project that originally implements the AutoDECK framework. AutoDECK (Automated DEClarative Performance Evaluation and Tuning Framework on Kubernetes) is an evaluation system of Kubernetes-as-a-Service (KaaS) that automates configuring, deploying, evaluating, summarizing, and visualizing benchmarking workloads with a fully declarative manner.
System | Description | Installation guide |
---|---|---|
Prometheus Operator | for monitoring and visualization | read more |
Openshift Build Controller | for image tracking | read more |
Node Tuning Operator | for node tuning | read more |
Cloud Object Storaget (COS) | for job result logging | read more |
Clone the repo and enter the workspace
git clone https://github.com/IBM/cpe-operator.git
cd cpe-operator
Deploy with either of the following choices:
kubectl create -f ./config/samples/cpe-operator/default.yaml
- require
Prometheus Operator
to be deployed. - need to replace
openshift-monitoring
with the namespace that prometheus has been deployed for correcting RBAC resource.
kubectl create -f ./config/samples/cpe-operator/recommended.yaml
2.3.1. Set IMAGE_REGISTRY to your registry and update image in kustomization.yaml
export IMAGE_REGISTRY=[your registry URL]
export VERSION=[your image version tag]
** VERSION value must be specified as a valid semantic version for operator-sdk (Major.Minor.Patch)
2.3.2. Modify kustomization in /config
TAG | description | dependencies | to-modify kustomization | note |
---|---|---|---|---|
[PARSER] | Deploy CPE parser | - | default | Can specifiy PARSER_IMG environment for custom parser image |
[PROMETHEUS] | Deploy ServiceMonitor and RBAC | Prometheus Operator | default | May need to modify namespace label in manager.yaml and RBAC depending on Prometheus deployment |
[AUTO-TUNE] | Deploy tuning namespace and mounted to controller | Node Tuning Operator | default | |
[LOG-COS] | Set environment for COS secret | Cloud Object Storage secret (see /output) | default (and parser if [PARSER] enabled) |
2.3.3. Deploy custom manifests
# require operator-sdk (>= 1.4), go (>= 1.13)
make deploy
# confirm cpe operator is running
kubectl get po -n cpe-operator-system
# see manager log
kubectl logs $(kubectl get po -n cpe-operator-system|grep controller|tail -1|awk '{print $1}') -n cpe-operator-system -c manager
To remove this operator run:
make undeploy
See How to use the off-the-shelf/your own operator
Deploy simple coremark benchmark deployment:
kubectl create -f examples/none/cpe_coremark.yaml
Try benchmark with custom benchmark operator:
kubectl create -f examples/benchmark_operator/cpe_v1_benchmarkoperator_helm.yaml
# confirm ripsaw operator is running
kubectl get po -n my-ripsaw
kubectl create -f examples/benchmark_operator/cpe_v1_benchmark_iperf3.yaml
# confirm the job
kubectl get po -n my-ripsaw
- design custom resource; see benchmark_types.go, benchmarkoperator_types.go
- integrate to off-the-shelf benchmark operator; see benchmarks
- implement build tracker; see tracker
- raw output collector/parser; see output
- integrate wrapper from snafu
- iteration support; see iteration
- app-parameter variation (scenario)
- spec configuration
- node profile tuning
- visualize multi-cluster; see multi-cluster
- insert a sidecar if set
- combine resource usage metric; see metric
- prometheus-export metrics
- app-export metrics
- eBPF metric collector