-
Install and launch minikube
-
Create a namespace to host the operator
kubectl create namespace baremetal-operator-system
-
Install cert-manager
kubectl apply -f https://github.com/jetstack/cert-manager/releases/download/v1.5.4/cert-manager.yaml
-
Verify that it is deployed correctly. Note: don't move to the next step before you see all three pods in Running state.
kubectl get pods --namespace cert-manager NAME READY STATUS RESTARTS AGE cert-manager-5c6866597-zw7kh 1/1 Running 0 2m cert-manager-cainjector-577f6d9fd7-tr77l 1/1 Running 0 2m cert-manager-webhook-787858fcdb-nlzsq 1/1 Running 0 2m
-
Install operator in the cluster
eval $(go env) mkdir -p $GOPATH/src/github.com/metal3-io cd $GOPATH/src/github.com/metal3-io git clone https://github.com/metal3-io/baremetal-operator.git cd baremetal-operator kustomize build config/default | kubectl apply -f -
-
OR Launch the operator locally
export OPERATOR_NAME=baremetal-operator export DEPLOY_KERNEL_URL=http://172.22.0.1/images/ironic-python-agent.kernel export DEPLOY_RAMDISK_URL=http://172.22.0.1/images/ironic-python-agent.initramfs export IRONIC_ENDPOINT=http://localhost:6385/v1/ make run
-
Create the CR
kubectl create namespace metal3 kubectl apply -f examples/example-host.yaml -n metal3
In environments where Ironic is not available, and the only real need
is to be able to have some test data, use the test fixture provisioner
instead of the real Ironic provisioner by passing -test-mode
to the
operator when launching it.
make run-test-mode
There is a script available that will run a set of containers locally using
podman
to stand up Ironic for development and testing.
See tools/run_local_ironic.sh
.
Note that this script may need customizations to some of the podman run
commands, to include environment variables that configure the containers for
your environment. All ironic related environment variables are set by default
if they are not passed through the environment.
The following environment variables can be passed to configure the ironic:
- HTTP_PORT - port used by httpd server (default 6180)
- PROVISIONING_IP - provisioning interface IP address to use for ironic, dnsmasq(dhcpd) and httpd (default 172.22.0.1)
- CLUSTER_PROVISIONING_IP - cluster provisioning interface IP address (default 172.22.0.2)
- PROVISIONING_INTERFACE - interface to use for ironic, dnsmasq(dhcpd) and httpd (default ironicendpoint)
- CLUSTER_DHCP_RANGE - dhcp range to use for provisioning (default 172.22.0.10-172.22.0.100)
- DEPLOY_KERNEL_URL - the URL of the kernel to deploy ironic-python-agent
- DEPLOY_RAMDISK_URL - the URL of the ramdisk to deploy ironic-python-agent
- IRONIC_ENDPOINT - the endpoint of the ironic
- CACHEURL - the URL of the cached images
- IRONIC_FAST_TRACK - whether to enable fast_track provisioning or not (default true)
- IRONIC_KERNEL_PARAMS - Kernel parameters to pass to IPA (default console=ttyS0)
- IRONIC_INSPECTOR_VLAN_INTERFACES - VLAN interfaces included in introspection,
all
- all VLANs on all interfaces, using LLDP information (default), interface - all VLANs on an interface, using LLDP information, interface.vlan - a particular VLAN interface, not using LLDP - IRONIC_BOOT_ISO_SOURCE - where the boot iso image will be served from, possible values are: local (default), to download the image, prepare it and serve it from the conductor; http, to serve it directly from its HTTP URL.
- IPA_DOWNLOAD_ENABLED - Enables the use of the Ironic Python Agent Downloader container to download IPA archive (default true)
- USE_LOCAL_IPA - Enables the use of locally supplied IPA archive. This condition is handled by BMO and this has effect only when IPA_DOWNLOAD_ENABLED is "false", otherwise IPA_DOWNLOAD_ENABLED takes precedence. (default false)
- LOCAL_IPA_PATH - This has effect only when USE_LOCAL_IPA is set to "true",
points to the directory where
the IPA archive is located. This variable is handled by BMO. The variable
should contain an arbitrary path pointing to the
directory that contains the
ironic-python-agent.tar
.
NOTES
In case both IPA_DOWNLOAD_ENABLED
and USE_LOCAL_IPA
are set to true then
the IPA archive will be downloaded and the use of local IPA
archive will be skipped.
In case both IPA_DOWNLOAD_ENABLED
and USE_LOCAL_IPA
are set to false then
the IPA download/copy process won't start at all. Skipping the IPA
download/copy process could help in avoiding unwanted overwrites of the
"ironic-python-agent.tar" in case somebody
is testing some functionality where the unwanted overwrite of the archive
would cause issues.
In case you want to run the local ironic containers with TLS and basic authentication enabled, you also need to export the following variables:
- IRONIC_CACERT_FILE
- IRONIC_CERT_FILE
- IRONIC_KEY_FILE
- IRONIC_USERNAME
- IRONIC_PASSWORD
The names of these variables are self explanatory. TLS variables expect the path of the corresponding certificate/key file as their value. Basic authentication variables expect the corresponding value as string. Note that, these variables do not have any default value. So if they are not set, the ironic container will run with TLS and basic authentication disabled.
It is easy to use Tilt for BMO deployment. Once you have a local instance of Ironic running, just run
make tilt-up
and clean it with
make kind-reset
It is also possible to develop Baremetal Operator using Tilt with CAPM3. Please refer to the development setup guide of CAPM3 and specially the Baremetal Operator Integration
In case the standard Tilt setup does not work as expected, you can use the following alternative method:
Create a Kind Cluster:
kind create cluster --name bmo
Install Cert-Manager:
kubectl apply -f https://github.com/cert-manager/cert-manager/releases/download/v1.14.2/cert-manager.yaml
Launch Tilt:
tilt up
Virtinst, libvirt-clients, libvirt-daemon-system, and Virtualbmc are required to to create BareMetalHosts this way. The network and VBMC needed for making a BareMetalHosts can be initialized with
tools/bmh_test/run_local_bmh_test_setup.sh
When Tilt is up, it is possible to make BareMetalHosts by pressing a button in the Tilt localhost interface. This is currently only supported for Unix based systems. This button runs the content of file
tools/bmh_test/create_bmh.sh <NAME> <VBMC_PORT>
and adds the values given to the button as arguments. Controlplane host can be created with
tools/bmh_test/create_bmh.sh <NAME> <VBMC_PORT> <CONSUMER> <CONSUMER_NAMESPACE>
The network, VBMC, and virtual machines can be cleaned with
tools/bmh_test/clean_local_bmh_test_setup.sh
In order to use VMs as hosts, they need to be connected to
vbmc
and the bootMACAddress
field needs to be set to the MAC address of the
network interface that will PXE boot.
For example:
apiVersion: metal3.io/v1alpha1
kind: BareMetalHost
metadata:
name: worker-0
spec:
online: true
bmc:
address: libvirt://192.168.122.1:6233/
credentialsName: worker-0-bmc-secret
bootMACAddress: 00:73:49:3a:76:8e
The make-virt-host
utility can be used to generate a YAML file for
registering a host. It takes as input the name of the virsh
domain
and produces as output the basic YAML to register that host properly,
with the boot MAC address and BMC address filled in.
$ go run cmd/make-virt-host/main.go openshift_worker_1
---
apiVersion: v1
kind: Secret
metadata:
name: openshift-worker-1-bmc-secret
type: Opaque
data:
username: YWRtaW4=
password: cGFzc3dvcmQ=
---
apiVersion: metal3.io/v1alpha1
kind: BareMetalHost
metadata:
name: openshift-worker-1
spec:
online: true
bmc:
address: libvirt://192.168.122.1:6234/
credentialsName: openshift-worker-1-bmc-secret
bootMACAddress: 00:1a:74:74:e5:cf
The output can be passed directly to kubectl apply
like this:
go run cmd/make-virt-host/main.go openshift_worker_1 | kubectl apply -f -
When the host is a controlplane host, include the -consumer
and
-consumer-namespace
options to associate the host with the existing
Machine
object.
$ go run cmd/make-virt-host/main.go -consumer ostest-controlplane-1 \
-consumer-namespace openshift-machine-api openshift_controlplane_1
---
apiVersion: v1
kind: Secret
metadata:
name: openshift-controlplane-1-bmc-secret
type: Opaque
data:
username: YWRtaW4=
password: cGFzc3dvcmQ=
---
apiVersion: metal3.io/v1alpha1
kind: BareMetalHost
metadata:
name: openshift-controlplane-1
spec:
online: true
bmc:
address: libvirt://192.168.122.1:6231/
credentialsName: openshift-controlplane-1-bmc-secret
bootMACAddress: 00:c9:a0:f2:e0:59
consumerRef:
name: ostest-controlplane-1
namespace: openshift-machine-api
The make-bm-worker
tool may be a more convenient way of creating
YAML definitions for workers than editing the files directly. If
deploying baremetal hosts you might want to consider setting
IRONIC_KERNEL_PARAMS="", when deploying ironic as the default directs
the console to ttyS0 and is lost in Bare Metal Hosts.
$ go run cmd/make-bm-worker/main.go -address 1.2.3.4 \
-password password -user admin worker-99
---
apiVersion: v1
kind: Secret
metadata:
name: worker-99-bmc-secret
type: Opaque
data:
username: YWRtaW4=
password: cGFzc3dvcmQ=
---
apiVersion: metal3.io/v1alpha1
kind: BareMetalHost
metadata:
name: worker-99
spec:
online: true
bmc:
address: 1.2.3.4
credentialsName: worker-99-bmc-secret
disableCertificateVerification: true